The era of the internet has been a boon for empirical and evidence-based research. By providing ever increasing amounts of data, the internet offers numerous opportunities for new empirical studies. While some research questions require data that was previously more time-consuming to collect, other data was simply not available before the creation of the internet. However, publicly available information is still often unstructured and its collection can be highly resource-intensive. In this paper we present DataGorri, a software enabling the user-friendly and automated collection of repetitive and non-repetitive tabular data that is freely available on websites. This paper depicts the motivation underlying the software’s creation, describes its usage, and discusses its advantages and limitations.
Software DataGorri Web scraper Data scraper Crawler Data collection
This is a preview of subscription content, log in to check access.
We would like to thank everyone who has contributed to current or previous versions of DataGorri: Ivaylo Dimitrov, Matthias Franze, Stefan Hentschel, Lukas Holzner, Florian Kreitmair, Daniel Krieger, Michael Legenc, and Marc Müller. A list of DataGorri’s developers and contributors can also be found at https://www.julianhackinger.com/software/datagorri/. Furthermore, we thank Christian Feilcke and Miriam Leidinger, and two anonymous reviewers for comments, and Alexander Schlimm for research assistance.
Abramo, G., Cicero, T., D’Angelo, C.A. (2012). Revisiting size effects in higher education research productivity. Higher Education, 63(6), 701–717.CrossRefGoogle Scholar
Edelman, B. (2012). Using internet data for economic research. Journal of Economic Perspectives, 26(2), 189–206.CrossRefGoogle Scholar
Einav, L., & Levin, J. (2014a). The data revolution and economic analysis. Innovation Policy and the Economy, 14(1), 1–24.CrossRefGoogle Scholar
Einav, L., & Levin, J. (2014b). Economics in the age of big data. Science, 346(6210), 1243089.CrossRefGoogle Scholar
Faria, J.R., & Goel, R.K. (2010). Returns to networking in academia. Netnomics, 11(2), 103–117.CrossRefGoogle Scholar
Golden, J., & Carstensen, F.V. (1992a). Academic research productivity, department size and organization: Further results, comment. Economics of Education Review, 11(2), 153–160.CrossRefGoogle Scholar
Golden, J., & Carstensen, F.V. (1992b). Academic research productivity, department size and organization: Further results, rejoinder. Economics of Education Review, 11(2), 169–171.CrossRefGoogle Scholar
Hamermesh, D.S. (2013). Six decades of top economics publishing: Who and how? Journal of Economic Literature, 51(1), 162–172.CrossRefGoogle Scholar
Jordan, J.M., Meador, M., Walters, S.J. (1988). Effects of department size and organization on the research productivity of academic economists. Economics of Education Review, 7(2), 251–255.CrossRefGoogle Scholar
Jordan, J.M., Meador, M., Walters, S.J. (1989). Academic research productivity, department size and organization: Further results. Economics of Education Review, 8(4), 345–352.CrossRefGoogle Scholar
Meador, M., Walters, S.J., Jordan, J.M. (1992). Academic research productivity: Reply, still further results. Economics of Education Review, 11(2), 161–167.CrossRefGoogle Scholar