Assessing academic impacts of machine learning applications on a social science: Bibliometric evidence from economics

Machine learning (ML) methods have recently been applied in diverse fields of study. ML methods provide new toolkits and opportunities for social sciences, but they have also raised concerns with their black-box nature, irreproducibility, and emphasis on prediction rather than explanation. Against this backdrop, we study the bibliometric impact of leveraging ML methods in economics using publications indexed in Microsoft Academic Graph. We use our four-dimensional bibliometric framework by which we gage citation intensity, speed, breadth, and disruption to compare two groups of publications in economics (2001-2020)-those using ML methods and others not. We find that economics papers applying ML methods started to have advantages in citation counts and speed after 2010. Our analysis also shows that they received attention from more diverse research communities and had more disruptive citations over the past two decades. Then, we demonstrate that economics papers using ML methods obtained more disruptive citations within economics than outside. These findings suggest bibliometric advantages for applying ML methods in economics, especially in the recent decade, but we also discuss cautions and potential opportunities missed.