An Empirical View on Consolidation of the Web

The majority of Web content is delivered by only a few companies that provide Content Delivery Infrastructures (CDIs) such as Content Delivery Networks (CDNs) and cloud hosts. Due to increasing concerns about trends of centralization, empirical studies on the extent and implications of resulting Internet consolidation are necessary. Thus, we present an empirical view on consolidation of the Web by leveraging datasets from two different measurement platforms. We first analyze Web consolidation around CDIs at the level of landing webpages, before narrowing down the analysis to a level of embedded page resources. The datasets cover 1(a) longitudinal measurements of DNS records for 166.5 M Web domains over five years, 1(b) measurements of DNS records for Alexa Top 1 M over a month and (2) measurements of page loads and renders for 4.3 M webpages, which include data on 392.3 M requested resources. We then define CDI penetration as the ratio of CDI-hosted objects to all measured objects, which we use to quantify consolidation around CDIs. We observe that CDI penetration has close to doubled since 2015, reaching a lower bound of 15% for all .com, .net, and .org Web domains as of January 2020. Overall, we find a set of six CDIs to deliver the majority of content across all datasets, with these six CDIs being responsible for more than 80% of all 221.9 M CDI-delivered resources (56.6% of all resources in total). We find high dependencies of Web content on a small group of CDIs, in particular, for fonts, ads, and trackers, as well as JavaScript resources such as jQuery. We further observe CDIs to play important roles in rolling out IPv6 and TLS 1.3 support. Overall, these observations indicate a potential oligopoly, which brings both benefits but also risks to the future of the Web.