Towards Web-scale Web Archaeology

Web-scale Web research is difficult. Information on the Web is vast in quantity,unorganized and uncatalogued, and available only over a network with varyingreliability. Thus, Web data is difficult to collect, to store, and to manipulate efficiently.Despite these difficulties, we believe performing Web research at Web-scale isimportant. We have built a suite of tools that allow us to experiment on collectionsthat are an order of magnitude or more larger than are typically cited in the...