An Analysis of Internet Data Update Behaviors

The growing application of caching in Internet applications have heretofore relied largely on qualitative observation and empirical data on the update behavior of Internet data in their design. While it is empirically known that the update behavior of such data is distinctly bimodal, much less is known about the details of these behaviors and the processes that drives them. A detailed study of the composition of modern web sites that includes in-depth analyses of the changes made to the data of which those sites are composed offers a large potential benefit not only to Internet caching protocol developers but also to a broad cross-section of information technology professionals and researchers. The first results of just such a study are reported in this paper. A variety of popular Internet web sites were selected and monitored for changes in both composition and content over a period of several weeks. Data collected in this study is then analyzed using traditional stochastic methods. The results of this investigation are summarized and suggested research directions conclude this article.