TY - GEN

T1 - The Dawn of Today's Popular Domains

T2 - 16th ACM/IEEE-CS Joint Conference on Digital Libraries, JCDL 2016

AU - Holzmann, Helge

AU - Nejdl, Wolfgang

AU - Anand, Avishek

PY - 2016

Y1 - 2016

N2 - The Web has been around and maturing for 25 years. The popular websites of today have undergone vast changes during this period, with a few being there almost since the beginning and many new ones becoming popular over the years. This makes it worthwhile to take a look at how these sites have evolved and what they might tell us about the future of the Web. We therefore embarked on a longitudinal study spanning almost the whole period of the Web, based on data collected by the Internet Archive starting in 1996, to retrospectively analyze how the popular Web as of now has evolved over the past 18 years. For our study we focused on the German Web, specifically on the top 100 most popular websites in 17 categories. This paper presents a selection of the most interesting findings in terms of volume, size as well as age of the Web. While related work in the field of Web Dynamics has mainly focused on change rates and analyzed datasets spanning less than a year, we looked at the evolution of websites over 18 years. We found that around 70% of the pages we investigated are younger than a year, with an observed exponential growth in age as well as in size up to now. If this growth rate continues, the number of pages from the popular domains will almost double in the next two years. In addition, we give insights into our data set, provided by the Internet Archive, which hosts the largest and most complete Web archive as of today.

AB - The Web has been around and maturing for 25 years. The popular websites of today have undergone vast changes during this period, with a few being there almost since the beginning and many new ones becoming popular over the years. This makes it worthwhile to take a look at how these sites have evolved and what they might tell us about the future of the Web. We therefore embarked on a longitudinal study spanning almost the whole period of the Web, based on data collected by the Internet Archive starting in 1996, to retrospectively analyze how the popular Web as of now has evolved over the past 18 years. For our study we focused on the German Web, specifically on the top 100 most popular websites in 17 categories. This paper presents a selection of the most interesting findings in terms of volume, size as well as age of the Web. While related work in the field of Web Dynamics has mainly focused on change rates and analyzed datasets spanning less than a year, we looked at the evolution of websites over 18 years. We found that around 70% of the pages we investigated are younger than a year, with an observed exponential growth in age as well as in size up to now. If this growth rate continues, the number of pages from the popular domains will almost double in the next two years. In addition, we give insights into our data set, provided by the Internet Archive, which hosts the largest and most complete Web archive as of today.

KW - Analysis

KW - Longitudinal

KW - Retrospective

KW - Statistics

KW - Web Dynamics

UR - http://www.scopus.com/inward/record.url?scp=84989902968&partnerID=8YFLogxK

U2 - 10.1145/2910896.2910901

DO - 10.1145/2910896.2910901

M3 - Conference contribution

AN - SCOPUS:84989902968

T3 - Proceedings of the ACM/IEEE Joint Conference on Digital Libraries

SP - 73

EP - 82

BT - JCDL 2016 - Proceedings of the 16th ACM/IEEE-CS Joint Conference on Digital Libraries

PB - Institute of Electrical and Electronics Engineers Inc.

Y2 - 19 June 2016 through 23 June 2016

ER -