OCLC researchers measure the World Wide Web
DUBLIN, Ohio, Oct. 16, 2000--In their annual review of the World Wide Web, researchers at OCLC have determined that the Web now contains about 7 million unique sites; that the public Web-sites that offer content that is freely accessible by the general public-constitutes about 40 percent of the total Web; and that the Web continues to expand at a rapid pace, but its rate of growth is diminishing over time.
According to the group's latest estimates, there were 7.1 million unique web sites, a 50 percent increase over the previous year's total of 4.7 million. Although the number of web sites has nearly tripled in size in the last two years, year-to-year growth rates are declining, falling from almost 80 percent between 1998 and 1999, to only about 50 percent between 1999 and 2000.
Public web sites constitute 41 percent of the Web, or about 2.9 million sites. Private sites- whose content is subject to explicit access restrictions (e.g., Internet Protocol filters or password authentication), or is not intended for public use (e.g., web interfaces to privately owned hardware devices such as printers or routers)-comprise 21 percent of the Web, or 1.5 million sites. The remaining 2.7 million sites-or about 38 percent of the Web-are provisional sites: their content is in an unfinished or transitory state (e.g., server default pages or "Site under construction" notices).
Adult sites-those offering sexually explicit content-now constitute about 2 percent of the public Web, or 70,000 sites. The proportion of the public Web occupied by adult sites has remained unchanged since 1998.
"The Web continues to grow at a substantial rate," said Ed O'Neill, manager of the OCLC Web Characterization Project. "But a comparison of the year-to-year growth rates suggests that the Web's expansion is slowing. This trend is even more pronounced in the public Web, which grew by about 80 percent between 1997 and 1998 but only by about a third between 1999 and 2000. Even in absolute terms, growth seems to be slowing: the public Web increased by 713,000 sites in the past year, compared to 772,000 sites between 1998 and 1999."
Brian Lavoie, a research scientist working on the Web Characterization Project, notes the increasing incidence of non-public web content. "For most people, the Web is the public Web-that's where most web browsing takes place. But there's a lot of content out there that you would probably never encounter in the course of casual browsing; in other words, the private and provisional sites. Private sites in particular have exhibited steady growth relative to public sites in the past few years, accounting for about 12 percent of the Web two years ago, compared to over 20 percent today."
The Web Characterization Project, conducted by the OCLC Office of Research, has collected a random sample of web sites annually since 1997. Current results are based on analysis of the June 2000 sample. For analytical purposes, a web site is defined as content accessible through the HTTP protocol at a given location on the Internet.
More information on the Web Characterization Project is on the project web site < http://wcp.oclc.org/>.