Cho, J. and Garcia-Molina, H. and Page, L. (1998) Efficient Crawling Through URL Ordering. In: Seventh International World-Wide Web Conference (WWW 1998), April 14-18, 1998, Brisbane, Australia.
Preview
PDFIn this paper we study in what order a crawler should visit the URLs it has seen, in order to obtain more "important" pages first. Obtaining important pages rapidly can be very useful when a crawler cannot visit the entire Web in a reasonable amount of time. We define several importance metrics, ordering schemes, and performance evaluation measures for this problem. We also experimentally evaluate the ordering schemes on the Stanford University Web. Our results show that a crawler with a good ordering scheme can obtain important pages significantly faster than one without.
Item Type: Conference or Workshop Item (Paper) Uncontrolled Keywords: crawling, crawler, URL ordering, archive Subjects: Computer Science > Databases and the Web Projects: Digital Libraries Related URLs: Project Homepage http://www-diglib.stanford.edu/diglib/pub/ ID Code: 347 Deposited By: Import Account Deposited On: 25 Feb 2000 16:00 Last Modified: 29 Dec 2008 10:29 Download statisticsRepository Staff Only: item control page
RetroSearch is an open source project built by @garambo | Open a GitHub Issue
Search and Browse the WWW like it's 1997 | Search results from DuckDuckGo
HTML:
3.2
| Encoding:
UTF-8
| Version:
0.7.4