6S: Distributing Crawling and Searching Across Web Peers

L-S. Wu, R. Akavipat, and F. Menczer (USA)

Keywords

Peer collaborative search, Web information retrieval

Abstract

A collaborative peer network application called 6Search (6S) is proposed to address the scalability limitations of centralized search engines. 6S peers depend on a lo cal adaptive routing algorithm to dynamically change the topology of the peer network and search for the best neigh bors to answer their queries. We validate prototypes of the 6S network via simulations with 70 − 500 model users based on actual Web crawls and find that the network topol ogy rapidly converges from a random network to a small world network, with clusters emerging from user commu nities with shared interests. We finally compare the quality of the results with those obtained by centralized search en gines such as Google, suggesting that 6S can draw advan tages from the context and coverage of the peer collective.

Important Links:



Go Back