EFFICIENT BANDWIDTH UTILIZATION FOR DOWNLOADING WEB PAGES

Anirban Kundu

References

  1. [1] How to accelerate your internet, 1st ed. (BMO Book Sprint Team: INASP/ICTP, ISBN: 0-9778093-1-5, October 2006).
  2. [2] B. Pinkerton, Finding what people want: Experiences with the WebCrawler, Proc. First World Wide Web Conference,Geneva, Switzerland, 1994.
  3. [3] P. Boldi, B. Codenotti, M. Santini, and S. Vigna, Ubicrawler: A scalable fully distributed web crawler, Proc. Eighth Australian World Wide Web Conference, 2002.
  4. [4] C. Castillo, M. Marin, and A. Rodriguez, Scheduling Algorithm for web crawling, Proc. Web Media & LA-Web 2004: Joint Conference of 10th Brazilian Symposium on Multimedia and the Web and 2nd Latin American Web Congress, 2004.
  5. [5] A. Arasu et al., Searching the Web, ACM Transactions onInternet Technology, 1(1), August 2001.
  6. [6] B.D. Davison, Topical locality in the Web, Proc. 23rdAnnual Int. Conf. on Research and Development in Infor-mation Retrieval (SIGIR 2000), ACM, Athens, Greece, July2000, 272–279.
  7. [7] S. Brin and L. Page, The anatomy of a large-scale hypertextual Web search engine, Proc. 7th Int. World Wide Web Conference, Brisbane, Australia, April 1998.
  8. [8] S. Chakrabarti et al., Mining the Web’s link structure, IEEE Computer, 32(8), 1999, 60–67.
  9. [9] J. Cho, H. Garcia-Molina, and L. Page, Efficient crawlingthrough URL ordering, Proc. 7th Int. World Wide Web Con-ference, 1998.
  10. [10] V. Shkapenyuk and T. Suel, Design and implementation of a high-performance distributed web crawler, 18th Int. Conf. on Data Engineering, IEEE, 2002.
  11. [11] http://en.wikipedia.org/wiki/Crawling
  12. [12] M. Kobayashi and K. Takeda, Information retrieval on the web, ACM Computing Surveys, 32(2), 2000, 144–173.
  13. [13] K.M. Risvik and R. Michelsen, Search engines and Web dynamics, Computer Networks, 39, June 2002.
  14. [14] A. Kundu, R. Dutta, R. Dattagupta, and D. Mukhopadhyay, Mining the Web with hierarchical crawlers – a resource sharing based crawling approach, International Journal of Intelligent Information and Database Systems, 3(1), 2009.
  15. [15] A. Kundu, R. Dutta, D. Mukhopadhyay, and Y. Kim, Ahierarchical Web page crawler for crawling the Internet faster, IEEE/CBNU/IEEK International Conference on Electronics & Information Technology Convergence, Republic of Korea, December 8, 2006.
  16. [16] A. Kundu et al., An alternate downloading methodology of Web-pages, 7th Mexican Int. Conf. on Artificial Intelligence, Mexico, IEEE CS, October 27–31, 2008.

Important Links:

Go Back