Downloads: 136
Research Paper | Computer Science & Engineering | India | Volume 1 Issue 1, November 2012
Faster and Resourceful Multi-core Web Crawling
Arun Kumar Dewangan | Asha Ambhaikar [10]
Abstract: Due to massive growth of World Wide Web, search engines have become crucial tools for navigation of web pages. In order to provide fast and powerful search facility, search engine maintains indexes for documents and its contents on the web by downloading web pages for processing in iterative manner. Web indexes are created and managed by web crawlers which work as a module of search engines and traverse the web in systematic manner for indexing its contents. A web crawling is a process which fetches data from various servers. It is a time taking process as it gathers data from various servers. Hence, to speed up the searching process in search engines the crawling should be fast enough. The aim of the proposed system is to enhance the speed of crawling process and CPU usage by use of multi-core concept.
Keywords: Web crawling, multi core, indexing, parallel crawler, CPU, URL
Edition: Volume 1 Issue 1, November 2012,
Pages: 12 - 15
Similar Articles with Keyword 'Web crawling'
Downloads: 106
Review Papers, Computer Science & Engineering, India, Volume 4 Issue 12, December 2015
Pages: 2212 - 2215Focused and Adaptive Crawling for Topic Specific and Hidden Web Entries
Vrutuja Pande | Pratap Singh [59]
Downloads: 116
Research Paper, Computer Science & Engineering, India, Volume 3 Issue 7, July 2014
Pages: 610 - 612Reducing Duplicate Content Using XXHASH Algorithm
Rahul Mahajan | Dr. S. K. Gupta | Rajeev Bedi