International Journal of Science and Research (IJSR)

International Journal of Science and Research (IJSR)
Call for Papers | Fully Refereed | Open Access | Double Blind Peer Reviewed

ISSN: 2319-7064

Downloads: 136 | Views: 210

Research Paper | Computer Science & Engineering | India | Volume 1 Issue 2, November 2012 | Rating: 6.2 / 10

Faster and Resourceful Multi-core Web Crawling

Arun Kumar Dewangan | Asha Ambhaikar [10]

Abstract: Due to massive growth of World Wide Web, search engines have become crucial tools for navigation of web pages. In order to provide fast and powerful search facility, search engine maintains indexes for documents and its contents on the web by downloading web pages for processing in iterative manner. Web indexes are created and managed by web crawlers which work as a module of search engines and traverse the web in systematic manner for indexing its contents. A web crawling is a process which fetches data from various servers. It is a time taking process as it gathers data from various servers. Hence, to speed up the searching process in search engines the crawling should be fast enough. The aim of the proposed system is to enhance the speed of crawling process and CPU usage by use of multi-core concept.

Keywords: Web crawling, multi core, indexing, parallel crawler, CPU, URL

Edition: Volume 1 Issue 2, November 2012,

Pages: 12 - 15

How to Download this Article?

Type Your Valid Email Address below to Receive the Article PDF Link

Verification Code will appear in 2 Seconds ... Wait