Search for Articles:

Recently Downloaded: Paper ID: ART2019861, Total 5878 Articles Downloaded Today

Issues and Challenges with the Web Crawlers


Abstract: A Web crawler is an Internet bot which helps in indexing the web. It crawls one page at a time through a website until all pages has been indexed. Web crawler helps in collecting information about a website and the links related to them. A web crawler is defined as mechanized program that accurately scans through Internet pages and downloads any page that can be reached by links A Web crawler is also known as a Web spider, automatic indexer or simply crawler. Nowdays finding the relevant information over millions bytes of data is very difficult. For this task we need to use efficient crawler so that it can give us best result within minimum time.But there are so many issues associated with each crawler so This paper discusses the issues and challenges involved in the design of the various types of crawlers.

Keywords: Indexer,Website,Scans.

Country: India, Subject Area: Computer Engineering

Pages: 1791 - 1795

Edition: Volume 8 Issue 3, March 2019

How to Cite this Article?

Vishakha, "Issues and Challenges with the Web Crawlers", International Journal of Science and Research (IJSR),, Volume 8 Issue 3, March 2019, 1791 - 1795

Download PDF

Viewed 49 times.

Downloaded 18 times.