Downloading: SmartCrawler: For Site Locating and Balancing
International Journal of Science and Research (IJSR)

International Journal of Science and Research (IJSR) | Open Access | Fully Refereed | Peer Reviewed International Journal

ISSN: 2319-7064

To prevent Server Overload, Your Article PDF will be Downloaded in Next Seconds

SmartCrawler: For Site Locating and Balancing

Sphurti S. Bhosale, Dr. S. B. Sonkamble

Abstract: The number of web pages in Internet are changed day by day. Therefore relevant information searching is difficult task. Most of this data is hidden behind query forms which is interface to unexplored databases containing high quality structured data. Existing search engine has drawback that they cannot access and index this hidden data of the Web, in fact it is very hard to derive the unseen info. For this, we propose a framework, namely SmartCrawler, just to gather all the web interfaces in depth. Initially that is site locating, in this step, center pages are searched using search engines which is used to avoid visiting a large number of pages. For achieving precise results for a focused crawl, SmartCrawler gives ranks to websites to prioritize highly relevant ones for a given topic. The second step, adaptive link-ranking gives fast in-site searching by searching most relevant links. In third step, i.e. Web Navigation, Space complexity problem of other crawler are managed. The required relevant links are stored in database. Parsing of web pages is done in web navigation step. To reduce time in visiting some highly related links in unseen web directories, we have designed a link tree data

Keywords: Ranker, Form Classifier, Deep web, Crawler, Adaptive learning