Filters Based Focus Crawler Using Utility Theory

Manpreet Kaur, Yasmeen Kaur Dhaliwal

Abstract: In the research work, we have developed a filters based focus crawling algorithm and a scoring system in which the relevant web pages are retrieved by the crawler. In this proposed algorithm, three filters such as ego, degree and partition filters are used to find the relevant web pages for crawling. These filters are used for comparing crawled webpage tokens with the master graphs, which reflects the interconnection and closeness between the words between considered and web page that is executed by the crawler. This algorithm calculates high precision, recall value as compared to the best first search algorithm.

Keywords: Ego Filter, Degree Filter, Partition Filter, Multi domain, Best first search