International Journal of Science and Research (IJSR)

International Journal of Science and Research (IJSR)
www.ijsr.net | Most Trusted Research Journal Since Year 2012

ISSN: 2319-7064

Research Paper | Computer Science & Engineering | India | Volume 1 Issue 2, November 2012

Faster and Resourceful Multi-core Web Crawling

Arun Kumar Dewangan, Asha Ambhaikar

Due to massive growth of World Wide Web, search engines have become crucial tools for navigation of web pages. In order to provide fast and powerful search facility, search engine maintains indexes for documents and its contents on the web by downloading web pages for processing in iterative manner. Web indexes are created and managed by web crawlers which work as a module of search engines and traverse the web in systematic manner for indexing its contents. A web crawling is a process which fetches data from various servers. It is a time taking process as it gathers data from various servers. Hence, to speed up the searching process in search engines the crawling should be fast enough. The aim of the proposed system is to enhance the speed of crawling process and CPU usage by use of multi-core concept.

Keywords: Web crawling, multi core, indexing, parallel crawler, CPU, URL

Edition: Volume 1 Issue 2, November 2012

Pages: 12 - 15


How to Cite this Article?

Arun Kumar Dewangan, Asha Ambhaikar, "Faster and Resourceful Multi-core Web Crawling", International Journal of Science and Research (IJSR), https://www.ijsr.net/search_index_results_paperid.php?id=IJSR11120205, Volume 1 Issue 2, November 2012, 12 - 15

30 PDF Views | 29 PDF Downloads

Download Article PDF



Similar Articles with Keyword 'Web crawling'

Research Paper, Computer Science & Engineering, India, Volume 3 Issue 7, July 2014

Pages: 610 - 612

Reducing Duplicate Content Using XXHASH Algorithm

Rahul Mahajan, Dr. S. K. Gupta, Rajeev Bedi

Share this article

M.Tech / M.E / PhD Thesis, Computer Science & Engineering, India, Volume 4 Issue 3, March 2015

Pages: 1810 - 1813

Intelligent Semantic Web Image Search Engine

Soorya P. S., Soya Chandra C.S.

Share this article

Review Papers, Computer Science & Engineering, India, Volume 4 Issue 12, December 2015

Pages: 2212 - 2215

Focused and Adaptive Crawling for Topic Specific and Hidden Web Entries

Vrutuja Pande, Pratap Singh

Share this article

Research Paper, Computer Science & Engineering, India, Volume 4 Issue 7, July 2015

Pages: 859863 -

Comparative Analysis of Web PageRank Algorithm using DFS and BFS Crawling

Poonam Kumari, Gaurav Kakhani

Share this article

Research Paper, Computer Science & Engineering, India, Volume 5 Issue 9, September 2016

Pages: 527 - 531

Deep Web Mining Using C# Wrappers

Rakesh Kumar Baloda, Praveen Kantha

Share this article

Similar Articles with Keyword 'multi core'

Survey Paper, Computer Science & Engineering, India, Volume 3 Issue 5, May 2014

Pages: 1166 - 1171

Literature Survey on a New Advanced Refactoring Based Approach for Parallelism Using Heterogeneous Parallel Architectures

Shanthi Makka, Bharat Bushan Sagar

Share this article

Survey Paper, Computer Science & Engineering, India, Volume 5 Issue 8, August 2016

Pages: 1873 - 1875

Image Processing Application Using Parallel Computing

Vengayil Nayana Murali, Rahul C

Share this article

Research Paper, Computer Science & Engineering, India, Volume 5 Issue 10, October 2016

Pages: 684 - 686

A Survey on Dynamic Scheduling Method for Heterogeneous, Multi-Core Processors

Dr. S. Sai Satyanarayana Reddy, Debashree Rupalin, K. Sravanthi

Share this article

Research Paper, Computer Science & Engineering, India, Volume 1 Issue 2, November 2012

Pages: 12 - 15

Faster and Resourceful Multi-core Web Crawling

Arun Kumar Dewangan, Asha Ambhaikar

Share this article

Similar Articles with Keyword 'indexing'

Survey Paper, Computer Science & Engineering, India, Volume 4 Issue 10, October 2015

Pages: 1434 - 1436

Method for Repossession of Content Based Video using Speech and Text Information

Manasi A. Kabade, U.A. Jogalekar

Share this article

Research Paper, Computer Science & Engineering, India, Volume 3 Issue 9, September 2014

Pages: 1698 - 1704

Maintenance of User Query Integrity and Voronoi Neighbors Using Multiple Signers in Signature Aggregation

V. Maruvoor Arasi, Carolene

Share this article

Research Paper, Computer Science & Engineering, India, Volume 3 Issue 7, July 2014

Pages: 610 - 612

Reducing Duplicate Content Using XXHASH Algorithm

Rahul Mahajan, Dr. S. K. Gupta, Rajeev Bedi

Share this article

Research Paper, Computer Science & Engineering, India, Volume 4 Issue 4, April 2015

Pages: 1471 - 1474

Web Crawler: Essential Component of Search Engine

Akshada K. Dhakade, Deepak C. Dhanwani

Share this article

Research Paper, Computer Science & Engineering, India, Volume 3 Issue 7, July 2014

Pages: 969 - 973

Improved Indexing and Advanced Relevance Ranking Score for Multi-Keyword Search over Encrypted Cloud Data

Amol D. Sawant, M. D. Ingle

Share this article

Similar Articles with Keyword 'parallel crawler'

Review Papers, Computer Science & Engineering, India, Volume 5 Issue 6, June 2016

Pages: 61 - 64

Design Issues in Web Crawlers and Review of Parallel Crawlers

Komal, Dr. Ashutosh Dixit

Share this article

Research Paper, Computer Science & Engineering, India, Volume 1 Issue 2, November 2012

Pages: 12 - 15

Faster and Resourceful Multi-core Web Crawling

Arun Kumar Dewangan, Asha Ambhaikar

Share this article

Similar Articles with Keyword 'CPU'

M.Tech / M.E / PhD Thesis, Computer Science & Engineering, India, Volume 4 Issue 5, May 2015

Pages: 2065 - 2069

Preventing Cache Based Side-Channel Attacks for Security in Cloud over Virtual Environment

S. Vengadesan, B. Muthulakshmi

Share this article

Research Paper, Computer Science & Engineering, Nigeria, Volume 6 Issue 4, April 2017

Pages: 1823 - 1831

Computing Devices as a Technology

Olaniyi Olaposi

Share this article

Survey Paper, Computer Science & Engineering, India, Volume 3 Issue 5, May 2014

Pages: 1082 - 1086

A Comprehensive Analysis of Existing Load Balancing Algorithms in Cloud Network

Pinki, Nida

Share this article

Survey Paper, Computer Science & Engineering, India, Volume 3 Issue 5, May 2014

Pages: 1166 - 1171

Literature Survey on a New Advanced Refactoring Based Approach for Parallelism Using Heterogeneous Parallel Architectures

Shanthi Makka, Bharat Bushan Sagar

Share this article

Review Papers, Computer Science & Engineering, India, Volume 3 Issue 6, June 2014

Pages: 2499 - 2504

Review on Load Balancing Techniques in Cloud Computing Environment

Sukhvir Kaur, Supriya Kinger

Share this article

Similar Articles with Keyword 'URL'

Research Paper, Computer Science & Engineering, India, Volume 4 Issue 6, June 2015

Pages: 2880 - 2884

Inferring User Search Goals Using Feedback Session

Harshada P. Bhambure, Mandar Mokashi

Share this article

M.Tech / M.E / PhD Thesis, Computer Science & Engineering, India, Volume 4 Issue 2, February 2015

Pages: 1825 - 1828

Link-Anomaly Detection in Twitter Streams

Shari P S

Share this article

Research Paper, Computer Science & Engineering, India, Volume 3 Issue 6, June 2014

Pages: 2083 - 2089

An Enhanced Web Mining Technique for Image Search using Weighted PageRank based on Visit of Links and Fuzzy K-Means Algorithm

Rashmi Sharma, Kamaljit Kaur

Share this article

Research Paper, Computer Science & Engineering, India, Volume 3 Issue 7, July 2014

Pages: 610 - 612

Reducing Duplicate Content Using XXHASH Algorithm

Rahul Mahajan, Dr. S. K. Gupta, Rajeev Bedi

Share this article

Research Paper, Computer Science & Engineering, India, Volume 3 Issue 6, June 2014

Pages: 885 - 888

Half Hourly Electricity Load Prediction using Echo State Network

Shivani Varshney, Toran Verma

Share this article

Top