Faster and Resourceful Multi-core Web Crawling
International Journal of Science and Research (IJSR)

International Journal of Science and Research (IJSR)
www.ijsr.net | Open Access | Fully Refereed | Peer Reviewed International Journal

ISSN: 2319-7064

Research Paper | Computer Science & Engineering | India | Volume 1 Issue 2, November 2012

Faster and Resourceful Multi-core Web Crawling

Arun Kumar Dewangan, Asha Ambhaikar

Due to massive growth of World Wide Web, search engines have become crucial tools for navigation of web pages. In order to provide fast and powerful search facility, search engine maintains indexes for documents and its contents on the web by downloading web pages for processing in iterative manner. Web indexes are created and managed by web crawlers which work as a module of search engines and traverse the web in systematic manner for indexing its contents. A web crawling is a process which fetches data from various servers. It is a time taking process as it gathers data from various servers. Hence, to speed up the searching process in search engines the crawling should be fast enough. The aim of the proposed system is to enhance the speed of crawling process and CPU usage by use of multi-core concept.

Keywords: Web crawling, multi core, indexing, parallel crawler, CPU, URL

Edition: Volume 1 Issue 2, November 2012

Pages: 12 - 15

Share this Article

How to Cite this Article?

Arun Kumar Dewangan, Asha Ambhaikar, "Faster and Resourceful Multi-core Web Crawling", International Journal of Science and Research (IJSR), https://www.ijsr.net/search_index_results_paperid.php?id=IJSR11120205, Volume 1 Issue 2, November 2012, 12 - 15

88 PDF Views | 82 PDF Downloads

Download Article PDF



Similar Articles with Keyword 'Web crawling'

Review Papers, Computer Science & Engineering, India, Volume 4 Issue 12, December 2015

Pages: 2212 - 2215

Focused and Adaptive Crawling for Topic Specific and Hidden Web Entries

Vrutuja Pande, Pratap Singh

Share this Article

Research Paper, Computer Science & Engineering, India, Volume 3 Issue 7, July 2014

Pages: 610 - 612

Reducing Duplicate Content Using XXHASH Algorithm

Rahul Mahajan, Dr. S. K. Gupta, Rajeev Bedi

Share this Article

M.Tech / M.E / PhD Thesis, Computer Science & Engineering, India, Volume 4 Issue 3, March 2015

Pages: 1810 - 1813

Intelligent Semantic Web Image Search Engine

Soorya P. S., Soya Chandra C.S.

Share this Article

Research Paper, Computer Science & Engineering, India, Volume 4 Issue 7, July 2015

Pages: 859863 -

Comparative Analysis of Web PageRank Algorithm using DFS and BFS Crawling

Poonam Kumari, Gaurav Kakhani

Share this Article

Research Paper, Computer Science & Engineering, India, Volume 1 Issue 2, November 2012

Pages: 12 - 15

Faster and Resourceful Multi-core Web Crawling

Arun Kumar Dewangan, Asha Ambhaikar

Share this Article

Similar Articles with Keyword 'multi core'

Survey Paper, Computer Science & Engineering, India, Volume 3 Issue 5, May 2014

Pages: 1166 - 1171

Literature Survey on a New Advanced Refactoring Based Approach for Parallelism Using Heterogeneous Parallel Architectures

Shanthi Makka, Bharat Bushan Sagar

Share this Article

Survey Paper, Computer Science & Engineering, India, Volume 5 Issue 8, August 2016

Pages: 1873 - 1875

Image Processing Application Using Parallel Computing

Vengayil Nayana Murali, Rahul C

Share this Article

Research Paper, Computer Science & Engineering, India, Volume 1 Issue 2, November 2012

Pages: 12 - 15

Faster and Resourceful Multi-core Web Crawling

Arun Kumar Dewangan, Asha Ambhaikar

Share this Article

Research Paper, Computer Science & Engineering, India, Volume 5 Issue 10, October 2016

Pages: 684 - 686

A Survey on Dynamic Scheduling Method for Heterogeneous, Multi-Core Processors

Dr. S. Sai Satyanarayana Reddy, Debashree Rupalin, K. Sravanthi

Share this Article

Similar Articles with Keyword 'indexing'

Research Paper, Computer Science & Engineering, India, Volume 3 Issue 9, September 2014

Pages: 1910 - 1915

An Efficient Hash Count Indexing and Searching Scheme for Audio Fingerprinting

Simarjeet Singh Bhatia, Rupali Bhartiya

Share this Article

Survey Paper, Computer Science & Engineering, India, Volume 4 Issue 11, November 2015

Pages: 2523 - 2525

Study of Semantic Retrieval by Data Similarity of Trademarks

Deepak Mali, P. D. Lambate

Share this Article

Review Papers, Computer Science & Engineering, India, Volume 3 Issue 12, December 2014

Pages: 2407 - 2409

A Review on Content-Based Image Retrieval from Videos using Self Learning Object Dictionary

Vrushali A. Wankhede, Prakash S. Mohod

Share this Article

Survey Paper, Computer Science & Engineering, India, Volume 4 Issue 10, October 2015

Pages: 1434 - 1436

Method for Repossession of Content Based Video using Speech and Text Information

Manasi A. Kabade, U.A. Jogalekar

Share this Article

Research Paper, Computer Science & Engineering, India, Volume 3 Issue 8, August 2014

Pages: 829 - 833

Comparison of Video Shot Detection and Video Summarization Techniques

Vijay Kumar D, Santhosh Kumar K.L, Jharna Majumdar

Share this Article

Similar Articles with Keyword 'parallel crawler'

Review Papers, Computer Science & Engineering, India, Volume 5 Issue 6, June 2016

Pages: 61 - 64

Design Issues in Web Crawlers and Review of Parallel Crawlers

Komal, Dr. Ashutosh Dixit

Share this Article

Research Paper, Computer Science & Engineering, India, Volume 1 Issue 2, November 2012

Pages: 12 - 15

Faster and Resourceful Multi-core Web Crawling

Arun Kumar Dewangan, Asha Ambhaikar

Share this Article

Similar Articles with Keyword 'CPU'

Survey Paper, Computer Science & Engineering, India, Volume 7 Issue 1, January 2018

Pages: 81 - 84

Novel Approach to Virtual Machine Migration In Cloud Computing Environment - A Survey

Priyanka H, Dr. Mary Cherian

Share this Article

Survey Paper, Computer Science & Engineering, India, Volume 3 Issue 5, May 2014

Pages: 1166 - 1171

Literature Survey on a New Advanced Refactoring Based Approach for Parallelism Using Heterogeneous Parallel Architectures

Shanthi Makka, Bharat Bushan Sagar

Share this Article

Survey Paper, Computer Science & Engineering, India, Volume 4 Issue 5, May 2015

Pages: 1987 - 1990

A Study on Setting Processor or CPU Affinity in Multi-Core Architecture for Parallel Computing

Saroj A. Shambharkar

Share this Article

Research Paper, Computer Science & Engineering, India, Volume 3 Issue 6, June 2014

Pages: 1525 - 1529

Blowfish Algorithm by Modify Randomness for S-Boxes using Fuzzy Value and Apply Encryption or Decryption on Image

Maulik P. Chaudhari, Neha Parmar

Share this Article

Survey Paper, Computer Science & Engineering, India, Volume 4 Issue 3, March 2015

Pages: 1179 - 1184

Survey on Schedulers Optimization to Handle Multiple Jobs in Hadoop Cluster

Shivaraj B. G., Nagaraj Naik

Share this Article

Similar Articles with Keyword 'URL'

M.Tech / M.E / PhD Thesis, Computer Science & Engineering, India, Volume 9 Issue 7, July 2020

Pages: 1888 - 1890

Spammer Detection and Identification on Social Network Using Machine Learning

Dr. Shameem Akhter, Noorain Saba

Share this Article

Research Paper, Computer Science & Engineering, India, Volume 4 Issue 6, June 2015

Pages: 2880 - 2884

Inferring User Search Goals Using Feedback Session

Harshada P. Bhambure, Mandar Mokashi

Share this Article

M.Tech / M.E / PhD Thesis, Computer Science & Engineering, India, Volume 4 Issue 2, February 2015

Pages: 1825 - 1828

Link-Anomaly Detection in Twitter Streams

Shari P S

Share this Article

Research Paper, Computer Science & Engineering, India, Volume 5 Issue 6, June 2016

Pages: 1895 - 1898

Implementation of Intellectual Browser for User Search Goal Discovery and Reordering System

Hemlata Gaikwad, Manisha Darak

Share this Article

M.Tech / M.E / PhD Thesis, Computer Science & Engineering, India, Volume 6 Issue 6, June 2017

Pages: 785 - 788

Efficient Top-k List Extraction from the Web

Jyoti H. Yadav, M. M. Deshpande

Share this Article
Top