International Journal of Computer Engineering in Research Trends

Back to Current Issues

Survey on Big Data using Apache Hadoop and Spark

Priya Dahiya , Chaitra.B , Usha Kumari , ,

Affiliations
Information Science Dept. , Acharya Doctor Sarvepalli Radhakrishnan Rd, Bengaluru, Karnataka 560107, India.

:10.22362/ijcert/2017/v4/i6/xxxx [UNDER PROCESS]

Abstract

Big data is growing rapidly regarding volume, variability, and velocity which make it difficult to process, capture and analyze the data. Hadoop uses MapReduce which has two parts Map and Reduce whereas Spark uses Resilient Distributed Datasets (RDD) and Directed Acyclic Graph (DAG) for processing of large datasets. To store data both of them uses Hadoop Distributed File System (HDFS).This paper shows the architecture and working of Hadoop and Spark and brings out the differences between them and the challenges faced by MapReduce during processing of large datasets and how Spark works on Hadoop YARN.

Citation

Priya Dahiya et.al, â€œSurvey on Big Data using Apache Hadoop and Sparkâ€, International Journal of Computer Engineering In Research Trends, 4(6):pp:195-201,June -2017.

Keywords : Big data, Spark, Hadoop, HDFS, MapReduce, YARN

References

1. Varsha B.Bobade, â€œSurvey Paper on Big Data and Hadoopâ€, International Research Journal of Engineering and Technology (IRJET) , Volume: 03 Issue: 01 | Jan-2016, e-ISSN: 2395-0056  p-ISSN: 2395-0072.

2. S. Justin Samuel, Koundinya RVP, Kotha Sashidhar and C.R. Bharathi,  â€œA SURVEY ON BIG DATA AND ITS RESEARCH CHALLENGESâ€, VOL. 10, NO. 8, MAY 2015 ISSN 1819-6608, ARPN Journal of Engineering and Applied Sciences.

3.Ms. Vibhavari Chavan, Prof. Rajesh. N. Pursue â€œSurvey Paper On Big Dataâ€, Vibhavari 
Chavan et al, / (IJCSIT) International Journal of Computer Science and Information Technologies, Vol. 5 (6) , 2014, 7932-7939.

4. Ankush Verma ,Ashik Hussain Mansuri ,Dr. Neelesh Jain â€œBig Data Management Processing with Hadoop MapReduce and Spark Technology: A Comparisonâ€ 2016 Symposium on Colossal Data Analysis and Networking (CDAN) , 978-1- 5090-0669-4/16/$31.00 Â© 2016 IEEE. 

5. Wei Huang, Lingkui Meng, Dongying Zhang, and Wen Zhang, â€œIn-Memory Parallel Processing of Massive Remotely Sensed Data Using an Apache Spark on Hadoop YARN Modelâ€ , IEEE JOURNAL OF SELECTED TOPICS IN APPLIED EARTH OBSERVATIONS AND REMOTE SENSING, VOL. 10, NO. 1, DECEMBER 2016.

6. Katarina Grolinger, Michael Hayes, Wilson A. Higashino, Alexandra L'Heureux1 David S.Allison ,Miriam A.M. Capretz, â€œChallenges for MapReduce in Big Data â€, 978-1- 4799-5069-0/14 $31.00Â©2014IEEEDOI10.1109/SERVICES.2014.4.

7. Xiuqin LIN, Peng WANG, Bin WU, â€œLOG ANALYSIS IN CLOUD COMPUTING ENVIRONMENT WITH HADOOP AND SPARKâ€, 978-1-4799-0094-7/13/$31.00Â©2013

8. K..Naga Maha Lakshmi et al., International Journal of Computer Engineering In Research Trends ,Volume 3, Issue 3, March-2016, pp. 134-142.

9. Sunil B. Mane et.al, â€œProduct Rating using Opinion Miningâ€, International Journal of Computer Engineering In Research Trends, 4(5):161-168 ,May -2017.

DOI Link : Not yet assigned

Download :

V4I6001.pdf

Refbacks : There are Currently no refbacks

Announcements

Authors are not required to pay any article-processing charges (APC) for their article to be published open access in Journal IJCERT. No charge is involved in any stage of the publication process, from administrating peer review to copy editing and hosting the final article on dedicated servers. This is free for all authors.

News & Events

Latest issue :Volume 10 Issue 1 Articles In press

A plagiarism check will be implemented for all the articles using world-renowned software. Turnitin.

Digital Object Identifier will be assigned for all the articles being published in the Journal from September 2016 issue, i.e. Volume 3, Issue 9, 2016.

IJCERT is a member of the prestigious.Each of the IJCERT articles has its unique DOI reference.
DOI Prefix : 10.22362/ijcert

IJCERT is member of The Publishers International Linking Association, Inc. (“PILA”)

Emerging Sources Citation Index (in process)

IJCERT title is under evaluation by Scopus.

Key Dates

☞ INVITING SUBMISSIONS FOR THE NEXT ISSUE :

☞ LAST DATE OF SUBMISSION : 31st March 2023

☞ SUBMISSION TO FIRST DECISION : In 7 Days

☞ FINAL DECISION : IN 3 WEEKS FROM THE DAY OF SUBMISSION

Important Announcements

All the authors, conference coordinators, conveners, and guest editors kindly check their articles' originality before submitting them to IJCERT. If any material is found to be duplicate submission or sent to other journals when the content is in the process with IJCERT, fabricated data, cut and paste (plagiarized), at any stage of processing of material, IJCERT is bound to take the following actions.
1. Rejection of the article.
2. The author will be blocked for future communication with IJCERT if duplicate articles are submitted.
3. A letter regarding this will be posted to the Principal/Director of the Institution where the study was conducted.
4. A List of blacklisted authors will be shared among the Chief Editors of other prestigious Journals
We have been screening articles for plagiarism with a world-renowned tool: Turnitin However, it is only rejected if found plagiarized. This more stern action is being taken because of the illegal behavior of a handful of authors who have been involved in ethical misconduct. The Screening and making a decision on such articles costs colossal time and resources for the journal. It directly delays the process of genuine materials.

Citation Index

Citations Indices	All
Citations	1026
h-index	14
i10-index	20
Source: Google Scholar

Acceptance Rate (By Year)

Acceptance Rate (By Year)
Year	Rate
2021	10.8%
2020	13.6%
2019	15.9%
2018	14.5%
2017	16.6%
2016	15.8%
2015	18.2%
2014	20.6%

 Conference Proposal

 Peer Review

 Digital Library

 Register here

 Conference Proposal

 Special Issue Proposal

 Conference Proposal Guidelines

 Upcoming Conference

DOI:10.22362/ijcert

Welcome to IJCERT