MapReduce Job Optimization: A Mapping Study

Qinghua Lu; Liming Zhu; He Zhang; Dongyao Wu; Zheng Li; Xiwei Xu

doi:10.1109/CCBD.2015.33

MapReduce Job Optimization: A Mapping Study

Qinghua Lu, Liming Zhu, He Zhang, Dongyao Wu, Zheng Li, Xiwei Xu

Research output: Chapter in Book/Report/Conference proceeding › Paper in conference proceeding › peer-review

Abstract

MapReduce has become the standard model for supporting big data analytics. In particular, MapReduce job optimization has been widely considered to be crucial in the implementations of big data analytics. However, there is still a lack of guidelines especially for practitioners to understand how the MapReduce jobs can be optimized. This paper aims to systematic identify and taxonomically classify the existing work on job optimization. We conducted a mapping study on 47 selected papers that were published between 2004 and 2014. We classified and compared the selected papers based on a 5WH-based characterization framework. This study generates a knowledge base of current job optimization solutions and also identifies a set of research gaps and opportunities. This study concludes that job optimization is still in an early stage of maturity. More attentions need to be paid to the cross-data center, cluster or rack job optimization to improve communication efficiency.

Original language	English
Title of host publication	Proceedings - 2015 International Conference on Cloud Computing and Big Data, CCBD 2015
Publisher	IEEE - Institute of Electrical and Electronics Engineers Inc.
Pages	81-88
Number of pages	8
ISBN (Electronic)	9781467383509
DOIs	https://doi.org/10.1109/CCBD.2015.33
Publication status	Published - 2016 Apr 8
Event	International Conference on Cloud Computing and Big Data, CCBD 2015 - Shanghai, China Duration: 2015 Nov 4 → 2015 Nov 6

Conference

Conference	International Conference on Cloud Computing and Big Data, CCBD 2015
Country/Territory	China
City	Shanghai
Period	2015/11/04 → 2015/11/06

Subject classification (UKÄ)

Information Systems

Free keywords

big data
job optimization
mapping study
MapReduce
systematic literature review

Access to Document

10.1109/CCBD.2015.33

Cite this

@inproceedings{ef9c319d6be8498683edaa9971c7be51,

title = "MapReduce Job Optimization: A Mapping Study",

abstract = "MapReduce has become the standard model for supporting big data analytics. In particular, MapReduce job optimization has been widely considered to be crucial in the implementations of big data analytics. However, there is still a lack of guidelines especially for practitioners to understand how the MapReduce jobs can be optimized. This paper aims to systematic identify and taxonomically classify the existing work on job optimization. We conducted a mapping study on 47 selected papers that were published between 2004 and 2014. We classified and compared the selected papers based on a 5WH-based characterization framework. This study generates a knowledge base of current job optimization solutions and also identifies a set of research gaps and opportunities. This study concludes that job optimization is still in an early stage of maturity. More attentions need to be paid to the cross-data center, cluster or rack job optimization to improve communication efficiency.",

keywords = "big data, job optimization, mapping study, MapReduce, systematic literature review",

author = "Qinghua Lu and Liming Zhu and He Zhang and Dongyao Wu and Zheng Li and Xiwei Xu",

year = "2016",

month = apr,

day = "8",

doi = "10.1109/CCBD.2015.33",

language = "English",

pages = "81--88",

booktitle = "Proceedings - 2015 International Conference on Cloud Computing and Big Data, CCBD 2015",

publisher = "IEEE - Institute of Electrical and Electronics Engineers Inc.",

address = "United States",

note = "International Conference on Cloud Computing and Big Data, CCBD 2015 ; Conference date: 04-11-2015 Through 06-11-2015",

}

Lu, Q, Zhu, L, Zhang, H, Wu, D, Li, Z & Xu, X 2016, MapReduce Job Optimization: A Mapping Study. in Proceedings - 2015 International Conference on Cloud Computing and Big Data, CCBD 2015., 7450534, IEEE - Institute of Electrical and Electronics Engineers Inc., pp. 81-88, International Conference on Cloud Computing and Big Data, CCBD 2015, Shanghai, China, 2015/11/04. https://doi.org/10.1109/CCBD.2015.33

MapReduce Job Optimization: A Mapping Study. / Lu, Qinghua; Zhu, Liming; Zhang, He et al.
Proceedings - 2015 International Conference on Cloud Computing and Big Data, CCBD 2015. IEEE - Institute of Electrical and Electronics Engineers Inc., 2016. p. 81-88 7450534.

Research output: Chapter in Book/Report/Conference proceeding › Paper in conference proceeding › peer-review

TY - GEN

T1 - MapReduce Job Optimization

T2 - International Conference on Cloud Computing and Big Data, CCBD 2015

AU - Lu, Qinghua

AU - Zhu, Liming

AU - Zhang, He

AU - Wu, Dongyao

AU - Li, Zheng

AU - Xu, Xiwei

PY - 2016/4/8

Y1 - 2016/4/8

N2 - MapReduce has become the standard model for supporting big data analytics. In particular, MapReduce job optimization has been widely considered to be crucial in the implementations of big data analytics. However, there is still a lack of guidelines especially for practitioners to understand how the MapReduce jobs can be optimized. This paper aims to systematic identify and taxonomically classify the existing work on job optimization. We conducted a mapping study on 47 selected papers that were published between 2004 and 2014. We classified and compared the selected papers based on a 5WH-based characterization framework. This study generates a knowledge base of current job optimization solutions and also identifies a set of research gaps and opportunities. This study concludes that job optimization is still in an early stage of maturity. More attentions need to be paid to the cross-data center, cluster or rack job optimization to improve communication efficiency.

AB - MapReduce has become the standard model for supporting big data analytics. In particular, MapReduce job optimization has been widely considered to be crucial in the implementations of big data analytics. However, there is still a lack of guidelines especially for practitioners to understand how the MapReduce jobs can be optimized. This paper aims to systematic identify and taxonomically classify the existing work on job optimization. We conducted a mapping study on 47 selected papers that were published between 2004 and 2014. We classified and compared the selected papers based on a 5WH-based characterization framework. This study generates a knowledge base of current job optimization solutions and also identifies a set of research gaps and opportunities. This study concludes that job optimization is still in an early stage of maturity. More attentions need to be paid to the cross-data center, cluster or rack job optimization to improve communication efficiency.

KW - big data

KW - job optimization

KW - mapping study

KW - MapReduce

KW - systematic literature review

UR - http://www.scopus.com/inward/record.url?scp=84969506247&partnerID=8YFLogxK

U2 - 10.1109/CCBD.2015.33

DO - 10.1109/CCBD.2015.33

M3 - Paper in conference proceeding

SP - 81

EP - 88

BT - Proceedings - 2015 International Conference on Cloud Computing and Big Data, CCBD 2015

PB - IEEE - Institute of Electrical and Electronics Engineers Inc.

Y2 - 4 November 2015 through 6 November 2015

ER -

MapReduce Job Optimization: A Mapping Study

Abstract

Conference

Subject classification (UKÄ)

Free keywords

Access to Document

Other files and links

Fingerprint

Cite this