Effect of counters in performance of hadoop

Authors

  • Jain P Department of Computer Science and Engineering Acropolis Institute of Technology and Research, Indore
  • Kanungo J Department of Computer Science and Engineering Acropolis Institute of Technology and Research, Indore

Keywords:

Big Data, Map Reduce, hadoop, Hadoop Distributed File System (HDFS), CPU time spent, Size of data, number of blocks, Heap Size, HDFS Bytes Read, Spilled Records

Abstract

Recent technological advancements have led to an overflow of data from distinctive domains (e.g., health care and scientific sensors, user-generated data, Internet and financial companies, and supply chain systems) over the past two decades [1]. Big data is commonly unstructured, huge in volume and require more real-time analysis. This paper discusses a Big Data problem from NCDC for huge volume of weather data collected from various parts of world. We had generated map () and reduce () function for solving this problem and experimental results of these applications on a Hadoop cluster are being discussed. In this paper, performance of above application has been shown with respect to some counters available.

References

Maurya and Mahajan, S., “Performance analysis of MapReduce programs on Hadoop cluster”, Information and Communication Technologies (WICT), 2012 World Congress, ISBN:978-1-4673-4806-5, Oct. 30 2012.

A. Alexandrov, R. Bergmann, S. Ewen, J.C. Freytag, F. Hueske, A. Heise, O. Kao, M. Leich, U. Leser, V. M. Felix Naumann, M. Peters, A. Rheinländer, M. J. Sax and S. Schelter, “The Stratosphere Plateform for Big Data Analytics”, The VLDB Journal, Springer-Verlag Berlin Heidelberg, July 2013.

Shilpa and Manjit Kaur, “Big Data Visualization tool with Advancement of Chalange” International Journal of Advanced Research in Computer and Software Engineering”, ISSN: 2277 128X, Volume 4, Issue 3, March 2014.

D. Borthakur, “The Hadoop Distributed File System: Architecture and Design”.

Han Hu, Yonggang Wen , Tat-Seng Chua and Xuelong Li, “Toward Scalable Systems for Big Data Analytics: A Technology Tutorial”, ISSN :2169-3536, INSPEC Accession Number: 14429402

P. Kumar and Dr. V. S. Rathore, “Efficient Capabilities of Processing of Big Data using Hadoop Map Reduce”, International Journal of Advanced Research in Computer and Communication Engineering, Vol. 3, Issue 6, June 2014.

Shilpa and Manjit Kaur, “Big Data Visualization tool with Advancement of Chalange” International Journal of Advanced Research in Computer and Software Engineering”, ISSN: 2277 128X, Volume 4, Issue 3, March 2014.

X. Zhang, “Simple example to demonstrate how does the map reduce work”, June 2013.

Downloads

Published

2024-02-26

How to Cite

Jain, P., & Kanungo, J. (2024). Effect of counters in performance of hadoop. COMPUSOFT: An International Journal of Advanced Computer Technology, 4(05), 1714–1716. Retrieved from https://ijact.in/index.php/j/article/view/302

Issue

Section

Original Research Article

Similar Articles

<< < 41 42 43 44 45 46 47 48 > >> 

You may also start an advanced similarity search for this article.