Culminating Project Title

Improving Big Data Processing Time

Date of Award

12-2016

Culminating Project Type

Starred Paper

Degree Name

Engineering: M.E.M

Department

Mechanical and Manufacturing Engineering

College

College of Science and Engineering

First Advisor

Ben Baliga

Second Advisor

Hiral Shah

Third Advisor

Balasubramanian Kasi

Creative Commons License

Creative Commons License
This work is licensed under a Creative Commons Attribution-Noncommercial-No Derivative Works 4.0 License.

Keywords and Subject Headings

Big data, Hadoop, Processing Time, Traditional Database

Abstract

The process of storing and processing massive amounts of data (big data) in a traditional database is expensive and consumes a lot of time to obtain desired results. This project has been implemented to solve these problems faced by an organization, with the implementation of Hadoop framework that stores huge data sets on distributed clusters and performs parallel data processing to achieve results quickly. It uses commodity hardware to store the data making it cost effective and provides data security by replicating the data sets. The main goals of the project were to improve the performance of processing huge data sets, reduce long term data storage costs and provide a platform that supports ad hoc analysis and provides real-time insights. The project was structured to follow agile model of software development and the data was collected and analyzed after the execution of the project. The results obtained by the analysis of data aided in arriving to the conclusion and validating that the stated goals were achieved.

Comments/Acknowledgements

I would like to take this opportunity to express my humble gratitude to my advisor Dr. Ben Baliga for his tremendous support and for the guidance he has provided me that led to the successful completion of my starred paper. I would also like to extend my thanks to Dr. Hiral Shah and Dr. Balasubramanian Kasi for serving as committee members and providing useful insights throughout the process. I am also very grateful for the necessary resources provided by the Department of Engineering Management, SCSU.

Share

COinS
 
 

To view the content in your browser, please download Adobe Reader or, alternately,
you may Download the file to your hard drive.

NOTE: The latest versions of Adobe Reader do not support viewing PDF files within Firefox on Mac OS and if you are using a modern (Intel) Mac, there is no official plugin for viewing PDF files within the browser window.