Date of Award
12-2016
Culminating Project Type
Starred Paper
Degree Name
Engineering: M.E.M
Department
Mechanical and Manufacturing Engineering
College
College of Science and Engineering
First Advisor
Ben Baliga
Second Advisor
Hiral Shah
Third Advisor
Balasubramanian Kasi
Creative Commons License
This work is licensed under a Creative Commons Attribution-Noncommercial-No Derivative Works 4.0 License.
Keywords and Subject Headings
Big data, Hadoop, Processing Time, Traditional Database
Abstract
The process of storing and processing massive amounts of data (big data) in a traditional database is expensive and consumes a lot of time to obtain desired results. This project has been implemented to solve these problems faced by an organization, with the implementation of Hadoop framework that stores huge data sets on distributed clusters and performs parallel data processing to achieve results quickly. It uses commodity hardware to store the data making it cost effective and provides data security by replicating the data sets. The main goals of the project were to improve the performance of processing huge data sets, reduce long term data storage costs and provide a platform that supports ad hoc analysis and provides real-time insights. The project was structured to follow agile model of software development and the data was collected and analyzed after the execution of the project. The results obtained by the analysis of data aided in arriving to the conclusion and validating that the stated goals were achieved.
Recommended Citation
Lnu, Ruqia Maihveen, "Improving Big Data Processing Time" (2016). Culminating Projects in Mechanical and Manufacturing Engineering. 62.
https://repository.stcloudstate.edu/mme_etds/62
Comments/Acknowledgements
I would like to take this opportunity to express my humble gratitude to my advisor Dr. Ben Baliga for his tremendous support and for the guidance he has provided me that led to the successful completion of my starred paper. I would also like to extend my thanks to Dr. Hiral Shah and Dr. Balasubramanian Kasi for serving as committee members and providing useful insights throughout the process. I am also very grateful for the necessary resources provided by the Department of Engineering Management, SCSU.