ENHANCEMENT OF RDB-MINER ALGORITHM FOR MULTI-CLUSTER HADOOPDB

Abstract

In recent times, volumes of data repositories have astronomically skyrocketed. Data is now regarded as big data and is now measured in the magnitudes of Yotabytes and Petabytes. This has been catalyzed by the rapid development of the Internet and the central role it has claimed in our daily lives. Commercial companies have taken this opportunity to gather as much data about their clients as possible. Governments have also invested heavily in data gathering and analytics. This obsessive collection of data has resulted in an enormous sea of data of all formats and structures. Data has become the most valuable asset for most organizations as it allows them to glean some business intelligence from it and enables them to have a competitive advantage over their rivals. However, it has become more complicated to extract or discover any meaningful patterns, associations or business intelligence from this large pool. Many technologies, platforms and techniques have been developed in order to aid in the data mining process of this BIG DATA. Unfortunately, most projects have been aimed at unstructured and semi-structured data. A blind eye has been turned on the structured data in big data. This project aims to enhance the RDB-Miner algorithm which is an association rule mining algorithm for relation data model (structured data). The project intends to make it applicable and adoptable to structured data in big data. We aim to parallelize the algorithm and implement it in a multi-cluster HadoopDB environment.