DATA CLUSTERING USING BLACK HOLE ALGORITHM USING MAPREDUCE ON HADOOP FRAMEWORK

Abstract

The major drawback of conventional data clustering algorithms is that they are inefficient for analyzing large-scale datasets as most of them are tailored for a centralized system, that means if the size of input dataset exceeds the size of storage or memory of such a system, it would make the job of clustering much more difficult. To solve this problem, an efficient clustering algorithm, called Black Hole using MapReduce on Hadoop framework is proposed to ascend the strength of the black hole algorithm and the MapReduce programming model of Hadoop to accelerate the clustering speed by virtue of both software and hardware. By using MapReduce, the algorithm will then divide a large dataset into a number of small data sets and cluster these smaller data sets in parallel. Moreover, it inherits the characteristics of the black hole algorithm, meaning that no parameters are to be set manually; thus, the implementation is easy. To evaluate the performance of the proposed algorithm, several datasets are used with different numbers of nodes. Experimental results show that the proposed algorithm can provide a significant speedup as the number of nodes increases.