Using Clustering Algorithm to Determine the Number of Clusters

Aditya Darak, Akhil Chaudhary, Prajwal Mogaveera

Abstract: Clustering is important technique in data mining. The process of clustering involves partitioning of data into groups on the basis of similarities and differences between them. Clustering is used in various fields such as psychology, biology, data mining, image analysis, economics, pattern recognition, bioinformatics, weather forecasting, etc. The result of clustering varies as the number of cluster parameter changes. Therefore, the main challenge to cluster analysis is that the number of clusters or the number of parameters is seldom known and must be determined before clustering. Several clustering algorithms have been proposed. Among them the k-means clustering is a simple and fast clustering technique. Here, we address the problem of selecting the number of clusters by using a k-means approach. We can ask the end users to provide number of clusters in advance. But it may not always be feasible as the end user requires the domain knowledge of each data set. The initial cluster centers varies directly as the number of clusters. Thus, it is quite important for k-means to have good initial clusters. There are many methods available to estimate the number of clusters such as variable based method, statistical indices, information theoretic, goodness of fit, etc.

Keywords: Clustering, Hierarchical Clustering, Partitioning clustering