Bendechache, Malika ORCID: 0000-0003-0069-1860, Le-Khac, Nhien-An and Kechadi, M-Tahar ORCID: 0000-0002-0176-6281 (2016) Efficient large scale clustering based on data partitioning. In: 2016 IEEE International Conference on Data Science and Advanced Analytics (DSAA), 17-19 Oct 2016, Montreal, Canada.
Abstract
Clustering techniques are very attractive for extracting and identifying patterns in datasets. However, their application to very large spatial datasets presents numerous challenges
such as high-dimensionality data, heterogeneity, and high complexity of some algorithms. For instance, some algorithms may
have linear complexity but they require the domain knowledge in
order to determine their input parameters. Distributed clustering
techniques constitute a very good alternative to big data
challenges (e.g., Volume, Variety, Veracity, and Velocity). Usually,
these techniques consist of two phases. The first phase generates
local models or patterns and the second one tends to aggregate
the local results to obtain global models. While the first phase can
be executed in parallel on each site and, therefore, efficient, the
aggregation phase is complex, time-consuming and may produce
incorrect and ambiguous global clusters and therefore incorrect
models.
In this paper we propose a new distributed clustering approach to deal efficiently with both phases; generation of local
results and generation of global models by aggregation. For the
first phase, our approach is capable of analysing the datasets
located in each site using different clustering techniques. The
aggregation phase is designed in such a way that the final clusters
are compact and accurate while the overall process is efficient
in time and memory allocation. For the evaluation, we use two
well-known clustering algorithms; K-Means and DBSCAN. One
of the key outputs of this distributed clustering technique is that
the number of global clusters is dynamic; no need to be fixed in
advance. Experimental results show that the approach is scalable
and produces high-quality results.
Metadata
Item Type: | Conference or Workshop Item (Paper) |
---|---|
Event Type: | Conference |
Refereed: | Yes |
Uncontrolled Keywords: | Big Data; spatial data; clustering; distributed mining; data analysis; k-means; DBSCAN |
Subjects: | Computer Science > Algorithms Computer Science > Computational complexity Computer Science > Machine learning |
DCU Faculties and Centres: | DCU Faculties and Schools > Faculty of Engineering and Computing > School of Computing Research Initiatives and Centres > INSIGHT Centre for Data Analytics |
Published in: | 2016 IEEE International Conference on Data Science and Advanced Analytics (DSAA), Proceedings. . IEEE. |
Publisher: | IEEE |
Official URL: | http://dx.doi.org/10.1109/DSAA.2016.70 |
Copyright Information: | © 2016 The Authors |
Use License: | This item is licensed under a Creative Commons Attribution-NonCommercial-Share Alike 3.0 License. View License |
Funders: | Science Foundation Ireland under Grant Number SFI/12/RC/2289. |
ID Code: | 24624 |
Deposited On: | 16 Jun 2020 15:39 by Malika Bendechache . Last Modified 16 Jun 2020 15:39 |
Documents
Full text available as:
Preview |
PDF
- Requires a PDF viewer such as GSview, Xpdf or Adobe Acrobat Reader
3MB |
Downloads
Downloads
Downloads per month over past year
Archive Staff Only: edit this record