Skip to main content
Article
A set theory based similarity measure for text clustering and classification
Journal of Big Data
  • Ali A. Amer, Taiz University
  • Hassan I. Abdalla, Zayed University
ORCID Identifiers

0000-0002-2002-948X

Document Type
Article
Publication Date
12-1-2020
Abstract

© 2020, The Author(s). Similarity measures have long been utilized in information retrieval and machine learning domains for multi-purposes including text retrieval, text clustering, text summarization, plagiarism detection, and several other text-processing applications. However, the problem with these measures is that, until recently, there has never been one single measure recorded to be highly effective and efficient at the same time. Thus, the quest for an efficient and effective similarity measure is still an open-ended challenge. This study, in consequence, introduces a new highly-effective and time-efficient similarity measure for text clustering and classification. Furthermore, the study aims to provide a comprehensive scrutinization for seven of the most widely used similarity measures, mainly concerning their effectiveness and efficiency. Using the K-nearest neighbor algorithm (KNN) for classification, the K-means algorithm for clustering, and the bag of word (BoW) model for feature selection, all similarity measures are carefully examined in detail. The experimental evaluation has been made on two of the most popular datasets, namely, Reuters-21 and Web-KB. The obtained results confirm that the proposed set theory-based similarity measure (STB-SM), as a pre-eminent measure, outweighs all state-of-art measures significantly with regards to both effectiveness and efficiency.

Publisher
Springer
Disciplines
Keywords
  • Empirical study,
  • Information retrieval,
  • Similarity measures,
  • Text classification,
  • Text retrieval
Scopus ID
85090913078
Creative Commons License
Creative Commons Attribution 4.0 International
Indexed in Scopus
Yes
Open Access
Yes
Open Access Type
Gold: This publication is openly available in an open access journal/series
Citation Information
Ali A. Amer and Hassan I. Abdalla. "A set theory based similarity measure for text clustering and classification" Journal of Big Data Vol. 7 Iss. 1 (2020) - 43 ISSN: <a href="https://v2.sherpa.ac.uk/id/publication/issn/2196-1115" target="_blank">2196-1115</a>
Available at: http://works.bepress.com/hassan-abdalla/5/