Skip to main content
Article
An Incremental Threshold Method for Continuous Text Search Queries
ICDE 2009: 25th IEEE International Conference on Data Engineering: Proceedings, 29 March-2 April 2009, Shanghai, China
  • Kyriakos MOURATIDIS, Singapore Management University
  • Hwee Hwa PANG, Singapore Management University
Publication Type
Conference Proceeding Article
Version
submittedVersion
Publication Date
4-2009
Abstract

A text filtering system monitors a stream of incoming documents, to identify those that match the interest profiles of its users. The user interests are registered at a server as continuous text search queries. The server constantly maintains for each query a ranked result list, comprising the recent documents (drawn from a sliding window) with the highest similarity to the query. Such a system underlies many text monitoring applications that need to cope with heavy document traffic, such as news and email monitoring. In this paper, we propose the first solution for processing continuous text queries efficiently. Our objective is to support a large number of user queries while sustaining high document arrival rates. Our solution indexes the streamed documents with a structure based on the principles of the inverted file, and processes document arrival and expiration events with an incremental threshold-based method. Using a stream of real documents, we experimentally verify the efficiency of our approach, which is at least an order of magnitude faster than a competitor constructed from existing techniques.

Keywords
  • Arrival rates,
  • E-mail monitoring,
  • Inverted files,
  • Monitoring applications,
  • Order of magnitude,
  • Sliding Window,
  • Structure-based,
  • Text filtering,
  • Text query,
  • Text search,
  • Threshold methods,
  • User interests,
  • User query
ISBN
9781424434220
Identifier
10.1109/ICDE.2009.197
Publisher
IEEE Computer Society
City or Country
Los Alamitos, CA
Creative Commons License
Creative Commons Attribution-NonCommercial-No Derivative Works 4.0 International
Additional URL
http://doi.ieeecomputersociety.org/10.1109/ICDE.2009.197
Citation Information
Kyriakos MOURATIDIS and Hwee Hwa PANG. "An Incremental Threshold Method for Continuous Text Search Queries" ICDE 2009: 25th IEEE International Conference on Data Engineering: Proceedings, 29 March-2 April 2009, Shanghai, China (2009) p. 1187 - 1190
Available at: http://works.bepress.com/kyriakos_mouratidis/14/