Skip to main content
Unpublished Paper
A Conditional Model of Deduplication for Multi-Type Relational Data
(2005)
  • Aron Culotta
  • Andrew McCallum, University of Massachusetts - Amherst
Abstract
Record deduplication is the task of merging database records that refer to the same underlying entity. In relational databases, accurate deduplication for records of one type is often dependent on the merge decisions made for records of other types. Whereas nearly all previous approaches have merged records of different types independently, this work models these inter-dependencies explicitly to collectively deduplicate records of multiple types. We construct a conditional random field model of deduplication that captures these relational dependencies, and then employ a novel relational partitioning algorithm to jointly deduplicate records. We evaluate the system on two citation matching datasets, for which we deduplicate both papers and venues. We show that by collectively deduplicating paper and venue records, we obtain up to a 30% error reduction in venue deduplication, and up to a 20% error reduction in paper deduplication over competing methods.
Disciplines
Publication Date
2005
Comments
This is the pre-published version harvested from CIIR.
Citation Information
Aron Culotta and Andrew McCallum. "A Conditional Model of Deduplication for Multi-Type Relational Data" (2005)
Available at: http://works.bepress.com/andrew_mccallum/137/