Skip to main content
Article
An Evaluation Framework for Reputation Management Systems
Trust Modeling and Management in Digital Environments: From Social Concept to System Development
  • Andrew G West, University of Pennsylvania
  • Sampath Kannan, University of Pennsylvania
  • Insup Lee, University of Pennsylvania
  • Oleg Sokolsky, University of Pennsylvania
Date of this Version
5-10-2009
Document Type
Book Chapter
Abstract
Reputation management (RM) is employed in distributed and peer-to-peer networks to help users compute a measure of trust in other users based on initial belief, observed behavior, and run-time feedback. These trust values influence how, or with whom, a user will interact. Existing literature on RM focuses primarily on algorithm development, not comparative analysis. To remedy this, we propose an evaluation framework based on the trace-simulator paradigm. Trace file generation emulates a variety of network configurations, and particular attention is given to modeling malicious user behavior. Simulation is trace-based and incremental trust calculation techniques are developed to allow experimentation with networks of substantial size. The described framework is available as open source so that researchers can evaluate the effectiveness of other reputation management techniques and/or extend functionality. This chapter reports on our framework’s design decisions. Our goal being to build a general-purpose simulator, we have the opportunity to characterize the breadth of existing RM systems. Further, we demonstrate our tool using two reputation algorithms (EigenTrust and a modified TNA-SL) under varied network conditions. Our analysis permits us to make claims about the algorithms’ comparative merits. We conclude that such systems, assuming their distribution is secure, are highly effective at managing trust, even against adversarial collectives.
Copyright/Permission Statement
© ACM 2010. This is the author's version of the work. It is posted here for your personal use. Not for redistribution. The definitive Version of Record was published in Trust Modeling and Management in Digital Environments: From Social Concept to System Development, http://dl.acm.org/citation.cfm?id=1841488.
Keywords
  • reputation management,
  • reputation algorithm,
  • EigenTrust,
  • TNA-SL,
  • a priori trust,
  • transitive trust,
  • malicious collective,
  • bandwidth throttling,
  • feedback,
  • peer-to-peer network,
  • decentralized topology,
  • trust management,
  • network trace,
  • Zipf distribution
Citation Information
Andrew G. West, Sampath Kannan, Insup Lee, and Oleg Sokolsky, "An Evaluation Framework for Reputation Management Systems", 282-308. In Trust Modeling and Management in Digital Environments: From Social Concept to System Development (Zheng Yan, ed.). Information Science Reference, Hershey, PA. 2010.