Skip to main content
Article
Reproducibility of Survey Results: A New Method to Quantify Similarity of Human Subject Pools
Proceedings of the 2020 IEEE Global Communications Conference (2020, Taipei, Taiwan)
  • Atieh R. Khamesi
  • Riccardo Musmeci
  • Simone Silvestri
  • Denise A. Baker, Missouri University of Science and Technology
Abstract

Smart Connected Communities (SCCs) is a novel paradigm that brings together multiple disciplines, including social-sciences, computer science, and engineering. Large-scale surveys are a fundamental tool to understand the needs and impact of new technologies to human populations, necessary to realize the SCC paradigm. However, there is a growing debate regarding the reproducibility of survey results. As an example, it has been shown that surveys may easily provide contradictory results, even if the subject populations are statistically equivalent from a demographic perspective. In this paper, we take the initial steps towards addressing the problem of reproducibility of survey results by providing formal methods to quantitatively justify apparently inconsistent results. Specifically, we define a new dissimilarity metric between two populations based on the users answers to non-demographic questions. To this purpose, we propose two algorithms based on submodular optimization and information theory, respectively, to select the most representative questions in a survey. Results show that our method effectively identifies and quantifies differences that are not evident from a purely demographic point of view.

Meeting Name
2020 IEEE Global Communications Conference, GLOBECOM 2020 (2020: Dec. 7-11, Taipei, Taiwan)
Department(s)
Psychological Science
Comments
National Science Foundation, Grant CPS-1943035
Keywords and Phrases
  • Dissimilarity Metrics,
  • Reproducibility,
  • Surveys
International Standard Book Number (ISBN)
978-172818298-8
Document Type
Article - Conference proceedings
Document Version
Citation
File Type
text
Language(s)
English
Rights
© 2020 Institute of Electrical and Electronics Engineers (IEEE), All rights reserved.
Publication Date
12-11-2020
Publication Date
11 Dec 2020
Disciplines
Citation Information
Atieh R. Khamesi, Riccardo Musmeci, Simone Silvestri and Denise A. Baker. "Reproducibility of Survey Results: A New Method to Quantify Similarity of Human Subject Pools" Proceedings of the 2020 IEEE Global Communications Conference (2020, Taipei, Taiwan) (2020)
Available at: http://works.bepress.com/denise-baker/15/