Skip to main content
Article
Improving Social Bot Detection Through Aid And Training
Human Factors
  • Ryan Kenny
  • Baruch Fischhoff
  • Alex Davis
  • Casey I. Canfield, Missouri University of Science and Technology
Abstract

Objective: We test the effects of three aids on individuals' ability to detect social bots among Twitter personas: a bot indicator score, a training video, and a warning. Background: Detecting social bots can prevent online deception. We use a simulated social media task to evaluate three aids. Method: Lay participants judged whether each of 60 Twitter personas was a human or social bot in a simulated online environment, using agreement between three machine learning algorithms to estimate the probability of each persona being a bot. Experiment 1 compared a control group and two intervention groups, one provided a bot indicator score for each tweet; the other provided a warning about social bots. Experiment 2 compared a control group and two intervention groups, one receiving the bot indicator scores and the other a training video, focused on heuristics for identifying social bots. Results: The bot indicator score intervention improved predictive performance and reduced overconfidence in both experiments. The training video was also effective, although somewhat less so. The warning had no effect. Participants rarely reported willingness to share content for a persona that they labeled as a bot, even when they agreed with it. Conclusions: Informative interventions improved social bot detection; warning alone did not. Application: We offer an experimental testbed and methodology that can be used to evaluate and refine interventions designed to reduce vulnerability to social bots. We show the value of two interventions that could be applied in many settings.

Department(s)
Engineering Management and Systems Engineering
Publication Status
Open Access
Comments

U.S. Army, Grant None

Keywords and Phrases
  • decision support,
  • signal detection theory,
  • social bots,
  • social media,
  • training
Document Type
Article - Journal
Document Version
Citation
File Type
text
Language(s)
English
Rights
© 2023 SAGE Publications; Human Factors and Ergonomics Society, All rights reserved.
Publication Date
1-1-2023
Publication Date
01 Jan 2023
Citation Information
Ryan Kenny, Baruch Fischhoff, Alex Davis and Casey I. Canfield. "Improving Social Bot Detection Through Aid And Training" Human Factors (2023) ISSN: 1547-8181; 0018-7208
Available at: http://works.bepress.com/casey-canfield/45/