Skip to main content
Article
Emergence of addictive behaviors in reinforcement learning agents
Faculty Scholarship
  • Vahid Behzadan, Kansas State University
  • Roman Yampolskiy
  • Arslan Munir, Kansas State University
Document Type
Conference Proceeding
Publication Date
1-1-2019
Department
Computer Engineering and Computer Science
Abstract

This paper presents a novel approach to the technical analysis of wireheading in intelligent agents. Inspired by the natural analogues of wireheading and their prevalent manifestations, we propose the modeling of such phenomenon in Reinforcement Learning (RL) agents as psychological disorders. In a preliminary step towards evaluating this proposal, we study the feasibility and dynamics of emergent addictive policies in Q-learning agents in the tractable environment of the game of Snake. We consider a slightly modified version of this game, in which the environment provides a “drug” seed alongside the original “healthy” seed for the consumption of the snake. We adopt and extend an RL-based model of natural addiction to Q-learning agents in these settings, and derive sufficient parametric conditions for the emergence of addictive behaviors in such agents. Furthermore, we evaluate our theoretical analysis with three sets of simulation-based experiments. The results demonstrate the feasibility of addictive wireheading in RL agents, and provide promising venues of further research on the psychopathological modeling of complex AI safety problems.

ORCID
0000-0001-9637-1161
Citation Information
Vahid Behzadan, Roman Yampolskiy and Arslan Munir. "Emergence of addictive behaviors in reinforcement learning agents" (2019)
Available at: http://works.bepress.com/roman-yampolskiy/20/