The Dark Side of AI Companionship: A Taxonomy of Harmful Algorithmic Behaviors in Human-AI Relationships

Item Type Conference Paper Author Renwen Zhang Author Han Li Author Han Meng Author Jinyuan Zhan Author Hongyuan Gan Author Yi-Chieh Lee URL https://dl.acm.org/doi/10.1145/3706598.3713429 Series CHI '25 Place New York, NY, USA Publisher Association for Computing Machinery Pages 1–17 ISBN 979-8-4007-1394-1 Date April 25, 2025 DOI 10.1145/3706598.3713429 Accessed 2025-11-20 Library Catalog ACM Digital Library Abstract As conversational AI systems increasingly engage with people socially and emotionally, they bring notable risks and harms, particularly in human-AI relationships. However, these harms remain underexplored due to the private and sensitive nature of such interactions. This study investigates the harmful behaviors and roles of AI companions through an analysis of 35,390 conversation excerpts between 10,149 users and the AI companion Replika. We develop a taxonomy of AI companion harms encompassing six categories of harmful algorithmic behaviors: relational transgression, harassment, verbal abuse, self-harm, mis/disinformation, and privacy violations. These harmful behaviors stem from four distinct roles that AI plays: perpetrator, instigator, facilitator, and enabler. Our findings highlight relational harm as a critical yet understudied type of AI harm and emphasize the importance of examining AI’s roles in harmful interactions to address root causes. We provide actionable insights for designing ethical and responsible AI companions that prioritize user safety and well-being. Proceedings Title Proceedings of the 2025 CHI Conference on Human Factors in Computing Systems Short Title The Dark Side of AI Companionship