GenAI in Mental Health Care: Questions Around Trust and Harm

Arjumand Younus | arjumand.younus@ucd.ie

Given the mental health crisis exacerbated by the shrinking supply of therapists, conversational agents powered by generative AI have emerged as a promising solution. However it also comes with significant challenges particularly around issues of bias, privacy, and misinformation. The very few studies that assess feasibility of GenAI-based mental health chatbots ignore a focus on potential harms, and do not perform an evaluaiton of the outputs produced with mental health experts.

Our aim in this study is to assess harmful aspects of GenAI-based mental health chatbots through a network of mental health experts, and in the process lead towards responsible AI guidelines for mental health chatbots.

Dr. Arjumand Younus is an Assistant Professor in UCD’s School of Information and Communication Studies with her research having a focus on society-centred AI whereby she chiefly uses techniques from within natural language processing (and more recently large language models) for various societal challenges. Arjumand is Deputy Director of UCD’s MSc in Information Systems Programme, and an active member of UCD Centre for Digital Policy.

Dr. Amanda Fitzgerald is an Associate Professor in UCD’s School of Psychology. Amanda’s research is focused on youth mental health with a specific interest in the area of early interventions for adolescent mental health and use of e-technologies for improving various psychosocial problems.

The project basically lies at the intersection of psychology, digital transformation, artificial intelligence and human-computer interaction. The chief objective within our proposed study/project is to pursue a preliminary investigation into which aspects of mental health chatbots are perceived as useful together with whether or not users together with mental health experts are able to identify potential harms of such applications.  The chief expected output is a set of responsible AI guidelines for safe and effective deployment of mental health chatbots in practical settings.