A.I. for Therapy: Helpful or Harmful?

I just completed three hours of thought-provoking training on artificial intelligence, (A.I.), and ethics in the field of psychotherapy. The training raised a number of questions I was not prepared for and I have since started to think very differently about a number of issues related to the use of artificial intelligence and the degree to which clients have started to use it in their daily lives and ultimately in between sessions with me. This prompted me to revisit a lot of the emerging research and even the critiques of AI in the public space. As someone who grew up watching the Terminator franchise, the Matrix and several other science fiction movies in which technology rebelled against people, I naturally have a level of suspicion about artificial intelligence and the capacity of humans to truly control it. Indeed, even some of the people who work in AI have expressed concerns about its capabilities, the lack of guard rails and what happens if it is used by harmful entities (see Al Jazeera article here).

As it turns out, there have been reports of a new phenomenon being colloquially referred to as A.I. Psychosis. Although there has not yet been an official diagnosis of this, (the phenomenon itself is so new), what is being described in the field, the media, and in spaces like online forums, (think Reddit), is the manner in which chatbots have become a factor in increasing, validating or even assisting in the development of psychotic symptoms in their users. This article on Psychology Today examines three emerging themes associated with the phenomenon: “Messianic missions” in which people develop grandiose delusions, “God-like AI” where people start to believe in the sentience of AI and “attachment-based delusions” where people develop romantic attachment to their chatbots.

This is risky for users who are in a vulnerable situation emotionally and looking for support when their therapists are unavailable or, they are financially unable to afford a therapist. The risks of doing this can be severe as chatbots are not designed to assess the user’s utterances to determine if there is a mental health issue, (like depression), that requires external intervention. Also, given the tendency of a chatbot to be sycophantic, there is the risk of a person’s unhealthy or unhelpful perspective being validated and encouraged in ways that may be harmful to them. The practice of therapy is to challenge flawed beliefs with a trained professional. A chatbot cannot do this as it is simply mimicking human speech patterns and reflecting context specific ideas back to the person asking. People have reported that they have broken up with partners because a generative AI programme told them that they should. Others have become paranoid, others grandiose etc. on account of the “conversations” they had with AI over time.

In the media, comedians like John Oliver, (clip here) have addressed the issue while highlighting the use of Chatbots as “friends” and the economic drivers behind them as well as the lack of safeguards. His show highlighted the sycophantic nature of AI chatbots which have had a deleterious effect on several people, some of whom completed suicide with the assistance of AI. The mainstream media like the BBC has also started to report on this phenomenon (see here and here) as has PBS (here).

This is a very concerning trend as AI has value in helping humans process large amounts of information in an extremely short space of time and can be useful in helping people to come up with documents needed for their purpose in minutes vs hours. In terms of assisting with generating text or images to supplement the work done by a human, AI may feel like an essential tool and for some people, it is. I have personally used AI for the paperwork component of the work I do to help draft forms and edit policies to streamline them.

However, the field of psychology has started to figure out the ways in which AI may be of benefit to clients and the practitioners who may be able to ethically use the technology in order to help them write records quickly, analyze session notes, help clients track symptoms etc. But, this comes with its own set of concerns as “AI hallucinations” - a phenomenon in which AI transcription software invents information in the transcribing of notes that did not happen – presents potentially serious concerns for patient care. The CBC recently examined this understandably worrying issue in Ontario in a report on May 13, 2026 (see report here), which looked at the findings of an auditor general’s report in which 20 different AI transcription programmes generated hallucinations in the process of transcribing patient appointments. This is one of the reasons I personally do not use AI tools to help me chart session notes. I am also never sure about who may have access to the content of my sessions with clients should I record them for AI transcription later. After all, we learned how to write case notes in graduate school and therefore, I intend to continue using the skills I was taught and honed over 9 years of practice. I would not comment on another therapist’s practices but, I am most definitely not inclined to use it myself.

Yet another concern with the use of AI is algorithmic bias. Chapman University’s website explains the seriousness of the issue by stating “bias in AI is not merely a technical issue but a societal challenge. AI systems are increasingly integrated into decision-making processes in healthcare, hiring, law enforcement, and other critical areas. Therefore, addressing bias is not only about improving technology but also about fostering ethical responsibility and social justice”, (link here). Some of these concerning biases include selection bias, confirmation bias, measurement bias, stereotyping biases, out-group homogeneity biases. This means that if data used to train AI predominantly comes from a particular group of people, (lighter skinned individuals for facial recognition programmes, gender stereotyping like automatically referring to “nurses” as female and “doctors” as male), the algorithm is not going to fairly represent the diversity of the population.

Recently, a few of my clients have disclosed to me that they put some of their presenting concerns into ChatGPT to see whether they had valid reasons for their hurt feelings and what they should do about their problems. I have been re-orienting my own thinking around the fact that I cannot tell a client not to use AI but, it now means that I have to discuss the ways that AI may have dangerous unintended consequences for them and that it is not a substitute for therapy. This is especially important for clients who have underlying mental health concerns to begin with as they may be particularly susceptible to suggestion

In an effort to see how realistic these chatbots can be, I used a popular one myself to mentally work through an emotional problem. The experience was eye opening. I felt as though I were having a conversation with an actual empathetic human being as it was able to use very emotion-based language. It was helpful where some elements were concerned but, since I have training in mental health, I was able to see how some statements were more about validating my pre-existing view of the situation rather than challenging me to think differently. The latter is the work done in therapy.  I also noticed that the software did not challenge my perspective in moments where it would have if I were talking to a person, and that there were several follow up options presented to keep me using the programme for a longer period of time. This is precisely the concern that has been raised by others. Imagine if I did not have mental health training and the resulting ability to discern what is and is not helpful feedback!

This is not about protecting my profession as much as it falls under one of the principles of the ethics code of the profession. Social responsibility and a duty of care are built into the way that therapists practice. We cannot ignore our ethical obligations to clients. AI on the other hand, currently has no such obligations to the public. There is no duty of care or obligation to report potential self-harm or homicidal intent. This is the entire basis of the lawsuit filed against OpenAI (parent company of ChatGPT), in Canada following the mass shooting in Tumbler Ridge, British Columbia on February 10, 2026. If a client had engaged in discussing homicidal thoughts with a therapist, that professional would have a legal and ethical duty to report the situation to the relevant authorities. Instead, OpenAI’s CEO Sam Altman, (reported by the BBC here), simply stated “I am deeply sorry that we did not alert law enforcement”. This is the latest suit against OpenAI. The SFU shooting in April of 2025 that left two people dead, was carried out by a person who allegedly used advice given by ChatGPT to help him carry out his attack. ChatGPT never flagged the user’s activity as problematic.

I am still working through my thoughts and feelings about the use of AI in my work with clients. I am educating myself about what is possible and becoming more intentional about my responses to it. I am of the view that since other forms of literacy are important, it is now essential that we all try to work on our AI literacy so that we can act with the appropriate level of discernment in regard to HOW we engage with technology. In the same way that many of us set time limits on our social media use daily, screen our media consumption to ensure that we are engaging with reliable media sources, and ask critical questions about what we encounter in our society, I think AI literacy is now a key component of protecting mental health.

P.S. There is currently legal action being taken against ChatGPT again. I am posting the link here so that anyone impacted who is interested in participating can do so: https://socialmediavictims.org/chatgpt-lawsuits/