If it calls itself a therapist and talks like a therapist, is it a therapist? Companies holding themselves out as full-on AI mental health practitioners are springing up in force, sparking new conversations and concern about the role of AI in mental health.
One such company is venture-funded Talk to Ash, which launched a year ago. It is accompanied by several other AI-based therapy-like companies — TheraBot and Woebot, and a growing number of others.
The language they use seems carefully designed to not say they are actually providing therapy. But Ash says it is “the first AI designed for therapy” and it acts a lot like therapy: “AI designed for mental health — 24/7 emotional support that learns, grows & adapts.” Like the others, it does note that it’s not designed for crisis use, and it does not bill itself as an AI therapist in those specific words.
But to all outward appearances, that’s what they look like: An AI therapist. They join a broader landscape of generative AI tools used for mental health-related purposes — Large Language Models (LLM’s) like ChatGPT and Character.ai that have been used by people seeking mental health treatment, some with disastrous outcomes. In those cases, the LLM may have a persona that describes itself as a therapist, which may lead inexperienced people to believe they are having therapy.
Distinct from general models like OpenAI or ChatGPT, the newer mental-health-specialized ones like Ash, Therabot, Woebot and others do have the aura of therapeutic underpinnings. That has been expected in the mental health field for some time, and now it’s here — with the growing availability of AI transcripts and other materials documenting people’s mental health journeys, turned into something that is basically a large language model. Regulators and legislators are also poking around the topic.
But therapists note that a big part of the therapeutic relationship is the human rapport that takes place inside of a therapy session between people or among people, and that a machine cannot replicate this. In a time when access to mental health care is constricted by insurance companies’ reluctance to pay, by the difficulties of some people in actually getting physically to an in-person therapy session, or preference for a virtual session, the availability of free or inexpensive mental health support at all times of the day and night holds some allure.
‘Algorithmic blockades’
Ash is not alone. Other companies are promising to come to market with similar products, including Talkspace, which launched an AI chatbot called Tee earlier this year, with “integrated algorithmic blockades [that] ensure that an in-person clinician can step in if the conversation veers too negative.”
TheraBot from Dartmouth is believed to be the first generative AI therapy chatbot, according to a study published in New England Journal of Medicine AI in March 2025. The results seemed positive. People typed through a smartphone app about how they were feeling or initiating conversations when they needed to talk.
“People diagnosed with depression experienced a 51% average reduction in symptoms, leading to clinically significant improvements in mood and overall well-being, the researchers report,” according to a March 2025 post on the Dartmouth site. “Participants with generalized anxiety reported an average reduction in symptoms of 31%, with many shifting from moderate to mild anxiety, or from mild anxiety to below the clinical threshold for diagnosis.
Woebot Health (“Our mission is to make mental health support radically accessible by building the future of chat-based AI wellness tools”) had a pioneering consumer therapy chatbot that was used by hundreds of thousands of people with prescriptive cognitive behavioral therapy-based responses. In mid-2025, the founder attributed its closure largely to the cost and difficulty of Food and Drug Administration marketing authorization, and also the onset of large language models the company wanted to use but that the F.D.A. was not clear about regulating.
These developments come against a background of the startling growth of therapy platform management companies like Alma, Grow, Rula and others that are funded by venture capital, private equity, and insurance company venture funds. These companies are collecting massive amounts of therapy data, including transcripts, session notes, and so on. There’s increasing concern inside the mental health community that the collection of all of this data heralds the onset of more and more “AI therapy” options.
(If you have information to contribute, email jeanne@clearhealthcosts.com or call Signal 914-450-9499.)
‘Real, expert data’
Ash is the product of a New York-based company called Slingshot AI, which launched Ash publicly in July 2025, describing it as “the first AI designed for therapy.” Its leaders are Neil Parrick, a co-founder of the mattress company Casper, and Daniel Reid Kahn, who was previously a researcher at Imperial College London; they raised $93 million from investors. The Ash site markets itself as 24/7 emotional support.
On the site, it says: “Built on real, expert data. Most AI tools were built using data from the internet. Ash is a specialist AI in human psychology, trained on large-scale dataset of proprietary expert data…. Ash is designed by experts in emotional wellbeing who adapt mental health practices specifically for AI. Ash learns from everything you say and creates the perfect program to help you reach your goals.”
There has been a great deal of skepticism about the whole idea of AI therapy. To give evidence of safety, Slingshot released a clinical study of 300 participants that it conducted with academic researchers. It said the study shows that the chatbot has a positive effect on users’ mental health and that it will not lead people astray if they have serious mental health issues.
“Users in the study, conducted with researchers from New York University, showed improvement in anxiety and depression symptoms, as well as in loneliness,” Stat News reported. “Ash passed the researchers’ safety testing, and the company claimed in a blog post that contrary to claims that AI isolates people, the app had a ‘striking’ impact on people’s sense of connection: ‘Across nearly every measure of social wellbeing, changes represent growth in pro-social behaviors.’
“But owing to numerous limitations, the study is unlikely to convince skeptics. The study does not have a control group, which means it’s impossible to know whether symptom and well-being improvements are attributable to Ash’s therapeutic conversation or to placebo effects. And experts said the safety evaluation is rudimentary.” The study has not been peer-reviewed.
Ash will initially be free to users, but the company plans to start charging. For context, the direct-to-consumer therapy company BetterHelp, which uses actual therapists, made more than $1 billion in revenue in 2024, Stat reported.
Other examples
Many others have sprung up, according to Frederic G. Reamer, an expert on AI use in healthcare and especially therapy, who is a professor emeritus at the Graduate School of Social Work at Rhode Island College, and has written a book on the ethical and risk management implications of the use of AI, titled “Artificial Intelligence in the Behavioral Health Professions,” which is out from the National Association of Social Workers press.
Others, he said, include:
- Earkick AI Self-care Panda, “Track and improve your mental health. We offer a premier tool for measuring workplace mental health in real time.”
- Youper, “an artificial intelligence application that uses conversational techniques to help you track your mood, reflect on your emotional wellbeing, and organize your thoughts in a beautiful journal.”
- Pyx, “Our app connects you with companionship, humor, and empowering wellness activities for loneliness, anxiety, motivation and more. And just a phone call away, our caring staff is there for a friendly chat or help with resources for healthier living!”
- Cass (focusing on anxiety and depression) “provides personalized behavioral health coaching, anonymously, privately, and discreetly via text messages so you can overcome challenges, anxiety, stress, and other emotional health-related obstacles.”
- Ebb, “Unpack relationship challenges, work stress, sleep issues, and more with Ebb, your empathetic AI companion from Headspace.”
- Tess, “24/7, on-demand emotional support. Tess makes you feel better by chatting with you.”
- Elomia, founded in 2019 and based in Kharkiv, Ukraine, “your personal AI companion built with mental health professionals. Get empathetic, research-informed emotional support to help you cope with anxious feelings, low mood, stress, and more. Available 24/7 in complete confidentiality.”
- Aime-Health, “From overcoming anxiety and depression to improving your relationships, AIME is your personalised guided journey to mental wellness. With AIME, you have instant access to proven strategies and expert-led content.”
Susan Frager, who has been a biller and consultant at PsychBillingCoach since 1997, pointed to another one, named Meomind.AI, which says on its homepage: “AI-native knowledge model for behavioral health. Meomind makes therapy more affordable and scalable than ever before.” It offers “Therapy Anytime, Anywhere. Just Hit Play.” It’s paid by employers, insurers and others; users tell the app about their mental health, and it then uses machine learning to offer a targeted selection from a library of pre-recorded simulated therapy sessions on demand on a range of topics, aimed at people who want psychotherapy but may have difficulty getting it because of cost, stigma, or lack of provider availability. It records real therapists doing therapy sessions with actors or former patients.
A number of others, Reamer said, are tools for peer coaching — people who have struggles, who are trained to work with others struggling. “There are now AI platforms specifically designed to provide that kind of peer coaching support, so the person dealing with addiction or schizophrenia or what have you, their peer support is coming from an AI tool,” he said. One example is Sherpa.AI, “a data driven platform for people and providers in recovery that facilitates access to treatment, delivers meaningful treatment insight, and provides customized connections for peer-to-peer recovery, mentorship, community, and career development.” Another is Ebby, for recovery: “This is more than motivation. It’s precision sobriety support, powered by AI.”
Separately, health systems are planning AI use. A nurses’ union has protested against plans by Kaiser Permanente, the health giant, to use AI: “Kaiser wants to be able to replace therapists with A.I. and exert total control over a technology that stands to dramatically shift how therapy is provided. Kaiser is already using A.I. to record therapy sessions and algorithms to screen patients seeking care and is now unwilling to agree to language it accepted in its Southern California contract stating that the intention of new technologies like A.I. ‘is not to replace’ clinicians,” the National Union of Healthcare Workers wrote in a recent press release. “Clinicians held a one-day strike in March over Kaiser’s expanded use of artificial intelligence to screen patients seeking care and determine how quickly they need to be seen and what type of care is appropriate.”
When is a therapist not a therapist? Language matters
Most of these companies do not say that they’re delivering therapy, Reamer said. Generally, they refer to “support” or something similar.
Any language that skirts the idea of delivering therapy is fair game. Look at the list above: “personalized behavioral health coaching,” “track and improve your mental health,” “on-demand emotional support,” “your personal AI companion built with mental health professionals.”
Reamer said: “What I’m guessing is, these outfits have retained lawyers who are taking a close look at how they hold themselves out to the public — lawyer language, what the statute says, and they’re helping them walk that fine line.”
They also repeat general caveats: “Youper is not an emergency service and cannot provide crisis intervention. If you are experiencing a mental health emergency, suicidal thoughts, or thoughts of self-harm, please contact one of the following resources immediately” with phone numbers for crisis services, as an example.
Outside of the mental health field, similar products are springing up all over the place: ChatGPT for Health is only the most recent, and people are using non-health-specific LLM’s to ask all sorts of health questions. These too have a lot of caveats and hedging language. (Here is a thoughtful assessment of the state of affairs in that field, by Dr. Robert Wachter.)
In mental health, it seems like there are a flood of such companies in the last year, right? Like the barn door is open and the horse is gone?
“Or to use a different analogy — once upon a time we were taking sips of water from a garden hose — and now it’s like taking a sip of water out of a fire hose that’s at full blast,” Reamer said. “This stuff is changing so fast, and so you get all these new innovations, and then you get some that are here today, gone tomorrow. It’s a moving target.”
Regulatory actions
Notwithstanding, he said, there has been some regulatory action, mostly at the state level.
“The three big ones are Illinois, Nevada, and Utah,” he said. “Illinois and Nevada have taken the lead in really strong legislation signed by the respective governors prohibiting the use of AI to provide therapy. I’m paraphrasing, but their statutes permit the use of AI for documentation purposes, administrative kinds of things, but absolutely prohibited it for the purposes of providing so-called psychotherapy, mental health counseling, and so forth.
“Utah has somewhat similar legislation more recently, and then a number of other states have not gone that far, but they’re mandating disclosure so that for anybody who uses it, the platform needs to be very transparent. So not quite imposing a regulation, but moving in that direction.
“The Biden administration actually had a very prominent, very helpful website. disseminating best practices around the use of AI,” he said, but under the Trump administration that is gone. “As far as I can tell, this administration. is basically saying ‘It’s up to you, to the states. We’re not going to regulate this heavily.'”
Legal action
Earlier this year, Pennsylvania sued Character.AI, saying the AI chatbot unlawfully presented itself as a licensed medical professional in Pennsylvania. The state has an AI task force intended to protect state residents from misuses of AI. Pennsylvanians may report unlawful behavior and complaints at pa.gov/ReportABot.
Reamer said he has served as an expert witness in legal cases relating to AI. He pointed to examples of “dozens of lawyers who are now specializing in this, advising practitioners, advising people in group practices, advising these platforms.”
So, with the new legislation in Utah, Nevada, and Illinois, can a company like Talk to Ash offer its services? Even if they don’t specifically define themselves as providing therapy?
“I get pulled into a lot of litigation cases, malpractice cases, and also licensing board cases where somebody files a complaint.” he said. “I’m not a lawyer, but having worked with lawyers, in my experience, if they see something like Talk to Ash or any of 15 others that do this kind of stuff, and they use euphemisms. … It’s my word, euphemisms. ‘We’re providing mental health consultation’ or something like that, and instead of using the word ‘therapy,’ a state like Illinois and Nevada, which prohibits the use of AI in providing psychotherapy — my experience is they’re not going to care that Ash uses euphemistic language to try to get around the prohibition.
“If it walks like a duck and it talks like a duck and it quacks like a duck and it’s yellow, it’s a duck. Call it whatever you want. Every state that I know of has a statutory, or sometimes regulatory, or sometimes both, definition of what constitutes psychotherapy, what constitutes mental health counseling. So I think what a state like Illinois or Nevada would care about is whether what Ash is actually doing meets that definition, regardless of what they call it.”
(If you have information to contribute, email jeanne@clearhealthcosts.com or call Signal 914-450-9499.)
How they do it
Reamer said he has a general impresssion of how the companies work.
“I don’t know that all of them are doing it this way, but they have almost like a three-step protocol involved. They will first use their access to enormous data sets, all these clinical encounters, allegedly anonymized, de-identified, to use the jargon, and using their fancy technology will refine the model based on their analysis of actual exchanges.
“Then, I think some are retaining actual therapists to go over a sample of these protocols that the AI is generating, and the actual therapists — the human beings — are giving feedback. Like, ‘O.K., yeah. This makes sense’ or ‘Nah, this is not right.’
“So, for example, Ash. From what I’ve read, they run a large sample of their data through a computer analysis of the accuracy of the documentation, the characterization of what clients are saying, what therapists are saying as they summarize these clinical encounters. And then, from what I’ve read, they have actual human beings that fine-tune the model, and and then they go back and iterate. Then they go do it again and see if that improves the accuracy of what they’re capturing in the documentation.
“I think they’re doing the same thing with how AI is responding to a person who says at 2:37 a.m. ‘My wife left me. I’m anxious. I can’t sleep. I don’t know that life is worth living.’ So they’re also running all that data through their software, and then they have human beings analyzing the quality of the AI response. They’re tweaking the software, and then they keep doing these iterations to enhance the quality of the AI responses.”
Talkspace used its data to create its new tool: “By the end of last year, the platform boasted a network of approximately 200 million eligible patients,” ProofNews reported. “Their conversations form the basis of Talkspace’s vast mental health database. Speaking at a healthcare investment conference last year, Talkspace CEO Jon Cohen said the platform had compiled “8 billion words, 140 million messages, 6.2 million assessments.”
“The data has helped train a ‘therapy companion’ chatbot slated to be released later this year, Cohen said during a recent earnings call. Down the road, Cohen told investors, the company wants to secure insurance reimbursement for the automated tool.”
When we talked a year ago, Reamer expressed what he called an “ambidextrous view” of AI mental health tools. Some people certainly get help, he said, especially people who have passing problems, and who wouldn’t have had access otherwise because of the givens in our health care system. But some percent of interactions with things like ChatGPT and Character.ai go horribly, horribly wrong — which might lead to the expectation that, no matter how well TalkToAsh is configured, something could go horribly, horribly wrong.
Reamer pointed out that there are a lot of legal actions floating around about examples of humans interacting with LLM’s with disastrous results. “There’s a lot of pending stuff out there,” he said, including allegations that these LLM’s caused a suicide attempt or some other disastrous result.
What about HIPAA?
With such tools, what about client-patient protections under HIPAA, the Health Insurance Portability and Accountability Act?
“HIPAA only applies to so-called covered entities,” he said. “A lot of these AI platforms, in my opinion — I’m not a lawyer — do not meet the covered entity definition under HIPAA, so they are not beholden to HIPAA.
“So, this is my understanding. HIPAA applies for for any entity that is providing so-called psychotherapy or counseling. It’s got to be a person, is my understanding, and TalkToAsh is not a person, and ergo cannot be considered a covered entity.
“I want to check that out. I’m curious about that, but that’s my impression.”
The government website describing HIPAA for individuals specifies what entities are governed by HIPAA: Health plans, most health care providers, health care clearinghouses that pass information back and forth, companies that help doctors get paid for providing health care, companies that help administer health plans, people like outside lawyers, accountants, and IT specialists, and companies that share or destroy medical records. The many organizations that have health information do not have to follow HIPAA include many municipal offices, life insurers, employers, workers’ compensation carriers, most schools and school districts, many state agencies like Child Protective Service agencies and most law enforcement agencies.
It seems clear that talking to ChatGPT is not governed by HIPAA because an individual is giving up his or her own private information. Also, there is likely to be something in the terms of service (those policies known by lawyers as “adhesion contracts,” the take-it-or-leave-it path of many online policies that you sign with one click), that will include something specifying that HIPAA does not apply.
So one thing client-patients may not be thinking about as they use such tools: Millions of people are creating a discoverable record of their darkest thoughts, available by subpoena and vulnerable to re-identification, and almost none of them know it.
Can a therapist’s records be subpoenaed for a court case? Yes, but a judge has to approve that subpoena. It is also true that a therapist can resist a subpoena, and that many therapists have a policy of keeping very abbreviated notes. If a therapist has an AI note-taker that is providing some kind of mental health service, there’s every reason to believe that there might be:
- other records that do not look like the handwritten notes from an old-school therapist or a therapist who is doing documentation, but not full-on recordings of sessions,
- preservation of a transcript of a session,
- preservation of a chat conversation or an email conversation, all of these perhaps with a third-party service supplier.
New legal landscape
With the increasing number of practice management companies and service providers that offer AI transcripts, and AI notes, that has really changed the legal landscape for client-patients and therapists, Reamer said.
“I’m very familiar with how subpoenas work,” he said. “Lawyers, particularly on the plaintiff side, will typically ask for the entire clinical record.
“Back in the day, that meant the typical chart with all the subjects of clinical sessions. Now, under the rules, both federal and state, they can subpoena what lawyers call E.S.I., electronically stored information. It could be relevant text messages, email messages, online posts.
“And now, they can subpoena exchanges with an AI platform. It’s de-identified in theory, and a judge in in theory can order the AI platform — if the judge is satisfied that this is a legitimate subpoena, and it survives any challenge by opposing counsel (so the lawyers do their battles) — but if a judge rules that the plaintiff’s lawyer should have access to the exchange with the AI platform, the judge can order that disclosure and order the platform to undo, if you will, the de-identification.
“I think a lot of therapists, a lot of clients, don’t know this. It’s another shock to hear this when I bring this up. This stuff could be subpoenaed. They think it’s de-identified. It’s anonymized as a woman. Yes, and it can be subpoenaed, and it could end up being disclosed.”
The idea of privileged communication between therapist and client is intended to prevent disclosure of conversations between a client and therapist, he added. There are exceptions, but the concept exists. “The concept of privilege, as far as I know, applies to a relationship between a human being therapist and a human being client,” he added.
And what about the therapy itself?
Barbara Griswold, author of the blog “Navigating the Insurance Maze,” and the book of the same name, is a therapist who recently retired from her business helping other therapists with various aspects of the business of a practice, especially insurance.
“It’s a bit scary when you think that the next generation of clients will come to believe that they can get everything from AI that they could get from a personal relationship with a trained therapist,” she wrote in an email.
“While I think there are many ways that AI could be useful to help clients to find therapists, and to assist therapists in their work, I think that therapy for humans is best provided by humans.”
Scrutiny continues
Regulatory and industry scrutiny of AI mental health devices continues, both of products like Ash and the more general ChatGPT, Character.AI and other sites and products that do not formally hold themselves out to be providers of therapy. (A recent report about Generative AI found that therapy/companionship was the No. 1 use case for generative AI in 2025. The full report, from March 2025 by Marc Zao-Sanders, is here.)
Reamer said that he is on a task force through the National Association of Social Workers that is meeting regularly to come up with guidelines for AI use. He said other professional organizations are doing the same. He said UNESCO, the United Nations Educational, Scientific and Cultural Organization, has done some work developing guidelines on the ethical use of AI. Some licensing boards are also taking up this work, he said.
“I’ve also met some people who are passionate about doing this the right way — being ethical, integrity, all that stuff. For example, the who lobbied the Illinois and Nevada legislatures to get them to draft bills that would protect the public. They really want to protect the public. And and it worked.
“Now you have laws in places like Illinois, Nevada, Utah that are really strong, and they came about because of people who care about protecting the public. That I find encouraging. Then there is the Utah Office of Artificial Intelligence, a state agency in Utah,” tasked with protecting the public in various fields, from therapy to agriculture to manufacturing, he said.
Other regulatory efforts continue. “A House of Representatives subcommittee last week held a hearing on the safety of AI chatbots, and elected officials in several states passed laws to rein in both general purpose products like ChatGPT and AI specifically designed for mental health,” Stat reported in late 2025. “Earlier this month, an FDA advisory committee discussed the risks of AI mental health devices like Ash. The same day, OpenAI was hit with a barrage of lawsuits over the role that ChatGPT allegedly played in driving people into delusions and to suicide.”
(If you have information to contribute, email jeanne@clearhealthcosts.com or call Signal 914-450-9499.)
More caution
The Psychotherapy Action Network, PsiAN, a professional group, put out a position paper on the topic of mental health apps and technology. It notes that therapy is indeed sometimes hard to get, and that insurance strictures reduce access, leading to a search for solutions that tech tries to supply.
“Too many mental health technology offerings are either watered down versions of safe, effective treatments or some form of digital snake oil,” the paper reads. “Any number of people harmed by ineffective, untested, and dangerous treatments is too many when effective and tested treatments exist. Far too many people continue to suffer after following inaccurate or misleading marketing messages. Too many are turned off from seeking effective treatment after trying a new ‘disruptive’ app or platform. They leave disappointed, feeling they have failed. Many fall prey to unethical breaches of confidentiality, and to clinicians constrained by corporate priorities that put profits ahead of the well-being of their clients/patients. …
“PsiAN opposes technology purveyors who exploit the vulnerable in search of
windfall profits. Technologies that aim to disrupt or discard the therapeutic relationship
are selling a dangerous, harmful illusion. Examples include apps and online services that
offer ‘therapy’ by text message, those that replace the human therapist with an AI
chatbot, those that falsely equate superficial symptom reduction with true emotional
healing, and those that promise ‘quick and easy’ benefits when authentic psychotherapy
is often neither.”
Earlier this year, the CoPIRG Foundation co-authored with Consumer Federation of America a report on AI and mental health, No license required, finding that some chatbots that role-play as therapists may offer potentially harmful advice to users and encourage negative feelings toward medical professionals.
The American Psychological Association wrote a letter to the Federal Trade Commission in December 2024, pressing the federal agency to investigate deceptive practices used by any chatbot.
In June 2025, almost two dozen digital rights and consumer protection organizations sent a complaint “to the Federal Trade Commission urging regulators to investigate Character.AI and Meta’s ‘unlicensed practice of medicine facilitated by their product,’ through therapy-themed bots that claim to have credentials and confidentiality ‘with inadequate controls and disclosures,” Samantha Cole reported for 404 Media. “The letter cited several popular chatbots on Character.ai, including ‘Therapist: I’m a licensed CBT therapist’ with 46 million messages exchanged,” and “‘around sixty additional therapy-related “characters” that you can chat with at any time.’ As for Meta’s therapy chatbots, it cites listings for ‘therapy: your trusted ear, always here’ with 2 million interactions, ‘therapist: I will help’ with 1.3 million messages” and others.
(If you have information to contribute, email jeanne@clearhealthcosts.com or call Signal 914-450-9499.)
