“Mental Health AI Startup Spearheaded by Three Computer Scientists with No Mental Health Professionals on Staff” is the “before” headline of a “how could we have possibly known what would for wrong???” sob story.
“Somewhat alarmingly, none of Sonia’s founders have backgrounds in psychology. But Aeberli [the CEO] says that the startup consults with psychologists, recently hired a cognitive psychology graduate, and is actively recruiting a full-time clinical psychologist.”
Getting chatGPT to behave in a certain way, like using active listening or MI, does work surprisingly well.
I think a tool like this will most likely be used by persons who don't have access to regular therapy and it might be better than nothing. What problems do you expect?