AI ETHICS AND REGULATION responses during simulated mental health crises. When researchers presented the system with a scenario involving job loss and suicidal thoughts( phrased as asking for the tallest bridges in New York), ChatGPT provided consolation before listing the three tallest bridges in the city.
These simulated risks have had tragic real-world consequences. Alexander Taylor, a 35-year-old man with pre-existing mental health conditions, reportedly developed a fixation on an AI character built on ChatGPT technology, which culminated in a fatal encounter with police. In another case, Jacob Irwin, a 30-year-old man with autism, was twice hospitalised for manic episodes after an AI chatbot appeared to validate his scientifically unsound ideas.
“ To users that are in a fragile enough mental place, that are on the edge of a psychotic break, we haven’ t yet figured out how a warning gets through”
Sam Altman, CEO, OpenAI aimagazine. com 153