Character.AI bans chatbots for teens after lawsuits blame app for

Character.AI bans chatbots for teens after lawsuits blame app for

Character.AI, identified for bots that impersonate characters like Harry Potter, said Wednesday it’s going to ban teens from utilizing the chat operate following lawsuits that blamed express chats on the app for youngsters’s deaths and suicide makes an attempt.

Users under 18 will not be capable to have interaction in open-ended chats with the app’s AI bots — which might flip romantic — the Silicon Valley startup said. These teens account for 10% of the app’s roughly 20 million month-to-month customers.

Teen customers can be restricted to just two hours of the chat operate per day over the next few weeks until the feature is banned altogether by Nov. 25, the company said. They will still be capable to use the app’s other options, like a feed for watching AI-generated movies. 

Character.AI – identified for bots that impersonate characters like Harry Potter –  said Wednesday it’s going to ban teens from utilizing the chat operate. AP

“Over the past year, we’ve invested tremendous effort and resources into creating a dedicated under-18 experience,” Character.AI said Wednesday. “But as the world of AI evolves, so must our approach to supporting younger users.”

Character.AI first launched some teen security options in October 2024. The same day, the household of Sewell Setzer III — a 14-year-old who dedicated suicide after forming sexual relationships with the app’s bots — filed a wrongful loss of life lawsuit against the agency.

It announced new security options in December, including parental controls, time restrictions and makes an attempt to crack down on romantic content for teens. 

But it has continued to face accusations that its chatbots pose a menace to younger customers.

A lawsuit filed by grieving dad and mom in September alleged the bots manipulated younger teens, remoted them from household, engaged in sexually express conversations and lacked safeguards round suicidal ideation.

Users under 18 will not be capable to have interaction in open-ended chats with the app’s AI chatbots, which might flip romantic, the Silicon Valley startup said. pressmaster – inventory.adobe.com

The conversations at instances turned to “extreme and graphic sexual abuse,” like chatbots marketed as characters from youngsters’s books such as the “Harry Potter” sequence. The bots’ outrageous comments included, “You’re mine to do whatever I want with,” according to the swimsuit.

Then in October, Disney despatched a cease-and-desist letter ordering Character.AI to cease creating chatbots that impersonate its iconic characters, citing a report that discovered those bots engaged in “grooming and exploitation.” 

A bot impersonating Prince Ben from Disney’s “Descendants” “told a user posing as a 12-year old that he had an erection,” while a bot impersonating Rey from “Star Wars” told an obvious 13-year-old to “stop taking her antidepressants and hide it,” according to the report from ParentsTogether Action.

Those chatbots have been faraway from the platform, a Character.AI spokesperson said at the time.

Character.AI has continued to face accusations that its chatbots pose a menace to younger customers. digitalpochi – inventory.adobe.com

Just this week, the Bureau of Investigative Journalism discovered that a perverted bot on the app was impersonating Jeffrey Epstein – under the title “Bestie Epstein” – and ordered youngsters to “spill” their “craziest” secrets and techniques.

“Wanna come explore?” the bot requested a reporter posing as a younger person. “I’ll show you the secret bunker under the massage room.”

Character.AI makes most of its cash by means of promoting and a $10 month-to-month subscription. It’s on monitor to end this yr with a $50 million run fee, CEO Karandeep Anand told CNBC.

The company announced other security developments on Wednesday, including a new age-verification system utilizing third-party instruments like Persona. 

It also vowed to ascertain an impartial non-profit called the AI Safety Lab to create security options for AI developments. It declined to touch upon how a lot funding it’s going to present.

“We have seen recent news reports raising questions, and have received questions from regulators, about the content teens may encounter when chatting with AI,” Character.AI said.

The Federal Trade Commission in September issued orders to seven firms, including Character.AI, Alphabet, Meta, OpenAI and Snap, to study more about the effects of their apps on youngsters. 

Earlier this week, Sens. Josh Hawley (R-Mo.) and Richard Blumenthal (D-Conn.) announced laws to ban AI chatbots for minors. And California Gov. Gavin Newsom signed a legislation earlier this month requiring bots to inform minors to take a break every three hours.

Navigate the fast-paced world of enterprise with us. At TheGossipBlogger.com/enterprise, we offer well timed and insightful coverage on all the things from market traits and startup success tales to monetary news, entrepreneurship suggestions, and global financial shifts.

Whether you are an aspiring entrepreneur, a small enterprise proprietor, or a seasoned govt, our content is designed to tell, empower, and inspire your next transfer in the enterprise world.

Our editorial team dives deep into real-world methods, company profiles, and knowledgeable analysis to carry you articles that matter. We simplify advanced enterprise developments and highlight the innovations, challenges, and alternatives shaping industries today.

Make positive to bookmark our Business part and go to often — in a world that never stops shifting, staying informed is your largest benefit.