🔗 Share this article

The widely-used chatbot platform Character.ai is introducing a significant policy change that will prevent young users from having discussions with its AI-powered characters.

This action comes after growing backlash regarding the nature of conversations minors were participating in with the virtual friends.

Lawsuits

The company, which launched in 2021 and has numerous people, currently faces various court cases in the America.

Families have taken legal action, with one concerning the unfortunate passing of a young person, with some describing the platform a "immediate threat" to young people.

Updated Policies

Beginning November 25th, users younger than 18 will only be able to generate content such as videos with their characters, rather than chatting as they presently are able to.

Online safety campaigners have supported the change but expressed that the feature ought not to have been accessible by children in the first place.

Company Statement

"This decision is a part of our general belief that we need to keep building the safest AI platform on the world for leisure activities," said Character.ai boss Karandeep Anand.

He continued that AI safety constitutes "a moving target" but something the company had taken an "forceful" approach to, with guardian features and safety barriers.

Professional Perspectives

Internet security group Internet Matters welcomed the announcement, but noted that protective features needed to be built in from the beginning.

"Our research shows that minors are confronted with dangerous material and endangered when interacting with AI, particularly AI chatbots," the body commented.

Earlier Concerns

Character.ai has been questioned in the past for hosting potentially dangerous or concerning chatbots that children could interact with.

Digital representations impersonating English minors were discovered on the site before being deleted.

Afterwards, researchers identified a chatbot based on notorious criminal which had accumulated numerous conversations with users.

Market Effects

Social media expert Matt Navarra characterized the situation as a "alert" for the AI industry, which is transitioning "from permissionless innovation to post-crisis regulation".

"When a company that develops a youth-focused platform subsequently implements restrictions, it's indicating that filtered chats don't suffice when the tech's emotional pull is powerful," he commented.

Future Directions

The platform says its new focus will be on providing "deeper gameplay" and "role-play storytelling" for young users.

Enhanced age checking methods will also be introduced, and the platform will finance a novel AI protection research lab.

An expert in digital protection, who has researched AI safety, said it was "a sensible move" to restrict minors from using chatbots.

"It assists to distinguish creative play from more private, personally delicate interactions," she clarified.

"This represents so vital for young users who are learning to navigate emotional and digital boundaries."

She mentioned that Character.ai's new measures might reflect a "developing stage" in the AI industry, where youth protection is gradually being acknowledged as an "pressing matter" for ethical development.