The ghouls turned the murdered child into a Character. Artificial intelligence bot on the ‘Lonely Hearts’ platform was held responsible for the child’s suicide

Character.AI, a platform that allows people to talk to various AI chatbots, has come under fire after one of its bots allegedly goaded a young boy into killing himself earlier this year.

A new lawsuit filed this week alleges that 14-year-old Sewell Setzer III was talking to a Character.AI friend she had a crush on when she committed suicide in February.

In response to a request for comment, Character.AI told Daily Mail.com that they were “creating a different experience for users under 18 that includes a stricter model to reduce the likelihood of encountering sensitive or sexually explicit content.”

But Character.AI faces a number of other controversies, including ethical concerns regarding user-created chatbots.

Drew Crecente lost his teenage daughter, Jennifer, who was shot and killed by her ex-boyfriend in high school in 2006.

Eighteen years after her murder, she learned that someone had resurrected her as a character on Character.AI, using Jennifer’s name and likeness.

A spokesperson told DailyMail.com that Jennifer’s character has been removed.

There are also two Character.AI chatbots using George Floyd’s name and likeness.

Floyd was killed by Minneapolis police officer Derek Chauvin, who pressed his knee on his neck for more than nine minutes.

‘This Character was created and removed by the user,’ Character.AI said in a statement to DailyMail.com.

‘Character.AI takes security seriously on our platform and manages Characters proactively and responsively to user reports.

‘We have a dedicated Trust and Safety team who review reports and take action in line with our policies.

‘We also conduct proactive detection and monitoring through a variety of means, including industry standard block lists and custom block lists, which we expand regularly.

‘We are constantly evolving and evolving our security practices to help prioritize the safety of our community.’

‘We are working quickly to implement these changes for younger users,’ they added.

As the loneliness epidemic grips the country, Sewell’s death has raised questions about whether chatbots that act as texting buddies are doing more to help or harm. young people who use them disproportionately.

However, there is no doubt that Character.AI has helped its founders, Noam Shazeer and Daniel de Freitas, currently achieve spectacular success, wealth and acclaim in the media. Both men were named in a lawsuit filed against their company by Sewell’s mother.

Shazeer, who graced the cover of Time Magazine’s 100 Most Influential People in the Field of Artificial Intelligence list last year, was featured in Character.AI’s ‘super, super useful‘ to people struggling with loneliness.

On March 19, 2024, less than a month after Sewell’s death, Character.AI was introduced voice chat feature It makes role-playing on the platform more lively and realistic for all users.

The company first introduced the service in November 2023 as a beta for C.AI+ subscribers, including Sewell, who pay $9.99 a month for the service.

So now, Character.AI’s 20 million global users They can speak verbally 1:1 with their preferred AI chatbots, and most of them can carry on flirting or romantic conversations. many Reddit users They confirmed it.

“I use this mostly at night, so feelings of loneliness or anxiety don’t creep up on me. It’s so nice to fall asleep without feeling lonely or hopeless, even if it’s just a little fake role play,” one person wrote in the archive. Thread on Reddit. ‘It’s not perfect because I’d rather have a real partner, but my options and chances are limited right now.’

Another wrote: ‘I use this mainly for role-playing as well as therapeutic purposes. But romance comes from time to time. I don’t care if they come from my laid-back characters.’

Shazeer and de Freitas, who were once software engineers at Google, founded Character.AI in 2021. Shazeer serves as the company’s general manager, while de Freitas is its president.

They left Google after refusing to release their chatbot, CNBC reported.

In an interview at a tech conference in 2023, de Freitas said that he and Shazeer were inspired to leave Google and start their own startup because “there’s too much brand risk in big companies to launch anything fun.”

Character.AI has enjoyed tremendous success, reaching a $1 billion valuation last year following a fundraising round led by Andreesen Horowitz.

According to a report published last month Wall StreetJournalGoogle wrote a check for $2.7 billion to license the Character.AI technology and rehire Shazeer, de Freitas, and some of their researchers.

Following the lawsuit filed over Sewell’s death, a Google spokesperson told the NYT that the licensing agreement with Character.AI did not give it access to any chatbots or user data. The spokesperson also said that Google does not incorporate any of the company’s technologies into its products.

Shazeer has become a forward-looking executive and often gives a variety of answers when asked what the company’s stated goals are.

In an interview with Axios HeadquartersShazeer said: ‘We’re not going to think about all the great use cases. There are millions of users. ‘They can think of better things.’

Shazeer also said he wants to make a personalized superintelligence that is cheap and accessible to everyone; This is a similar statement to the mission expressed on the ‘About Us’ page. Character.AI’s website.

But as part of its goal to be more accessible, Character.AI also has to deal with a lot of copyright claims, since many of its customers build chatbots that use copyrighted material.

For example, the company removed the Daenerys Targaryen character Sewell chatted with, in part because HBO and others owned the copyright.

In addition to existing scandals, Character.AI may face more criticism and legal issues in the future.

Attorney Matthew Bergman, who represented Garcia in his lawsuit against the company, told DailyMail.com that he has heard from numerous other families with children who have been negatively affected by Character.AI.

He declined to specify exactly how many families he had spoken to, citing the fact that their cases were ‘still preliminary’.

Bergman also said that Character.AI should be completely removed from the internet because it was ‘brought to market before it was secure.’

However, Character.AI also emphasized that there are ‘two ways to declare a Character’.

‘Users can do this by going to the Character profile picture and clicking the ‘report’ button, clicking on the ‘author’ option.

‘Or they can go to the Security Center and there will be a “submit request” link at the bottom of the page.’