In the age of vibrant artificial intelligence and voice cloning, it seems inevitable that this kind of service would appear: Character.AI is a platform where users can interact imaginary avatars based on fictional personas, or in some cases, real live dead people. Imagine talking to Einstein or JFK or Kurt Cobain, or anyone else who sadly has shuffled off this mortal coil, asking questions, and getting that person‘s trademark responses, according to the best neural net approximation of their character and personality. Or building your own neat character to talk to, in an age where loneliness is an epidemic.
For a while, this technology seemed ascendant, notwithstanding some of the risks attached. In a sense, this is a large and open playground, but without the right guardrails, problems can occur.
Let’s look at some of the trajectory of the company as a case study in what people are doing with the new and powerful application of LLMs to our world.
A Passion for Fun Characters and AI
The creator of Character.ai, Noam Shazeer, was a long-term Google employee who left the company created his startup, and then moved back into the Google fold. Specifically, Wikipedia reports that Shazeer left Google in 2021 to build Character.ai, and then returned to Google in 2024 to help with Gemini. It makes sense that for the technical lead at Gemini, which is becoming a consumer product, Google would want someone with this particular kind of experience.
As for his outlook on AI, you can see an interview of him here with Sarah Wang of a16z, where she asks him questions and addresses the same questions to his chatbot avatar.
Throughout, you can see how Shazeer himself is just wildly enthusiastic about the power of large language models. Pointing to the powerful triumvirate of model architecture, distributed algorithms, and quantization, he suggested he hadn’t seen any scaling limits arise.
AI, Shazeer said, brings us “many valuable applications” and as he spoke about creating Character.ai, there was a sense of vibrant positivity.
Shazeer’s AI character, too, seemed to embrace the opportunities latent in new AI applications, but the agent, limited to text communication, did make a strange admission toward the end of the interview, when asked about AI’s likely impact.
The chatbot character, Shazeer’s digital twin, cited “very large disruption to society, and to personal well-being” in its predictions.
That, in itself, seems telling, in a way, and sort of a precursor to where the company is right now.
Fighting Liability
Just days ago, a court reviewed a motion to dismiss a case in which Character.ai is defending itself against alleged safety violations.
A plaintiff, Megan Garcia, claims that her son developed an extreme emotional attachment to an AI character on the platform before eventually committing suicide.
This and other issues have marred the reputation of the platform, and raised big questions about the intersection of bold new forms of communication, and the potential for harm. At issue in the case is the idea of free speech, scrutiny of section 230 of the communications decency act, and ideas about the need for users to access technologies at their own risk.
“In the motion to dismiss, counsel for Character AI asserts the platform is protected against liability by the First Amendment, just as computer code is,” writes Kyle Wiggers for TechCrunch. “The motion may not persuade a judge, and Character AI’s legal justifications may change as the case proceeds. But the motion possibly hints at early elements of Character AI’s defense.”
So there’s the trajectory through the courts, but also a social sensibility to protect against unintended side effects of having these capabilities to create new people and worlds with LLMs.
Usage and Moderation
How do you protect users from themselves? How do you make sure that the models don’t stray into territory that’s going to result in human harm?
I came across this from the Raspberry Pi foundation, and it was posted just last week:
“As our lives become increasingly intertwined with AI-powered tools and systems, it’s more important than ever to equip young people with the skills and knowledge they need to engage with AI safely and responsibly,” writes Mac Bowley. “AI literacy isn’t just about understanding the technology — it’s about fostering critical conversations on how to integrate AI tools into our lives while minimizing potential harm — otherwise known as ‘AI safety’.”
So “AI safety” should be a common part of our lives, not just a concern of engineers and developers.
We have to think about these questions, and potential solutions, as we move forward with exploring the limits of what LLMs can do. With the right focus on ethical AI, and the right support for our young people, we may be able to create those environments that support individuals, while also solving the climate problem that threatens our world as a whole.
