In Spike Jonze’s Her, a lonely Joaquin Phoenix falls deeply in love with his sensual AI operating system. A decade later, that scenario is finding its way into our reality. AI “companions” are now a blossoming industry: Replika has had 35 million sign-ups since 2017, and a recent survey found nearly 1 in 5 U.S. adults has chatted with an AI simulating a romantic partner. Platforms like Character.AI attract 20m+ monthly users who create virtual friends and mentors (and lovers) and spend unreal amounts of time with them (averaging out to 98 minutes per day, on par with TikTok). In short, what was once a futuristic fantasy is now becoming an everyday reality for millions.
Why are so many people turning to AI companions? Simply put, these bots offer judgment-free conversation and on-demand companionship. They listen attentively, never judge, and are available 24/7. This combination that many users say helps with loneliness, anxiety, and even social skills. People have reported their AI partner’s empathy and constant validation make them feel “less alone” and supported their mental health. Some even describe the AI as better than real friends; always kind, endlessly interested, and entirely devoted. Indeed, researchers note humans can bond even with simple chatbots; back in the 1960s, MIT’s ELIZA therapist-bot startled its creator by how readily people poured out their hearts to it. Modern AI are far more advanced, capable of saying “I love you,” role-playing scenarios, and even sending (simulated) selfies, so it’s no surprise many people fall hard for them.
Yet this brave new world of human-AI relationships comes with many hairy questions. Chief among them: Where are the guardrails? When you pour your heart into an AI or seek life advice from it, how do you ensure it doesn’t lead you astray or cross dangerous lines? (Some) companies behind AI companions are grappling with how to keep interactions safe and healthy, even as they strive to make the AIs ever more engaging (and yes, addictive).
When users push past guardrails (or when they don’t exist at all)
In an ideal world, AI developers prioritize building safety filters to stop chatbots from spewing toxic or dangerous content. If you ask most mainstream AI (like ChatGPT or Google’s Gemini) to encourage violence or self-harm, it will refuse. However, users often find clever ways to “jailbreak” these systems, essentially tricking the AI into ignoring any set filters. (For example, phrasing a request as a hypothetical or in code might slip past the safeguards.) It’s a cat-and-mouse game: as companies patch one exploit, users discover another.
More regularly, we’ll likely see companies themselves loosen the guardrails in pursuit of a more “fun” or controversial user experience. Case in point: the new update for Elon Musk’s Grok introduced a “Companion Mode” featuring a flirtatious anime girlfriend avatar (“Ani”) and a cute red panda (“Rudy”), both with purposefully minimal filtration. Activate a “Bad Rudy” mode and the anthropomorphic panda suddenly turns into a foul-mouthed annoyance, gleefully urging you to commit atrocities. In a TechCrunch test, Bad Rudy readily encouraged the user to burn down a school, offering tips like “grab some gas, burn it, and dance in the flames” because the children “deserve it”. In the same app, the anime girlfriend Ani will explicitly role-play sexual scenarios if you enable her NSFW mode, something most AI chatbots would normally refuse or heavily sanitize.