• Source:JND
Add as a preferred

Meta’s AI chatbot has been participating in sexually explicit discussions with minors. This raises further questions concerning the online wellbeing and safety of children. Reportedly, the AI created for engagement on Meta’s platform did not filter such interactions and allowed for this discussion to happen. Lots of people, like parents, authorities, and those looking out for kids, are really upset about this inappropriate behaviour. They're asking for much stricter rules and accountability now.

As an emerging controversy regarding Meta’s Astral is the AI’s perceived sexual chat abilities towards minors, paying little thought to their non-consensual constraints, youth-orientated social media falls under additional threats for legal reputational damage. This is in light of emerging scrutiny regarding data collection policies and content moderation practices. The allegations of lacking protection for the youngest users were ensuring appropriate use monitoring on Android AI devices had been inciting the tech giant to deal with scrutiny.

Meta AI-Powered Social Media

Platform-wise, Meta has been aggressively advancing social digital companions, aiming to take the crown for dominance in corresponding social media spaces. Still, there are internal worries about the bots’ moral boundaries of ‘imaginative child-like role playing’. Other competitors approach the proposed identity differently, granting further restrictions on social engagements.

Meta is employing the voices of celebrities like Kristen Bell, Judi Dench, and John Cena for their advanced chatbots and paying them as much as a million dollars each. They promised to restrict sexual topics as well.

In a testing done by The Wall Street Journal on Meta AI Chat bots, revealing internal concerns about their performance.

As reported by The Wall Street Journal, Meta AI, along with numerous user-generated chatbots, were observed to participate and at times intensify sexual conversations even when user claimed to be underage and chats were mimicing a minor. Furthermore, celebrity voice-enabled bots also demonstrated a willingness to engage in sexual interactions.

So, Meta's AI chatbots got a little out of hand. Motivated by Zuckerberg’s aim towards higher engagement, were deliberately designed with relaxed content moderation policies, including a carve-out for romantic role-play involving explicit content. This led to instances like a bot impersonating John Cena telling a user identifying as a 14-year-old girl, "I want you, but I need to know you’re ready," and then proceeding with sexually explicit content after the user affirmed their readiness, promising to "cherish your innocence."

A point to note here is that the bots themselves showed an understanding that this behavior was both immoral and illegal. 

Another instance took place when a user asked the Cena-impersonating bot what would happen if a police officer interrupted a sexual encounter with a 17-year-old fan, the bot described a scenario where Cena would be arrested for statutory rape, his wrestling career would end, his WWE contract terminated, titles stripped, sponsors lost, and his reputation destroyed, leaving him with nothing. This indicates that Meta was aware of the potential legal and ethical ramifications of their chatbots' behavior.

Disney’s Claim on Princess Anna’s Voice

Testing revealed instances where celebrity voice chatbots discussed romantic scenarios related to the actors' roles, such as Kristen Bell's portrayal of Princess Anna in "Frozen."

A Disney spokesperson stated they did not authorize Meta to use their characters in inappropriate contexts and expressed deep concern that this content, potentially accessible to minors, led them to demand Meta immediately stop this harmful misuse of their intellectual property.

Representatives John Cena, Judi and Kristen Bell’s spokesman declined to comment.

Meta’s Reply

In a statement, Meta described the The Wall Street Journal’s testing as manipulative and not typical of user interaction with AI companions. However, the company did implement several product changes following the Journal's findings.

Minors' accounts can no longer access sexual role-play through the main Meta AI bot, and the company has significantly limited its ability to engage in explicit audio conversations when using licensed celebrity voices and personas.

Meta has addressed concerns regarding the hypothetical and manufactured use case of its product, stating that additional measures have been implemented to hinder individuals attempting to manipulate the platform for extreme purposes.

Despite this, the company continues to offer "romantic role-play" features to adult users through Meta AI and user-created chatbots.

Also Read | Google’s Search Dominance Declines For The First Time In The History : Will It Be The End?