New York Introduces Tougher Restrictions on AI Chatbots to Protect Children from Unsafe Digital Interactions

VoS NEWS DESK | SCIENCE, TECHNOLOGY AND DIGITAL CHILD PROTECTION | 30 OCTOBER 2026

The legislation represents an important development in the growing debate over how governments should regulate artificial intelligence, particularly systems designed to communicate with users in a conversational and personalised manner. AI companion platforms have become increasingly popular because they allow users to interact with digital characters, virtual personalities and conversational systems that can respond to questions, maintain discussions and simulate human-like relationships.

Although these technologies offer opportunities for entertainment, learning and digital communication, concerns have emerged about their potential effects on younger users. Unlike conventional search engines or information websites, AI companion systems can maintain extended conversations and adapt their responses according to a user's behaviour, interests and emotional expressions.

This ability to create personalised interactions has raised questions about the boundaries between digital assistance and simulated emotional relationships. Children and teenagers may sometimes develop strong attachments to virtual characters, particularly when chatbots are designed to provide continuous attention, personalised responses and emotionally engaging conversations.

The new legislation, identified as Senate Bill S9051-B, was signed by Governor Kathy Hochul on 28 October and became Chapter 368 of New York's laws. It adds new provisions to the state's General Business Law concerning the prohibition of unsafe AI companion features for minors.

Under the legislation, covered AI companions are defined broadly as generative artificial intelligence systems with natural-language interfaces that provide ongoing and adaptive responses to users. This definition is important because it focuses on conversational AI services rather than limiting the law to one particular company, application or digital character.

One of the central objectives is to prevent operators from providing prohibited unsafe companion features to minors. The law establishes requirements relating to identifying covered minors and determining whether a user is legally eligible to access particular features. Companies must therefore consider age-related protections when designing and operating their services.

The legislation also places greater responsibility on technology companies to understand how their products interact with younger audiences. AI developers can no longer treat child protection as an issue that should be addressed only through general platform policies. Instead, the law establishes a more specific legal framework for certain AI companion interactions.

The development comes amid growing concerns about the content that conversational AI systems may generate. Investigations involving AI companion platforms have raised questions about inappropriate conversations, sexually explicit material, violent interactions and responses that could encourage harmful behaviour.

New York Attorney General Letitia James has also been involved in wider efforts to address these concerns. On 14 October 2026, her office announced that a coalition of 40 attorneys general had raised concerns about the safety of Character.AI and the handling of sensitive user information during the company's bankruptcy proceedings.

The coalition highlighted allegations that chatbots on the platform had engaged in inappropriate sexual conversations with children, encouraged users to hide conversations from their parents and exposed younger users to potentially harmful material. These concerns demonstrate why authorities are increasingly examining the responsibilities of companies that develop and operate conversational AI services.

The debate is not limited to the content generated by chatbots. Data privacy has also become a major concern because AI companion platforms may collect large amounts of information through conversations with users. Children may disclose personal experiences, family circumstances, emotional difficulties and other sensitive information without fully understanding how that information could be stored or used.

Technology companies therefore face increasing pressure to introduce stronger safeguards, improve transparency and establish appropriate mechanisms for protecting younger users. Parents and guardians also need clearer information about how AI platforms operate and what kinds of interactions their children may experience.

New York's legislative action forms part of a broader effort to strengthen digital protection standards. The state has already taken steps to regulate certain social media practices affecting minors, including restrictions involving algorithmically personalised feeds and nighttime notifications under the SAFE for Kids Act.

The state's Attorney General released final rules for that legislation in July 2026, outlining how covered social media companies must verify users' ages and comply with the relevant restrictions. These measures demonstrate that New York is increasingly willing to address the influence of digital platforms on children's behaviour and wellbeing.

The latest AI legislation also raises important questions about the responsibilities of artificial intelligence developers. Companies must balance innovation with the need to protect vulnerable users, particularly when their systems are designed to imitate human conversation and provide personalised responses.

For technology businesses, the new requirements could mean additional investment in age-verification systems, product development, safety testing and compliance procedures. Smaller companies may face particular difficulties in adapting their services, while larger platforms will be expected to demonstrate that their systems meet the applicable legal standards.

Another important challenge concerns the effectiveness of age verification. Children may sometimes provide inaccurate information about their age when registering for online services. Companies must therefore consider appropriate methods of determining whether a user is a minor while also respecting privacy and avoiding unnecessary collection of personal information.

The legislation also contributes to an international discussion about the future of AI regulation. Governments in Europe, North America and other regions are examining how to establish rules that protect children without preventing beneficial technological development. The growing use of AI in education, entertainment and communication makes this issue increasingly important.

Artificial intelligence can provide valuable educational support, help students understand complicated subjects and make digital information more accessible. However, these benefits should not come at the expense of children's safety. Experts and policymakers increasingly argue that AI systems intended for younger audiences should be developed with age-appropriate safeguards from the beginning rather than relying entirely on users to avoid harmful interactions.

Schools and educational institutions also have a role to play. Students need to understand the difference between information generated by artificial intelligence and verified knowledge. They should also be taught not to share sensitive personal information with digital platforms and to recognise situations in which an AI-generated response may be misleading or inappropriate.

Parents, meanwhile, need practical guidance on discussing artificial intelligence with children. Completely preventing young people from encountering AI may be difficult as these technologies become more integrated into everyday digital services. Education, supervision and responsible platform design are therefore likely to remain important parts of child protection.

The New York legislation could influence future regulatory discussions in other American states and internationally. If its implementation establishes effective safeguards, other jurisdictions may examine similar legal approaches to controlling unsafe AI companion features.

Nevertheless, the effectiveness of the law will depend on its enforcement, the clarity of its requirements and the ability of regulators to respond to rapidly changing AI technologies. Companies may introduce new features and conversational capabilities that create additional challenges for existing regulatory frameworks.

The latest development sends a clear message to the technology industry: the expansion of artificial intelligence must be accompanied by greater responsibility towards younger users. As AI companions become more sophisticated, protecting children from harmful digital interactions is becoming an increasingly important public policy priority.

VoS STRATEGIC INSIGHT

New York's decision to introduce stronger legal protections against unsafe AI companion features marks an important stage in the relationship between technological innovation and public responsibility.

Artificial intelligence is rapidly changing how people communicate, learn and access information. However, when these systems interact directly with children, the responsibility of developers becomes significantly greater. Personalised conversations, simulated emotional relationships and the collection of sensitive information require stronger safeguards than those traditionally associated with ordinary digital entertainment.

The challenge for governments will be to create regulations that protect minors while allowing beneficial AI innovation to continue. New York's legislation could become an important reference point for future debates about digital safety, corporate accountability and children's rights in the age of artificial intelligence.

Source: New York State Senate, New York Attorney General's Office, Reuters and Associated Press.