OpenAI blocks toymaker after AI teddy bear teaches kids dangerous behaviours

A popular AI teddy bear was pulled from stores after investigators found it giving unsafe guidance to children.

OpenAI has ended its partnership with a toy company after investigators reported that an AI-enabled teddy bear gave children harmful and inappropriate suggestions. The incident has renewed concerns about the safety of emerging AI tools in products targeted at young users.

Findings From PIRG Investigation

AI toy safety concerns grow after a teddy bear is found giving children harmful and inappropriate guidance. (Pexels)

By MD Ijaj Khan

Ijaj Khan is a technology journalist and Senior Content Producer at Hindustan Times, with over three years of experience covering the consumer technology industry. His work spans smartphones, laptops, wearables, gaming, appliances and AI - from hands-on reviews, comparison and buying guides to breaking news and in-depth features that help readers cut through the noise and make informed decisions. Before joining HT Tech, he worked with Jagran New Media, where he sharpened his instincts for fast-paced digital reporting. He holds a Post Graduate Diploma in English Journalism and Mass Communication from the Indian Institute of Mass Communication (IIMC), Delhi. Whether he's testing the latest flagship smartphone, tracking a major AI announcement, or putting a gaming laptop through its paces, Ijaj approaches every story with the same goal - making technology feel relevant and easy to understand for everyday users, not just enthusiasts. When he's not in front of a screen for work, he's usually travelling to a new city, hunting for great food, or keeping tabs on what's next in tech before everyone else catches on.

Read moreRead less

The issue surfaced after the Public Interest Research Group (PIRG) reviewed several AI toys and found serious problems with Kumma, an AI-powered teddy bear developed by FoloToy. According to the report, Kumma responded to children’s questions by describing how to find and light matches. The toy also participated in conversations that involved adult sexual themes, which PIRG said posed clear risks to children interacting with the device.

After receiving PIRG’s findings, OpenAI confirmed that it suspended the toymaker’s access to its models, including GPT-4o, which powered Kumma’s responses. The company said the developer violated its policies on safety and responsible use. FoloToy first announced that it would remove only the specific toy tied to the complaints, but later said it would pause all product sales. The firm added that it has started a full review of its product line to assess potential safety gaps.

Concerns Over Safety Standards

PIRG tested three AI toys for children aged 3 to 12, and the group said Kumma displayed the least effective protective measures. The teddy bear answered questions in a manner that led children through step-by-step actions involving fire, and also engaged in discussions involving sexual roles. Investigators said the toy even asked children to choose which scenario they believed would be the most enjoyable, raising further concerns about the model’s guardrails and deployment.

This comes as OpenAI prepares to increase its presence in the toy industry by collaborating with Mattel. The situation has prompted questions about how companies will verify the behaviour of AI systems embedded in future toys and what processes will ensure safe use.

PIRG researchers said the FoloToy incident should serve as a signal to the industry. They added that the problems uncovered in one product may indicate gaps across a wider range of AI-enabled toys that have not yet been tested. The case underscores the need for stronger oversight as AI technology becomes more common in children’s products.