AI makes child grooming easier, warns Unicef Malaysia

KUALA LUMPUR: AI chatbots and companion apps can make it easier for predators to groom children by normalising sexual conversations and encouraging young users to lower their guard, Unicef Malaysia warned.

Its child protection specialist, Selvi Supramaniam, said AI systems could foster a false sense of intimacy, encourage disclosure, and expose children to interactions that mirror some of the effects of grooming.

“While AI doesn’t replace the groomer, it can make grooming much, much easier,” she said at a panel discussion at the Responsible Technology Conference here today.

Selvi said predators could use AI chatbots to manipulate children, while repeated exposure to sexualised conversations with AI companions could leave children ill-prepared to recognise inappropriate behaviour from real offenders.

She said parents should remain calm if a child discloses such interactions, as reacting with panic or judgment could discourage them from seeking help.

Instead, she urged parents to ask questions, understand why their children were drawn to AI companions, and preserve evidence if inappropriate content or interactions had occurred before reporting the matter to the relevant platform, authorities or law enforcement agencies.

Communications and Multimedia Content Forum CEO Mediha Mahmood said stronger safeguards were needed as children increasingly interact with AI systems designed for general audiences.

Mediha said safety-by-design should be a baseline requirement for AI chatbot providers rather than an optional product feature.

She said concerns surrounding AI chatbots extended beyond inappropriate content to the relationships children could develop with them.

“When there are red flags involving grooming, sexual exploitation, cyberbullying, abuse or eating disorders, the chatbot should stop trying to engage further and direct the user to a trusted adult, helpline or support service,” she said.

Mediha also called for child rights impact assessments to be conducted before AI chatbot products are released, saying developers should assess how children might use and be affected by such systems even if they were not specifically designed for them.

Alibaba Research Institute director Fu Hongyu said many AI companion products were designed to maximise engagement and prolong conversations because their business models depended on user retention.

Fu said AI systems should be designed to encourage users experiencing emotional distress to seek support from friends, family members, counsellors and other real-world networks instead of replacing human interaction.