Mattel's AI Hello Barbie may be an imaginary friend too far

You could almost sense the glee when Mattel announced in February that it was working on a doll that could not only speak, but actually understand what your child was saying. (I'm referring to the kind of glee that's fuelled by botched product launches and snowballing internet outrage.) "What could possibly go wrong?" smirked tech website The Register, as it was revealed that Hello Barbie would record your words (just like Siri or Google Now), send them across the internet (just like Siri or Google Now), analyse them (you get the idea) and send back a suitable response. The difference here, of course, is that children are involved.

Hello Barbie appeared in the shops last month, and there's been a concerted effort ever since to portray it as misguided at best, evil at worst. Its electronics and firmware have been pored over by security analysts, and a few days ago the US broadcaster NBC found an expert by the name of Matt Jakubowski who was willing to put the boot in, saying that he had hacked Barbie and got his hands on things he "shouldn't have" including "MP3 files". A bunch of people are now campaigning for her to be destroyed.

The digital footprint of kids is an emotive topic, and rightly so. It's the duty of every parent to keep tabs on internet use, and it's the duty of companies working in that field to take security very seriously. An example of how not to do things emerged last week, when the Hong Kong firm VTech managed to let the personal data of five million people slip into the hands of hackers, including details, pictures and audio files of children. But there's a difference between demonstrably poor security and the potential for a breach, because every device that connects to the internet can be breached on some level or other. Some of the criticism being levelled at Hello Barbie, however, makes it sound as if children are being lured into some kind of cult. "I would be very concerned that my child's intimate conversations… were being recorded and analysed," said one parent. A campaigner slammed parents who were "offering up sensitive information about their children… with the likelihood that [it] will end up in the hands of countless unknown parties". Needless to say, the sensitivity of that information, the nature of those unknown parties and the likelihood of the breach actually happening weren't specified.

Since NBC's report on Hello Barbie, it's emerged that the MP3s Jakubowski prised out of the device were merely recordings of Barbie's voice that are played when she can't connect to the internet. He added that an "outstanding job" had been done on her security protocols. This reassured me of the safety of this newfangled doll that can remember your favourite colour and deliver a few pre-scripted lines – but there are plenty of people, including tech experts, who are expressing doubts that AI and children are compatible, not least because speech recognition of this kind can be powered only by the cloud.

That's the deal. Their voices will travel across the internet, and the recordings – hopefully encrypted – will sit on servers in distant lands. The petition for Mattel to pull the toy, now 37,000 signatures strong, demonstrates a now-familiar technological tension: the wonder and delight we experience when an inanimate object responds to our voice is cancelled out by a constantly stoked fear that the magic will end up rebounding on us with disastrous consequences.

twitter.com/rhodri