Honey, I shrunk the data: The company reducing the size of videos, not the quality

Can you get a quart into a pint pot? You’d think not, but finally iSize has worked out how. At least as far as a quart of video is concerned. Honey, I shrunk the data. I was able to speak to Sergio Grce and Yiannis Andreopoulos, CEO and co-founder and CTO respectively, and find out how it’s done.

His mother is Croatian, his father Serbian. He lived in Brazil and was educated in London at the Cass Business School. He went to work for Merrill Lynch and RBS. But investment banking was never going to be quite enough for Sergio Grce: “I didn’t have the heart for finance. I was more into art.” In 2014 he gave up his six-figure salary in his comfortable corporate world and set up Fusion Art Works, creating art using AI, and discovered there was a demand for very large prints of Picasso, which posed an interesting challenge.

“The algorithm produced only a small picture, so we had to upscale,” says Sergio. “The question was how to stretch the image without losing the quality.”

But videos are only a series of individual frames, so Sergio was led to wonder if he could use AI to make smart videos. He initially experimented with shooting film around Lausanne in Switzerland. “It was beautiful and creative, but it was not smart – we were making video with huge amounts of data.” And this is where the great pivot in his thinking occurred: he found himself asking the question, was it possible to retain the same quality and yet minimise the amount of information?

Fortunately, in 2018, Sergio met Yiannis. “We’re like fire and water,” says Sergio. But they were thinking along the same data-shrinking lines and they clicked. Yiannis Andreopoulos is professor of data and signal processing systems at University College, London. Born in Greece, he did a post-doc at UCLA, then came to London in 2008.

Yiannis became first consultant at iSize and then CTO in 2019. “There’s a lot of effort in AI generally to replace hand-crafted with data-driven systems, in bio-sciences and pharmacology and finance,” says Yiannis. “But very little in video delivery.”

Traders on the floor of the stock exchange are increasingly being replaced by machines.

But AI has not made the same impact in video, which is still based on relatively old tech, pre-AI. “It’s like old cars polluting the world,” says Sergio. “What we’re doing – it’s like a hybrid car – and we’re working on producing a fully electric car.”

Yiannis’s research has zoomed in on “perceptual processing” – exactly how our neural networks make sense of film. “The viewer doesn’t see all the information,” he points out. “There is more information than the eye can process. A lot of the detail is filtered out by our brains.” The secret to the iSize methodology is to filter it out before the image even reaches the eye. Yiannis gives a simple example: “We leave the face alone but we change the T-shirt. It only has to be true-seeming. We hallucinate a lot of detail.”

Ed Catmull, found of Pixar, gives a good example of the kind of thing Yiannis is talking about in his book Creativity Inc. Catmull found that his animators for Ratatouille were producing massively detailed drawings of each dish that Remy the rat conjures up, and he had to go around slimming them down. They were overcooked – overly calorific, visually speaking – and the point was to take out a lot of the fat.

Yiannis and Sergio and iSize want video to go fat-free, for very good reasons. It will save a shedload of money, but also save the planet. Possibly the most misleading word in the online realm is “cloud”. To restate what everybody knows but perhaps forgets: there is no cloud. That is a metaphor, verging on a myth. In reality, there are only immense data centres, vast warehouses populated by processors and disk drives that require a lot of cooling.

“Now so many people are working at home, and more people are consuming film,” says Yiannis. “We have to work hard to avoid collapsing the network. The current crisis is a big opportunity to do something about the problem of data overload.”

open image in gallery

In 1992, global internet traffic was 100 gigabytes a day. By 2022 it will be 150,700 gigabytes – a second. Video accounts for nearly 80 per cent of internet traffic. Netflix alone accounts for 15 per cent of downloads. And 10 hours of HD film uses more data than all the articles in English on Wikipedia. “It’s getting worse as people binge-watch,” says Sergio. “There’s so much traffic – it’s become an elephant.” The point is that it’s not cost-free. There are consequences.

The internet is now using up so much electricity it has become a major contributor to climate change. According to the Royal Institute of Technology in Sweden, the internet is using up 10 per cent of the world’s electricity and emitting as much CO2 as the pre-Covid aviation industry. Online porn consumers are literally heating up the planet.

“Look at what happened to audio,” says Sergio. Hi-fi was producing a level of sound beyond what the human ear was capable of appreciating. MP3 and digital audio brought the bit rate down by between five and 20 times. “People were sceptical at first because they feared a loss in quality. But it managed to deliver audio to the masses via earphones or the web.”

MP3 works by exploiting the limits of the human auditory system, and iSize works by taking account of the intricacies of our visual processing, how we compute shapes and textures. So you get reduction of data without perceptible loss of quality. “At the moment,” says Yiannis, “we are treating all pixels equally. But we shouldn’t. Not all pixels are equal. We differentiate pixels.”

The catch-22 of existing compression software is that it uses more power, so you offset any gains you make – iSize reduces the bitrate by between 20-40 per cent. “This could reduce internet traffic by a huge 25 per cent,” says Sergio, thereby cutting energy use and mitigating the damage caused by our addiction to the image. “It’s good for the environment and it’s good for keeping networks running.”

iSize works with existing systems or, in their phrase, it’s “backward compatible”. Sergio says they can take all five seasons of Breaking Bad and compress them. Sounding a little bit like Heisenberg, he says: “We reduce the bits, we don’t break with standards, and we enhance quality.”