Full service digital agency.

We create awesome digital experiences that make people smile

Get to know us

EN slash NL

SXSW 16: How we are sliding into the Matrix without a blue pill

Had a busy day? Children in bed? Time to relax. So you plug in your ‘Matrix’ jack, close your eyes and you’re lying on the beach in Ibiza. You can feel the sand between your toes, the sun on your skin. You can you can hear the sea, taste your cocktail and see the sunlight glittering on the waves. All your senses are stimulated. Science fiction? Or perhaps reality one day?

As a physicist, I’m pretty rational. But after visiting SXSW 2016 I’m prepared to bet that the scenario outlined above is at least theoretically possible. And may even become a practical reality this century. That would mean that this is something my children could well experience. How? Let me illustrate with two interesting SXSW talks.

Hack yourself

Let’s start with George Hotz. He was the first hacker who managed to hack the iPhone and PlayStation 3. Last summer Elon Musk asked him to develop the AutoPilot feature on the Tesla. He could have earned millions. Yet Hotz politely refused and decided to do it himself. He bought a Honda, ripped it apart, connected wires, grabbed a powerful computer and installed off-the-shelf cameras and other sensors. He then developed a 99% self-driving car (‘level 3’) within a couple of months using open source ‘Machine Learning’ software. In a garage. On his own. With a budget of less than $50k, of which $30k was spent on the car.

As far as he’s concerned, a car is nothing more than a system. You have input in the form of millions of camera pixels, and output in the form of two numbers: the accelerator and the position of the steering wheel. Everything in between is one function. He didn’t program that function – that would be practically impossible. There are are innumerable options, and we ourselves often don’t even know why were driving the way we are, let alone that we can summarise it in lines of code. So he decided to use a ‘Machine Learning’ algorithm. That algorithm spent 10 hours watching how Hotz drove. And then Hotz was able to hand over control on the freeway: the computer drove the car safely and comfortably. It’s a nice example of the power of machine learning.


Justin Sanchez of DARPA (the US Army’s Defence Advance Research Projects Agency) is trying to do with humans what Hotz did with his car. In their work on the direct neural interface he shows how a woman can control a robot arm with her own brain despite being paraplegic. And she can also literally feel which finger is being touched.

DARPA does this by implanting a 4 mm card in the brain. The card has a set of pins which literally talk directly to the nerves and synapses. It’s a ‘brain-computer’ interface. It’s currently still running at ‘9k6 speed’, but in 20 years it will undoubtedly be doing gigabits. Most of the components for such a brain plug are already a relative commodity, both in terms of hardware and software.

Both men’s ‘hacker mentality’ is indefatigable. They combine a deep philosophical, mathematical, ICT, biological and engineering insight with a screwdriver and a soldering iron. The combination of the diversity, quality and low cost of commodity hardware with free open source software is resulting in enormous technological progress. Developments which are literally being pieced together in garages.

Hotz and DARPA’s inventions rely on three major tech developments:

1. Virtual Reality

The Oculus Rift, Samsung Gear VR; at SXSW you’re literally tripping over VR headsets. Manufacturers are involved in a race to make the virtual world in the headset literally indistinguishable from reality. So that you believe you are really looking into the eyes of your own children. The render engines are coming – just ask Kodak. They believed that digital photography could never be true to life. They know better now.

2. Artificial Intelligence/Machine Learning

This is not about a massive, overwhelming and evil AI which will wipe out humanity. This is about smart algorithms which can ‘learn’ in a limited number of hours how driving a car works, simply by observing a human. The AI which observes the input and output signals of the nervous system and learns how the mechanical and cognitive part of the human and the brain works.

3. Internet of Things/the sensornet/Data

Tumbling prices of hardware, sensors, data storage, and the capacity of (open source) systems to handle unimaginable quantities of data provide the ‘Machine Learning’ algorithms with training sets that are growing exponentially and are available to all. And the more they train, the faster they develop themselves, so the faster they can train again, etc.

These are three ingredients for a perfect storm.

You can view human beings as a system. We have a number of inputs from the senses, processing in the brain, and outputs in the form of speech and movement. And this system can be hacked. Suppose that DARPA finally manages to divert all inputs from the nerves and intercept all outputs. What does that mean? That the VR images from your PlayStation enter your brain directly, instead of through a (currently) clumsy headset.

With the current headsets the image goes from a digital environment (e.g. the PlayStation) to vision (e.g. the Oculus Rift) to electricity (the signals in your brain). It’s a bit like connecting your iPhone to your car radio using the headphone plug: it doesn’t sound quite right. With a future Oculus Rift the VR may be almost indistinguishable from the real world. With a DARPA plug it will be really ‘real’. But then what will ‘real ‘mean? That’s a question for philosophers.

What I suggested above for images will also apply in due course for synthetic sound, scent, taste, feel. Dozens of hackers are working in all these areas: full-body suits, scent experiences, 3D sound to accompany your VR headset – the list is endless.

If all the senses can be hacked, if we can ‘render’ vision, sound, touch, taste and smell, then you can briefly disconnect your brain from your body on a long flight back from SXSW and plug in on Ibiza. Spend some time ‘in the Matrix’. That sounds better than sitting cramped in a seat gazing at the small screen. The only disadvantage would be that this article wouldn’t have been written.

March 25, 2016
Bart Manuel, Partner

his article has been published in Dutch on Emerce