In the real world, the senses of touch, taste and smell are really critical for communication
They’re both examples of augmented reality, but not the kind we think of when we imagine Google Glass or data being overlaid on your smartphone’s camera view. They’re also both the work of Adrian Cheok, a professor of pervasive computing at City University in London, who has already moved onto augmenting our reality with the other senses we take for granted in day to day life – and take for granted they cannot be replicated electronically over the web.
“The internet will become multi-sensory,” the Australian educated founder and director of the Mixed Reality Lab in Singapore predicts boldly.
“I wanted to extend the augmentation beyond audio and vision. The internet now is very much about transmitting audio visual sense, verbal information, but in the real, physical world, the senses of touch and taste and smell are really critical for communication – in fact smell and taste are directly connected to the sensor of the brain responsible for emotion and memory. Without having these sensors it’s very difficult to communicate a real sense of presence through the internet.”
Headway is being made with the sense of touch, from creating devices that allow us to feel what we are looking at such as Katherine Kuchenbecker’s work in Haptography as well as employing touch to advance our interaction with digital technology, which companies such as Sharp are constantly looking to develop – the touchscreen being an obvious example.
But I wanted to find out about digital taste and so made my way to City University to check out one of Cheok’s latest prototypes: what’s been dubbed an “electronic lollipop” that can artificially create tastes using little more than an electrical current and a micro controller. First of all, I’m shown some of the inventions that led to Cheok’s current field of research, which focus on transmitting touch and smell over a network.
There’s the RingU, the world’s first tele-hug ring, which can transmit a gentle squeeze of a finger to another ring that simulates it – a second model fits this technology along with a force sensing resistor into a system on a chip that is just 1.7cm square. There are the pyjamas that can transmit a hug, so that parents can wish their kids good night from the other side of the planet, and even a virtual kissing machine that works much the same way as with lips.
Scoff if you like, but one of these technologies has already been commercialized by a close friend of Cheok’s, Japanese businessman Koki Tsubouchi. The Scentee, an accessory that plugs into the headphone socket of your smartphone, is based on an invention by Cheok that uses a motor and several vials of scent to create different smells wherever you are, so you can nudge a friend on Facebook with a pleasant/unpleasant odour or (theoretically, and very much in the future) capture the smell of what you’re cooking and send it as a smartphone alert to a loved one. The Scentee technology is also behind one of 2014’s most popular viral marketing stunts, a video by food giant Oscar Mayer that shows a smartphone waking shoppers up with the smell of bacon in the morning.
“We’re in the age of information, you can have infinite data about anything, but it’s still very difficult to have this sense of experience,” Cheok explains. “So not just to know for example you’re in this GPS location, here’s the restaurant, here are pictures of the food, but know what it really feels like to be there, what is the taste and smell like. This will lead to also new kinds of virtual reality, new kinds of augmented reality. We’ll be able to change the way that people can communicate, not just data but experiences.”
It suddenly occurs to me that I’m licking a lemon wet wipe
It’s high time I give the prototype a taste, and see if Cheok is selling snake oil to snake oil merchants, or really able to conjure up new sensory experiences with nothing more than 1s and 0s on a small computer on the desk. Delicately I place my tongue between the two metal plates (which, fret not, have been cleaned in alcohol), and the machine is turned on.
At first it’s difficult to detect whether I can taste anything sour, or if it’s simply the metallic tang of the electrodes, but there’s a certain soft flavour, a bit salty. And then, as the current is ramped up (so that there should be a citrusy bitter taste), it suddenly occurs to me that I’m licking a lemon wet wipe in a fast food restaurant.
Cheok tells me that to create other tastes such as sweetness and spiciness, the temperature must be altered as well as the current. A second prototype uses a semiconductor to allow for this, but is currently broken – which is probably just as well, since this perspex box with a hole in the side looks even more unappetising than the pair of metal prongs I am currently attached to.
Still, Cheok and his PhD students aren’t stopping there. Their next plan is to come up with an instrument that can electronically simulate smell as well, which could produce far more refined experiences, since the 30,000 or so smell receptors we have create much of what we actually consider to be the taste of our foods.
It could also be for more unpleasant. “We can’t use electrodes because it would probably be very painful for one and quite impractical, because people don’t want to insert anything into their nasal cavities,” says Cheok.
To get round this, the team hope to create a magnetic coil that you can put in your mouth – this will then produce electrical signals in the nearby olfactory bulb that trick it into smelling in much the same way.
It sounds like a roundabout way of eating food but the technology could have huge potential down the line. Cheok sees the media and advertising industries as being potentially interested in this – what if you could create any smell while walking past a billboard, or watching a movie? “Entertainment advertising is probably one of the big areas it will take off,” he says.
We’ve had smell-o-vision in cinemas before, it’s worth remembering, and it caused a bit of a stink with customers, pun very much intended. But Cheok’s research could do much more than pave the way for pungent billboards. It could help restore senses to those who lose their ability to taste or smell.
“It’s much more practical to give people a re-perception of smell, because they already know what that was like,” says Cheok.
“I think these devices will be able to give some perception of taste or smell. For example, people who lost the sense of taste, they will definitely be able to understand the differing electrical currents on their tongue. It might not trigger the exact sour like we perceive, but they can match it to what they previously knew was sour.”
It might happen soon, too. Cheok says he always strives to push the very limits of what is possible, rather than iterate on established technologies. “We want to push the very edge of the boundaries of technology. I tell my students, do research which is a quantum step ahead of what’s available now.”
Which make sense considering the time it takes for new ideas to go mainstream is getting shorter and shorter.
“Now grandparents use Skype to video chat. Ten years ago video conferencing on a computer was really a geeky thing. So you’ll see then in the future this becomes the mass market for communications. We can see in the future that we will really blend our physical reality with the virtual reality.”
You only have to look at the success of Oscar Mayer’s prototype – more than 500,000 views on YouTube and counting – to see that he may be right. In the meantime, at least internet citizens can sleep happy knowing that the smell of sizzling bacon can now be sent over the pipelines of the world wide web.