The science of engineering touch

How do you digitize touch?

You may have played video games where the controller shakes and buzzes, heightening the effect of onscreen crashes. In fact, you can find that digital touch sensation — known as haptic technology — in devices from your smartwatch to your cellphone.

But as engineer Katherine Kuchenbecker explains, the haptic possibilities don’t end with our entertainment and communication tools — haptics are now being used to “smarten” everything from virtual reality to robotics technology.

Kuchenbecker, a director at the Max Planck Institute for Intelligent Systems, has even worked on haptics that can suggest textures as detailed as cotton or wood, using a technique she calls haptography — recording “how surfaces feel to touch with the same accuracy that we can capture them with a camera.”

“Imagine you take a tool and you drag it back and forth over a piece of leather, or say, a rough piece of stone,” she says. “You can feel the tip of the tool bounce across the little bumps on the surface or be dragged. It might be more difficult to move across the leather. And also you can feel how hard or soft the surface is.”

“And so we created a little tool that we call a haptic camera, a touch-based camera that can record all those sensations. And then we do a bunch of math and make a model so that when you go touch a virtual version of that piece of leather or that stone, we can create sensations that make it feel almost as though you’re touching the real thing.”

Using the technology, users can drag a stylus across a tablet screen showing pictures of, say, the stone, or a scrap of leather. A little motor in the stylus vibrates according to the user’s movements and force, Kuchenbecker explains. “And then as you drag the stylus across the picture, you would feel like the tip was going over the stone or the leather, even though it’s really just going over glass.”

In a further step, her research has applied haptics to virtual reality, creating a device that can be used to feel virtual objects in 3-D space. She calls it haptography 2.0, and says that adding touch is crucial to making believable virtual environments.

“Life is a lot more than just what you see and what you hear,” she says. “You can’t do anything in the real world without reaching out and touching something.” And in fields like surgery, touch can have life-altering consequences. Not surprisingly, Kuchenbecker says, haptics are at the cutting edge of medical tools and training technology.

“Certainly, one big opportunity is to improve the kind of video games that doctors train on,” Kuchenbecker says. “There’s a big movement in medicine right now to help surgeons or dentists or other medical practitioners practice on a realistic computer simulator before they try out, say, suturing or drilling a cavity on a real human.”

Another promising path for haptics, she says, is in robotic surgery. Robots like Intuitive Surgical's FDA-approved da Vinci system allow doctors to perform minimally invasive surgeries deep inside the human body, but have “absolutely no touch feedback,” Kuchenbecker says. “So, the doctor can’t feel anything of what they’re doing.”

Haptography can change that by recording sensations during surgery, she explains. “And then instead of modeling [the sensations], you can just transmit them immediately, sort of like a microphone and a speaker where we measure what the tools are feeling and play it for the surgeon’s hands right away,” she adds.

Other robots can also benefit from touch technology. Kuchenbecker says haptics can help autonomous robots put objects in a dishwasher or safely hand something to a human. “When you try to program robots to do tasks like that, you really start to see how important … haptic signals are for coordinating these movements in the real world,” she says.

Haptic technology can connect us to each other, too. “People who’ve had an amputated upper limb, like, let’s say they’ve lost their hand, it’s not just that they want to be able to pick up their coffee. They would really love to be able to hold hands with their loved ones,” Kuchenbecker says. (Here too, the technology is making inroads.) “And so the emotional side of touch and physical contact between people is really important.”

It’s so important, Kuchenbecker says, that her lab is working to create a hugging robot — which is, just as it sounds like, “a robot that delivers really good hugs.” “Give me a few more years and I’ll have a PhD student, and maybe we’ll see if it might have some true benefits,” she says.

“Could it really uplift your mood or help figure out how you’re feeling or help people stay in touch long distance? That’s some of the things we’re thinking about.”

This article is based on an interview that aired on PRI's Science Friday with Ira Flatow.