Researchers aim to make digital assistants like Siri less annoying

Apple has Siri, Microsoft has Cortana, Amazon has Alexa — and the list goes on. Today’s tech titans all offer “digital assistants” integrated into their devices, to help our lives run just a little more smoothly.

Driven by artificial intelligence, these voice assistants can navigate for us, set reminders and prod us awake with alarms. But for many consumers, frustrations with the technology can outweigh its benefits. Have you ever asked Siri what time your appointment is on Sunday, and heard her say, “How about a web search for it?”

There’s a lot of money at stake right now in developing “smarter” digital assistants, and experts say the technology is only going to expand. So how can our relationships with these digital helpers be improved?

Justine Cassell, an artificial intelligence researcher and associate dean at Carnegie Mellon’s School of Computer Science, says that speech recognition is one area for development (accents in particular are still difficult for the technology). But for her, the crux of the matter is social: Our digital assistants need to be able to read our conversational cues — not just our words — and draw on an ongoing relationship for context.

“Siri or Cortana or M or Alexa, they all talk to us as if it's the first time they've met us,” Cassell says. “And it's no fun to have a personal assistant with amnesia. That's not what we want from our assistant. We want them to say, ‘You remember that thing you were looking at online last week? Well, the price has gone down. And [I] know what trouble you got in last year when you forgot your husband's birthday, so I suggest you look it up today and you buy it now, while the price is low.’

“That's the kind of not just context, but also the kind of pushy conversation you might expect from someone you've known for a long time.”

Cassell is working with a team to build “socially aware” artificial intelligence that can respond to us the way a friend or colleague does. As part of their research, they videotaped pairs of people doing tasks together for an hour each week, noting the differences between the pairs as time passed.

“We found, for example, that teasing went up from week to week,” she says. “And referring to shared experience went up from week to week. … But positivity and praise went down from week to week.”

Cassell’s team built those findings into Sara — their “socially aware robot assistant.” Right now, Sara helps attendees at conferences get introduced to other participants and find the sessions that best fit their interests.

“We find that people are surprisingly engaged — surprisingly because they themselves are surprised by how engaged they are and by, how compelling they find the experience,” she says.

But if digital assistants can become better conversationalists — more like a person — who do we want them to be most like? A friend? A colleague? A butler?

“I talk about it in Jarvis and Mary Poppins terms,” says Carolina Milanesi, a tech analyst at Creative Strategies in San Jose. “From a family perspective, you see a lot of advertising for Google Home, of these things that come in, and they are your Mary Poppins in the home and helping out and reminding everybody of everything.”

But Milanesi says that at the moment, many of these caretaker assistants operate on just one family member’s account.

“And that's not really, you know, what Mary Poppins does,” she adds. “She needs to know everything about everybody to be able to help the family.”

In contrast, Milanesi says that assistants like Siri are more suited for a Jarvis, or butler, experience. (Edwin Jarvis being the butler character in several Marvel Comic titles.)

“They are my assistant,” she says. “You know, with what Apple is doing with AirPods, adding in Siri’s whisper in your ear like you see in the movies for the ambassador, [where] that person behind their shoulder is just saying, ‘Oh, this is so and so, you met him two years ago, he has a daughter, and that’s what he does.’ That is a little bit the idea, but it’s not clear how are we going to get there.”

For now, Milanesi says, “advertising is way overselling what, in general … these assistants do.” But that may be experienced differently in the next generation of digital assistants — or by the next generation of humans.

When Milanesi was setting up Amazon’s Echo speaker in her home, a device powered by Alexa that is “this kind of jar-looking thing that you have on your shelf,” her 8-year-old checked in to see what was going on.

“And I said well, 'This is Echo, there’s Alexa,'” Milanesi says. “And she goes, ‘Well, is she like Siri?’ And she used ‘she.’ So it was, ‘Is she like Siri?’ And I said, ‘Yeah.’ And she asked the question, and she got her answer, and she said, ‘Well, if she’s smarter than Siri, can we keep her?’”

“Five minutes in, there was that emotional connection that she created with this thing.”

This article is based on an interview that aired on PRI's Science Friday.