How is it today in the industry there are different kinds of dialog systems for different purposes? What do you see in the marketplace? So in the market I think it's interesting that people are trying to build on their strengths. Obviously, that's why they're having conversational agents. So I think Amazon's idea maybe at the beginning was, "Well, we want people to be able to shop." Right. "We want people to be able to shop by phone." Then they realized that in order to get people to use their Alexa everyday, they needed to offer some kinds of functionalities that would help people do particular tasks like set timers or listen to music. The music I think is huge. When we first started using our Alexa, we found that the idea that you could call up any song from any decade without having to fiddle around with CDs; and find the CD and get it on the player and all that kind of stuff, we found it really fun. So I think you just have one hook like the music and you get people to start using it. So I think everybody in that space is looking at how many times per day somebody reviews the conversational assistant, the smart home device, how many times per day would somebody use it. They want to get that number up as high as possible. Google has Search. So their flagship technology is their Search, but they've had Voice Search around for a long time. So their challenge is moving their search capability to conversation. I don't know if inside Google they still believe this now, but I think when they started they probably thought that like, "Our Search works really well and our speech recognition works really well, so we're like 98 percent there to having a conversation." The thing is, it's a conversation. It's not just a sequence of search QAs kind of strung together. So there's still a lot of challenges there. Okay. So that's one kind of system. I think Google they're trying to treat it as though it's a sequence of- Of search queries. Of individuals searches. That somehow you build the context from the previous part of the conversation into the next search query. So it's more like an answer and question conversation. And keep going question and answer. Yeah. Like, who is the president of the United States? How tall is he? Who's he married to? Does he have any children? So each of those things in sequence. Where you take the answer to the first thing and then when you do the second query, you take the first part of the conversation and you just supplement the query with that. So you make a richer query. Right. You either take historical data or collaborative filtering in order to make predictions about what could be the next question. Yes, right. Right. So in Amazon, in Alexa, you said that would be a little bit different? So in Amazon, what they've done is they've taken more of a tools approach. For each task that they want Alexa to do, they would have built what is called a call flow. Okay. So they would have built a tree of how they expect the conversation to go with branches at each stage. Then they would try everything that the human says in response to a question or anything that the human would say. Like you'd say set a timer, then Alexa might know that she doesn't have the amount of time for the timer, so she could ask the question. Or if you say set a timer for 10 minutes, she knows that set or set a timer is cue to a particular tasks that she knows how to do and each next thing that could happen would have to be anticipated. What I think is the biggest challenge right now is, and I know people are working on it probably maybe in both companies. But definitely Google because of the Search paradigm focus that they have, is being able to take those structured call flows that let you do particular tasks, and interleave search with them. So the example that some of my friends inside Google use would be the set a timer example. Where you say set a timer and Google Home or Google Assistant or Alexa would say for how long and at that point you might say something like how long does it take to boil an egg? Right. Because you don't know. Right. Then what would have to happen at that point in the software is it would have to recognize that it's gone outside the flow that it expected and that there's a query there that has to go to search. Right. So at that point the query would go out to search. If you do this on Google, what you'll see is that if you do how long does it take to boil an egg, it might come back with the fact that it might just tell you a hard-boiled egg or it might tell you soft-boiled egg but some of the hits that you get. So it depends on how smart it is. Some of the hits that you get on that tell you that there's both soft and hard-boiled eggs, and it takes three minutes to do a soft boiled egg and six minutes do a hard-boiled egg. So it depends on how smart that Search thing is and how it gets integrated in the context. But ideally, what you would want is that those search results come back and that you could then ask the user a question like do you want a soft-boiled egg or a hard-boiled egg? Then you have a dialogue. Then you have a dialogue. You start having one. Those two things are starting to work together because of the Search instead of just returning a paragraph, like what you see on the search page, it actually returns the type of information that the structured dialogue is looking for. That's something that currently it doesn't work. You can't do it. That's kind of put you on the edge. That's where we need to kind of get. To where we can start to do something more interesting. You say it's on the one hand we can think about you have this query paradigm and the other one has the call flow? That's how you called it? Yes. Can you give us some [inaudible] the query? Okay. Again to summarize, its got if you can think about it, the Google and googling and googling. The call flow, can you give us a little summary definition for how we could put that in you own words? It would have particular phases of the dialogue. I think the easiest example is to move away from the set a timer and consider a task like booking a flight or something like that. All right. Okay. Where you know that you need like four or five pieces of information. So that's the classic case for dialogue systems. So in that case, your tree it would start out with a greeting. It might have three or four ways of doing greeting. So it would say, "Hi, how can I help you? Where do you want to go?" Or just, "How can I help you?" or whatever and then it would take the user's utterance. It would be looking in that user's utterance for the departure date, the departure city, the departure time, maybe an airline preference. Then have clarification questions as well. [inaudible] kind of fill in that whole form. In order to have a dialogue. In order to have it, yes. Not just being a one step; answer, question. Right. In the worst case, that system would have to have those slots filled in in a particular order. That's older technology, but still a lot of that is out there. Like it asks, it says, "What's your departure city, where are you flying to, what date do you want to fly?" Right. Just like when you call to these call centers. You know already which question will come next. It's like it reads the PDF document to you. Yes, exactly. But there's pretty good technology out there for what's called mixed-initiative dialogue. Where the caller could produce any combination of the properties that the system needs. So it could say, "I want to fly from San Jose on Tuesday at 10:00." So that maybe an odd combination of things and then the system would come back and say, "Where are you flying to?" Then we try to fill that in. Because it needs it. It's probably a target model but think like there's this form behind you. Yeah, exactly. That's a good way to think about it. There's a form back there and it's trying to fill it in. The question is, with the mixed-initiative task-oriented systems is, does it let you fill that form in in any order with any combination with your turn? Of course that depends on how well the speech recognition works. Because if the speech recognition is really limited, that's when you get the cases where it forces you to do it in a particular order and asks you for one thing at a time. But I think the speech recognition is good enough that the system would let you pick any combination of things that you want to talk about like whatever you thought was most important, you could say that thing first. We should be able to now have these mixed-initiative systems that don't make you go down a particular fixed path. Mixed-initiative, that's what it's called. Mixed-initiative. That's very interesting. Yes, because, of course, when somebody asks something, it's not because somebody might call and say, "I want to fly over Christmas today, but I want to be back before New Years." Right. Then the system shouldn't ask when is your return flight because I've already said that. Yes, right. So it has to fill out dynamically whatever I already told her than that. Right. So that's called mixed-initiative. Mixed-initiative dialogue.
