This is the transcript for “NLP on the Edge: Voice, AI and Hardware,” presented by Robert Daigle and Andi Huels of Lenovo, presented on day one of Project Voice X.
The transcript below has been modified by the Deepgram team for readability as a blog post, but the original Deepgram ASR-generated transcript was 94% accurate. Features like diarization, custom vocabulary (keyword boosting), redaction, punctuation, profanity filtering and numeral formatting are all available through Deepgram’s API. If you want to see if Deepgram is right for your use case, contact us.
[Shilp Agarwal:] Oh, hi, everyone. I’m Shilp Agarwal. I’m the CEO of Blutag. The title of my talk is retail restaurants and travel. I don’t know why travel ’cause I don’t know anything about travel. I don’t even like to travel. I think the only thing that happened was that I complained to Bradley about the fact that there was no nonstop flights from New York, so maybe that’s why he added that. So we’re not gonna talk about travel. We are a voice commerce platform focusing on retail experiences primarily for shopping of consumable items for for consumers.
So just a little background on the company, we’ve been around for over three years. Oh, we’re not gonna go there right now. We’ve been around for over three years. We work with… we have hundreds of retailers on our platform right now, some really prominent brands. And our focus has been primarily on using Alexa and Google Assistant. And I’ll go into a little bit more about, you know, how we’re venturing into other voice assistance as well. But it’s a really quick talk. I’m just gonna talk a little bit about the current state of where things are with the voice commerce, everything that we’ve seen working with all these brands over the last few years as to what has not worked, which is a lot of things, and and what’s working now. So just starting with kind of things, voice commerce related and smart speaker related, so there’s about hundred and fifty million smart speakers right now in the US.
If you just look at the overall voice commerce and and and the shopping usage, almost thirty five million people have made at least one purchase using a smart speaker last year. So that’s a, you know, significant number of people that are that are doing that. That accounted for almost twenty two billion dollars in voice commerce again last year. So those numbers are there. When we started this thing in twenty eighteen, the numbers that came out of twenty seventeen were about two billion dollars. So it has grown significantly since then. And now it’s on track to get two hundred and sixty four billion by twenty twenty five. So when you look at what’s happened, it has beaten expectations and it continues to grow really fast. At the same time, you know, about a year and a half ago… two years ago, there were a couple of articles out, you know, including this one which said that voice commerce has not taken off and it’s not working.
And, you know, and we were a part of the voice commerce journey at that time as well. So, you know, I wanna kind of shed a little light on kind of what we saw during that phase. You know? And and I think it’s important to see that if the numbers are there, if the voice comma… transactions are there and they’re growing faster than they were supposed to, but still people are complaining about it, where did we go wrong, or where were things going wrong? So I think one of the biggest things that happened with retail was a lot of people tried to do too much when it came to voice base commerce. We said that… you know, we have this website that does all these things. You know? You can browse products. You can look at products, look at reviews. You can make a product selection and buy them. Let’s bring all of that functionality to smart speakers or voice assistants. And that’s just not… that just didn’t work ’cause, you know, there’s a lot of elements of that that don’t make sense today and more of them that didn’t make sense even back then when the only smart speakers you had were primarily the ones that did not have screens. So when you did that, people started using those experiences and didn’t didn’t like ’em. You know? And and there was all sorts of issues with, you know, whether you couldn’t find the right product or the conversation was too long.
It just didn’t work. You know? And when you look at the journey of voice commerce, you know, it’s kinda the same thing when we went with early days of ecommerce, and and there’s some similarities and there’s some this… differences. This is what it looked like twenty years ago. Right? This this used to be a website where you would go and you would buy things. It wasn’t ideal, but it still kinda worked. You still ended up doing a purchase. I mean, the credit card validation did not work. You had to do it multiple times. Finally, you got a transaction and you were super happy that you placed this order. Because at that point, you know, we had a couple of options. We either have an option to try to figure out how to buy a product from this website or we had to go out there and to try to find that product in the real world outside. Right?
So because that was our only option, we figured out how to do it and we made it work. Now things are different right now with with voice commerce. Everything that you’re trying to do with voice, people can do that in another method. They have their computer. You know? They have this all the time. They’re able to perform all these things. The only way it would make sense for them is if you actually made it a lot more convenient and if it actually made better sense for them to perform that particular action through voice as opposed to going on their phone. And, especially, now with the introduction of, you know, all these ecommerce platforms, that problem with ecommerce has been solved. So it’s a lot easier to shop using web or mobile, but voice continues to be a challenge. So so this is kinda like what hasn’t worked. Right? What hasn’t worked is trying to do too much. What hasn’t worked is we’re trying to replace every functionality that that we have on the web and try to bring all of that through voice. So what is working? What what are all those large numbers of voice commerce that we see? And, you know, before we look into what’s working, it’s it’s important to understand. You know?
We had a few people talk about, you know, how the user behavior changes, so important to understand kind of how user behavior works. And and one of the biggest examples, I think, of… in recent times of user behavior is is is QR code. They’ve been around for over twenty five years. They came out in nineteen ninety four, the same year as Amazon came out. We had no idea what they were for twenty five years, and then all of a sudden we look at them and we feel hungry. Right? ‘Cause we figured out that that’s how you order food. So so that behavior changed even though things happen around. And and and and, similarly, when you start thinking about shopping behaviors, there’s a certain phase that we’ve gone through over the last few decades that has changed that as well. And I think the biggest force for digital behavior change has been Amazon. And and what’s happened is that Amazon has created certain shopping behaviors whether it’s about, you know, the speed of delivery, whether it’s about the ease of return, two days shipping. And all retailers have to kind of follow that just because that behavior has already been formed. The user has that expectation.
So same thing is with with voice. There are certain elements that people have been used to with voice. And and, you know, I’ll I’ll show a couple of examples of, you know, basic demos of kind of things that we’ve seen working. And, you know, a big one is being able to quickly reorder something from Amazon. You know, that’s something… was the first thing… you know, they used to have these dash buttons. They got rid of that because a lot of that functionality was replaced by by voice. People were able to quickly reorder something. We provide the same functionality, you know, with with somebody else who does not have products on Amazon and have their own brand with loyal customers and selling something like this, see if voice works, “Alexa, reorder my coffee from Black Rifle Coffee.”
[Amazon Alexa:] Last time, you ordered Coffee Or Die roast whole bean twelve ounces bag for thirteen dollars and ninety nine cents. Would you like to add it to your cart?
[Shilp Agarwal:] Yes. So this is a very basic example. It only does one thing. It lets you reorder your coffee. Now, you know, how many times does it happen that you’re running out of a particular product? You know, it could be your dog food or it could be a moisturizer. You know that you are gonna run out of it and you wait for it ’til it’s fully out and then you realize that it’s to order… too late to order it online and you go out to the store. With with smart speakers all around, you know, ambient computing, that, again, was talked earlier as well, is is something that, you know, captures that moment when you’re… you are actually thinking about that.
So if you were able to capture the moment when somebody knew that they were gonna run out of a product and give them the ability to order it, you would be able to save that sale. The people, you know… our our categories have been the replenishable ones. Our top categories are, you know, groceries, baby, pet, beauty for the same exact reason that that we mentioned. And we have seen people who have their customers using voice assistance have seen an increase of up thirty five percent in reorder frequency, simply for the same reason that they’re able to capture that moment when they’re when they’re running out of a product. Now the the other aspect when it when it comes to things like groceries is, you know, being able to add things. And I think Jeff had talked earlier about being able to quickly add things to your shopping list, whether it’s a voice assistant. We have customers that we’ve given them the ability to add items directly to their shopping cart. You know, again, a lot of these devices sit in the kitchens. You’re running out of milk. You’re running out of eggs. You can just quickly… simply ask your voice assistant to do that. So that experience looks something like this.
[Commercial Speaker:] Alexa, order milk from FreshDirect.
[Amazon Alexa:] Added. Anything else?
[Commercial Speaker:] Strawberries.
[Amazon Alexa:] Added. Anything else?
[Commercial Speaker:] Bananas.
[Amazon Alexa:] Added. Anything else?
[Commercial Speaker:] No.
[Amazon Alexa:] You’re now leaving FreshDirect.
[Commercial Speaker:] Alexa?
[Shilp Agarwal:] So what this has done is… do you guys hear me? Hello? Yeah. Did I turn it off, Bradley? I don’t even know how to turn it off. I think the battery died.
[Bradley Metrock:] Testing. Yeah. It might have.
[Shilp Agarwal:] I can just use this.
[Bradley Metrock:] Yeah.
[Shilp Agarwal:] Or I can just speak loud.
[Bradley Metrock:] This mic.
[Shilp Agarwal:] Can you guys hear me now? Alright. So, essentially, what we’ve done is that we’ve taken the same exact experience of being able to add things to your shopping list, which people have been used to and, given those customers, the ability to add it directly to their shopping cart. And in these cases, what we’ve seen is the same thing. It’s slightly different. It’s not the reorder frequency, but we’ve seen the in… increase in shopping cart sizes. So our average grocery customer sees an increase of eleven to twelve percent in their shopping cart sizes. If their shopping cart was about thirty, thirty five items, they’re seeing an additional three to four items getting added to the cart. And this is huge, especially when you are at a stage where you’re doing deliveries. You’re letting people do curbside pickup.
Having those three, four extra items in the cart makes all the difference ’cause that’s… that covers your cost of of a lot of different things that are not there yet. So, again, these are things… and and when you add items like the way we showed you, this is eight to ten times faster than any existing method of adding items to your shopping cart, and you’re capturing that moment. So now, because people have the ability to use their computer or their mobile app to perform these actions, the only way that behavior is gonna change is if you provide them something that is a lot better, and I think that’s the key. So what it comes down to is when we started doing this a couple of years ago, we tried designing a lot of conversations. Let them talk to this device. People don’t wanna talk. People don’t wanna have conversations. They just wanna perform tasks.
So when you start thinking about shopping in ecommerce, think about what kind of tasks you can let them provide. They wanna say something, and that’s what they wanna get done. And, you know, it goes back to the same thing as to what are the other things that people have been able to do really well. Right? They’ve been able to check the weather, set the timer, listen to music, these are all tasks. They’re not trying to talk with you. And and I think having that mindset makes it really important to to see that, ok, these are the different tasks that you can let your customers provide or the end consumer provide, and that’s where it’s gonna add value.
And, essentially, being able to use that to to add value is something that we’ve been able to do effectively. Again, you know, started with really long conversations, decided that a lot of those were ending up in, you know, very early drop-offs ’cause people don’t wanna have that conversation and and, you know, finally, saying that, ok. There’s few elements of this and there’s other other things. Right? You can check your order status. You can check for any sales, coupons, check for product that’s in stock, again, performing these tasks, and then directly relating that to to ROI for a customer. And and once you can figure out the ROI, you know, in… which in in our case is increasing the reorder frequency and increasing shopping cart sizes, then you actually have a customer that is willing to buy your product as well, ’cause at that point, they’re more… going more from saying that, ok. It’s a really cool technology. You should use it because everybody’s gonna use it in the future to saying that I can show you value today, and that’s where it starts making a lot of sense for them. So, yeah, I mean, that’s pretty much what I have more for the retail. You know?
The other things, you know, we know are gonna be big spaces are gonna be voice in in in auto and and while people are driving. That function is slightly different ’cause I think those are things where people are not gonna be able to do on a screen ’cause they’re driving. I mean, right now, you can still technically do that, but in a few years, we know that it’s gonna go away. You actually will not be able to use your phone when you’re driving. And those experiences, I think, are gonna be, you know, somewhere in between a task and a conversation because those are areas where people are driving and helping them do their grocery shopping when they’re driving back from work is something that’s gonna save them a lot of time, and and it’s gonna make a lot of sense. And that, combined with, you know, I think, a few other things, like restaurant and ordering and drive-throughs, are are are some areas that we see, you know, a lot of big opportunity. But, yeah, that’s that’s all I have and it’s it’s quick, and we’d love to talk to anybody else if they have any ideas. But hopefully, we all have a good show. Thank you.
If you have any feedback about this post, or anything else around Deepgram, we'd love to hear from you. Please let us know in our GitHub discussions .