Opening Keynote - Jeff Blankenberg, Principal Technical Evangelist, Amazon Alexa - Project Voice X

This is the transcript for the opening keynote presented by Jeff Blankenberg, Principal Technical Evangelist at Amazon Alexa, presented on day one of Project Voice X.

The transcript below has been modified by the Deepgram team for readability as a blog post, but the original Deepgram ASR-generated transcript was 94% accurate.  Features like diarization, custom vocabulary (keyword boosting), redaction, punctuation, profanity filtering and numeral formatting are all available through Deepgram’s API.  If you want to see if Deepgram is right for your use case, contact us.

[Jeff Blankenberg:] I just wanna make sure I’m really clear on that disclaimer because the last thing I need is my PR team saying, what did you say? That’s that’s the worst thing that could happen. Ok. So the future for us is still really fuzzy. We’re looking at it from a perspective like this. Right? We’re out in space, and we know there’s a thing there, but it’s really unclear. It’s it’s it’s quite fuzzy. But what we do know at the center of all of this is people. People are the reason for everything we do. We go to work to create products and services for people. We create free time to spend time with the people in our lives that are important to us.

People are the reason for pretty much everything we do in our lives. I want you to imagine for a minute, if you’re skeptical about that point, about being the only person left on earth. For a few of us, this probably sounds awesome. I’m so tired of all the noise and nonsense that some people can bring into our lives. But this guy specifically, his life has changed fundamentally. His entire world is about survival, and how am I gonna get on to the next thing. Because there aren’t people there to create and generate all of the things that we just take for granted in our lives. But it’s not just about the people. Right? It’s also about all the technology that surrounds us.

In fact, I wanted to think of a world where we had people and technology combined, and I wanted to find a way in which that specifically created a ton of stress and anxiety for all of us. Like, the the mixture of people in technology, and I found one. And so I want you guys to be prepared here when you see this next image ’cause this brings a lot of stress to my life. The incoming call. Anyone else? This this just gives me a little bit of tension, a little anxiety. When you think about the idea that someone’s just calling me out of the blue, no. That’s not how it works anymore. When someone says, hey. You know what? I’ll just call you later. I know what my response is. Chill. Just text me, bro. Like, I don’t need a phone call right now. And I thought a lot about this. Why is it that the incoming phone call is so off putting? Aside from the fact that we’re spammed to death from, you know, extended warranties on our cars and stuff like that,

Right? How many notifications a day do you get? If you guys looked on your phones, it’ll tell you how many notifications you get. I’m embarrassed by the number. And I’m… I’ve gone through, and I… I’ve tried to mute and turn a lot of them off, and it’s still well over a hundred notifications a day that are just calling for my attention because something happened on Twitter or, you know, whatever it happens to be. Here’s another example of how technology has changed our lives. I don’t know if it’s good or bad, but it’s an interesting one. You’re sitting in a restaurant with your friends, and someone raises a question. Maybe it’s a trivia question. But someone raises a question that probably is unanswerable without a deep, deep set of knowledge. I’ll give you an example of one of those questions. Which actor or actress has won the most Academy Awards? Does anyone know the answer? Good guess? Meryl Streep is the guess. Anyone else has a guess? So nobody knows, but I guarantee someone right now is pulling their phone out there like, let’s who… see who the thing is. Right? Because that’s what we do. We sit at a restaurant, and twenty five years ago, we just had to agree to disagree. Right? There was no looking it up. Nobody was running to the library to look this stuff up. It was just, I guess we’ll never know. By the way, the answer is Catherine Hepburn.

And this is the list of people that were probably in your heads also. They all have three. I had to do this because I know you guys can’t leave this stuff unresolved because we, as humans, can’t leave this stuff unresolved. I I need to know that now. That’s a technology problem. And we use all of this technology in a way, like I said, asynchronously to free up our time to do the things we want to do. Right? Some people, I’m not including myself in this, some people love to exercise and get stronger and fit. Some people are artists. Right? They love to… they… they’re optimizing their time around being able to create art, and that comes in a variety of forms. Right? This is a very visible painter artist, but there’s all sorts of different kinds of art. Right?

I have some friends that have gotten incredibly deep into woodworking. They are master woodworkers at this point. I’ve seen the work they’re creating. Beautiful chess boards with gorgeous inlays in them. I would not be able to touch the stuff they’re creating. But they’re not a professional woodworker. This is just what they’re doing with the time that they have. They’re optimizing around what is the thing I really want to do. I’m a programmer. I spend a lot of my time outside of work building things that I care about, that I’m interested in. One of the things that I’m actually building software for is because I got back into the habit of collecting baseball cards. And I wanted a really awesome way to organize them and share them with other people, so I started writing some software to help me do that ’cause there isn’t something like that today.

But for a lot of us, and there’s nothing wrong with this at all, this is how we optimize our time. I wanna be able to do more of this. I’ve got Netflix and Disney plus and HBO and all the other stuff, and there’s so much I wanna watch. We are in a golden era of television, but we’re optimizing around being able to do these things. So much so in fact, this is from Nielsen’s last fall report, the average American spends over ten hours a day consuming content. We are optimizing around the things we want to do. Ten hours a day is mind blowing if you think about that. Now it’s spread across a bunch of different things, but still, ten hours a day, it’s no wonder we’re also tired. We gotta work. We gotta consume. Somewhere in there, we gotta sleep. But everybody has their own set of priorities, their own interests, their own optimizations that they’re building around. But the thing that all of us in this room have, regardless of our interest or passions, we all have stuff. We all have those things that we have to do, but we don’t want to do. Right?

And in many cases, they float around in our heads as thoughts. They’re they’re taking up cycles in our brain’s computer. Sometimes they’re good. You know? I… my kid’s doing really great in school. I wonder if I should have conferences with her teacher. Or it might be something cool like, who won the most Academy Awards or when are we gonna finally land on Mars, and who’s gonna go? Right? You have all these things that are just floating around in your head, but you also have a bunch of these. Right? When’s the last time I went to the doctor? What’s that pain in my side? How are my kids doing in school? Are they doing well? Is my boss happy with my work? He did call me at four thirty on a Friday just to chat. I thought that was weird. Did I leave the oven on? Right? All of these things that are just floating in our heads. We all have those stressors that are just eating up time inside our brains, taking away from all those things we really wanna do.

This is where I want you to take a big jump forward. I want you to think about the idea that we all, in the future, have a personal artificial intelligence assistant. Now I don’t mean something like Google or Alexa or anything like that. I’m talking about something far more robust, far more impactful than what we see today. With this, I want you to think about what makes that possible. The number one thing that stands out to me is trust. If I were to have a human personal assistant, I would almost need to inherently trust them before I’m gonna start giving them access to portions of my life to run for me. Let’s talk about trust for a second. I want you to imagine for a moment that you’re standing in this field. The sunrise is coming up over the mountains. You can see the glue… the the the dew glistening on all the blades of grass. And out of that forest, all of a sudden, you see me charging at you just running out of the forest. My knees are covered all the way with dew, everything soaked. I look like a crazy person running at you, and I tell you, by the way, you better enjoy that sunrise because that’s the last one there will ever be. Now like all of you are doing now, you’re looking at me like I’m crazy ’cause that’s not likely something that you think is possible. If you think about a sunrise, you have enough evidence over your entire lifetime to say, I don’t believe you for a moment. There’s no way I believe you that the sun isn’t gonna rise tomorrow. You have full faith in the fact that that sun will rise tomorrow.

So let’s go back to that assistant. Let’s think about all of the things that it could do for us and all the trust that we need to make sure that that’s possible because that falls on us, the people in the technology, the ones that are building all of this stuff. Another good example? How many of you have a pair of Bluetooth headphones? How many of your Bluetooth headphones work every time you try to use them? Right? They’re amazing technology. The fact that I just have this wireless thing, I pop ’em on my head, they connect to my device in my pocket, unbelievable. Ninety five percent of the time. But that five percent doesn’t surprise you because you’ve had a history of experience where it falls down or fails or doesn’t work. That isn’t good enough when we’re talking about artificial intelligence. That isn’t good enough when we’re trusting it to manage parts of our lives, to solve all of these problems for us.

But if we do, oh my goodness, those Bluetooth headphones, when they work, they’re as good as that sunrise. Right? They’re as good as just standing there taking that fresh air. So let’s talk about some of the things that AI can handle for us literally today if we really wanted it to. The first one for me, this is a big one, nobody will pay their bills anymore. Everyone ok with that? We don’t pay our bills? I don’t physically mean we’re not paying our creditors. What I do mean is I don’t have to worry about it anymore. And I’ve had people challenge me on this, and they say, well, Jeff, you’re in a position where you probably make more money than the bills you have. That’s a nice luxury to have. And my argument is actually specifically for the opposite. I believe that the people that need this help the most, the people that don’t have enough money to even pay their bills are the ones that probably should most rely on artificial intelligence.

People will argue all the time that I need to make very subjective decisions about my finances and my money and where I’m gonna save and where I’m gonna put all of these things. But, realistically, you make a lot of subjective decisions that aren’t always good. How many people have said, you know, I need to save more money, but I stopped at Wendy’s. Or how many times have you gone to the grocery store and spent two hundred dollars on your week’s worth of groceries only then swing through McDonald’s on the way home? And it’s like, you’re kinda defeating the purpose there, man. You’re you’re spending on both sides. So the argument is I’m suggesting maybe we could let artificial intelligence start to pay our bills for us, start to make some monetary decisions on our behalf. We, as the human, always have the power to override these things. But another great example of financial instruments is the stock… oh, I’m sorry. I jumped ahead.

AI is a good thing as we think about all this stuff, but it does have its problems. Right? Think about the game of chess. The game of chess for the masters has mostly been ruined. They can still play, and they can still beat most humans all the time. But when it comes to beating AI, it’s really hard. Most of the time, they’re not successful. The stock market is another good example of this. If you think about how the stock market really works, most of the transactions that are happening are happening because of machine learning, because of artificial intelligence. The rent right around the New York Stock Exchange is astronomically high because the length of the wire from that building to the Stock Exchange servers is shorter, and that’s the only limitation to be able to make trades faster and more intelligently at a better pace to make more money. In fact, the stock exchange actually had to introduce miles and miles of cable inside the building just to slow everything down. This is happening today. This isn’t even something that we’re looking towards the future. Most of the trades that are happening in a stock market today are being made by a computer, not by a human.

On top of that, think about medicine. We’ve taken lots of situations where doctors are diving into cases and saying, oh, this person might have a certain kind of cancer, or they might be diagnosed with a certain disease. But what they found is that by feeding that same data, those same patient records into AI, that they’re finding that these systems can actually diagnose better than doctors. And doctors are doing a phenomenal job, but AI is often able to take the subjectivity out of things and say, no. You know what? Actually, this person looks like they are actually prone for Parkinson’s based on this factor, this factor, this factor across thousands of cases. Something a doctor may have overlooked or not had the ability to dive into the data the right way. That’s a little serious stock markets medicine.

Let’s talk about baseball. Baseball is something that coaches and players have been asking for forever, to eliminate the subjectivity of an umpire. It’s good and bad. Right? There’s good things and bad things about having a subjective umpire. But when it comes to a strike zone, when you watch TV and they draw that box on the TV, and they whip the ball in there and it’s outside the box, but the umpire calls it a strike, that’s frustrating to the fans, to the players, to everybody. So they’re talking about this. They’re experimenting with it at the minor league levels. They’re trying to figure out how they can best do this, and the the solution they’ve come up with so far is an earpiece for the umpire. It beeps if it’s a strike, it doesn’t if it’s a ball. So to the average viewer, the game looks exactly the same. But to the umpire, he has more information than he had before.

Another good example is making phone calls. We talked about that earlier. But when was the last time you wanted to call your dentist? Anybody? Guessing probably no. My dad’s a dentist, so I have an excuse. But I think for most people, that’s probably not a legitimate thing they like to do. And the reason is is because it takes up our time. It’s doing this mundane task that I don’t really care about. Right? I have to schedule an appointment. If you’ve ever done this, right, you’re sitting in the chair, and they’re like, hey. We’re gonna schedule for your next appointment. It’s six months from now. You pull out your phone, and you look at your calendar six months from now. And if you’re anything like me, six months from now looks wide open.

I have no idea what’s happening six months from now, so, yeah, let’s schedule that for Tuesday nine AM. Sounds great. Two weeks before that, they send me a text message or whatever. They’re like, hey. Just a reminder. Here’s your appointment coming up. I’m like, oh my gosh. Nine AM on a Tuesday. What was I thinking? That’s stupid. So now I’ve gotta call them, and I’ve gotta reschedule everything. So what was the point of the appointment in the first place? Now I’ve gotta call them. I’ll be like, how’s your next Wednesday after three o’clock? I’m sorry. The doctor won’t be in. Ok. How Thursday? Like… and we do this whole dance. Why couldn’t all of that be handled with AI? And instead, I just have an assistant pop up and say, hey. You have a dentist appointment next Tuesday at eight AM. I took into consideration your calendar and the dentist calendar. I even talked to your health care provider to make sure that you were within the six month window that’s appropriate. All of that’s handled for me automatically. And if I wanna reschedule it, I can still do that, but it can handle that stuff for me. And if a meeting pops up on my calendar or some other commitment, it recognizes that and does the process again and reschedules for me. I don’t have to do it.

I also wanna apply that exact same concept to my social life. How amazing would it be if you saw on your calendar that this Friday, you were getting together with some friends you haven’t seen in, like, eight months. Oh my god. That’d be amazing. I didn’t have to call them and, like, schedule and like, oh, I got kids and marching band and what. It just figured all that out because it was able to talk to everybody’s stuff and say, oh, look. They’re both free. Let’s let’s make a reservation even and handle all of that for them. That would be the best. I would have the greatest social life ever because I don’t have to be the one actively calling everyone and trying to schedule things. My last quick example of this is refinancing your house. For those of you that have mortgages, you get a mortgage thirty years, fifteen years, whatever. And over time, interest rates change, and sometimes it’s in your best interest to refinance your house. You’ll save money. But we don’t do that until, like, interest rates drop low enough to make headlines or… we’re not actively paying attention to interest rates every day. Most of us aren’t, anyway.

So how cool would it be if your assistant just popped up and they’re like, hey. I refinanced your house for you. I saved you seventy bucks a month. Oh my god. That would be amazing. This is where we’re headed. This is the stuff that I’m talking about. But there’s one more thing that we wanna address, and that is human behavior. How many different ways are you aware of to manage your grocery shopping list? Some people have a piece of paper stuck to their fridge. Some people might use the shopping list that’s available on Alexa. Maybe you have one of the nine hundred different apps that are available on your phone.

But what do all of those require, every one of those things that I just mentioned? They require a behavior change. You actually have to do a thing that you weren’t doing before in order to make this all work. Now I want you to introduce a spouse and children and other people living in your house, and it just gets an order of complexity bigger every single time. So as we think about these things, all of them tell us, all… there’s so many apps out there that are super helpful as long as we’re willing to change our behavior. Right? You always see things like this. Right? Just get a new mindset, and you’ll have all these new results in your life. Right? It’s the same reason that so many New Year’s resolutions are always successful because we’re not gonna change our behaviors.

We are who we are. We’re going to do the things that we do, and we’re just gonna be frustrated every time we think about, man. I should be better about making a shopping list or doing meal planning or whatever it happens to be. All of these things are really hard. But when we start to put this AI stuff together with our human stuff, it becomes a lot more interesting. And I’ll just give you a couple more examples before I wrap up here. One, the garage door. There are lots and lots of controllers and garage doors and all sorts of things that you can do to set up a way to check and see if your garage door is open or closed. You can even, with your voice in many cases, say, you know, Alexa, close the garage. But those require you to remember to do that. So as I pull out of my driveway and I’m distracted by my phone or whatever, like getting directions to the place I’m going, I forgot to close the garage.

Well, if I’m a person that forgets to close the garage, I’m probably also a person that forgets to check to see if I forgot to close the garage. Right? So why isn’t there already mechanisms in place? And we see this a little bit with our phones and services, like if this then that, where we can ring fence things. Hey. I I just noticed that you left your house, but the garage is still open. I closed it for you. Awesome. I didn’t have to change anything about my behavior, and the thing that I wanted to happen, happened. I’ll give you… actually, this is… isn’t fair. This is a picture of my garage. Right? No. It’s not true. But I want you to imagine for a moment that it is. My garage is proactive, and it uses context. I’ve closed that garage for you. Or I’ve noticed you’re on your way home, I’m going to open it as you’re pulling into your driveway. Now this uses a bunch of context and sensors. Right? Our cars have sensors, our phones have sensors, our house has cameras that could use AI and image recognition to recognize that your car is pulling in the driveway, all sorts of cool things we could do with that.

This is probably more of… this next example is probably a little more applicable to the ladies in the room. Have one of these at home? Ever had to come back home because you thought you left it plugged in? I know I have, I mean, for my wife. But I’ve come home dozens of times because we pulled out of the driveway and headed down the street and we’re like, did I unplug that? That’s a that’s a fire hazard. I probably should deal with that. These are the kinds of things we’re not gonna change our behavior. She will stand there, she will unplug it and say, I have unplugged the curling iron. And then she’ll go downstairs, get in the car, we drive five hundred feet down the street, did I unplugged that? We just can’t connect those things. Right? That’s not what our brains were designed to do. So I solved this temporarily with a smart plug. The smart plug’s name in our system is fire hazard. So I say turn off the fire hazard or, you know, whatever. That’s a… it’s a nice little simple thing. But I think we’re headed to to a world where the house is gonna be much, much smarter than this.

We’re gonna have smart fuse boxes and sensors. I even imagine a world potentially where our rooms don’t have light switches because the house is aware enough to know when a light should be on or off. But let’s head back to this shopping list. Think of all of the things that we could do if we didn’t have to change our behavior. If there were sensors in our fridge that could tell us that our seven year old just finished the milk and the Oreos, by the way, well, that tells me two things while I’m out. One, I don’t need to bring him dinner ’cause he ate all the Oreos and milk. But two, I need… I know I need more milk because tomorrow morning, they’re gonna need cereal for breakfast or whatever else it is. So I should stop on my way home. Imagine having that kind of knowledge and power in your life. Proactive notifications are the start of all this. Right? Being able to figure out where we are and what we need at the right time ’cause if I’m out, then I don’t know what’s happening at home. And if I’m at home, I don’t necessarily know what’s happening out in the world. But the first step, of course, is, hey. You’re out of milk. You probably should get some.

The second step, though, is, hey. I had some milk delivered to your house because you were out. Right? I’m just saving you time. I bought it from the place you wanted for the price you wanted, but milk is on the way. You see, some of this, the early, early days of this are in services like Amazon Dash, and this is not meant to be a commercial at all. I just think this is a really cool service. Printers and refrigerators and all sorts of other things that use consumables, washing machines, when they recognize they’re out of things, they have the smarts built into the device to make a call out to whatever the provider is and get more delivered to you so that you don’t run out. So that shopping… the the washing detergent shows up just at the time you need it. Printer ink shows up just at the time you need it.

Let’s get more proactive. We get in our car… this happened to me just last weekend. I didn’t make a picture of it because I’m not calling any company out for being bad. But I was heading to a company called Costco. Maybe you’ve heard of them. And they have a late opening time. I’m an early riser. I’m usually up and, like, trying to get stuff done by, like, seven AM most days. I was like you know what? We’re out of this stuff. We have people coming over. I’m just gonna run over to Costco this morning. It was maybe eight fifteen. Costco doesn’t open ’til ten. So I got in my car, typed in the GPS, head my way to Costco, pull into the parking lot, and it’s a ghost town ’cause I didn’t think to check and see when they open. Why… at Saturday morning at eight thirty, why wouldn’t you be open? Well, ’cause they don’t open ’til ten. They take good they take good care of their employees. But if I had had a little bit more information, a little bit more artificial intelligence to help me understand this, I wouldn’t have been staring at this sign saying, sorry, man. You… you’re gonna have to wait two hours ’til you can get in here.

The other cool thing about this is as we talk with our cars and our phones and our echo devices or whatever else you have in your home, every one of them today has a different assistant. Right? BMW has their own assistant, Alexa has their own assistant, Google has their own assistant, Bixby, everybody has their own thing. That’s not a bad world. We want everybody to be able to build their own technology. We have an entire technology called Alexa Custom Assistant that let you build your own thing on top of the same technology that Alexa uses. But the thing that I’m most excited about, and I think that a lot of you who are here should consider, is the idea of the voice inter… interoperability initiative. If you don’t know about this, this is the simple idea that every assistant should work on every device. So if you need to talk to a specific assistant, your car, whatever it happens to be, you can talk to them. And when they don’t know the answer, they’ll hand off to the one that does know. But you’re not limited by talking to each individual device for a specific task. It’s ambient computing. It’s meant to be where you are, when you are. So let’s take a step back. We have all of those crazy thoughts going on in our heads.

Let’s go back to a peaceful place. Let’s think about a world where we have artificial intelligence solving all of these problems for us. Our bills are paid. Our kids are where they need to be. Our our fridge is stocked. Our appointments are made. This is the future that I’m the most excited about because problems are being solved without my direct input every single time. It frees me up to do the things that I want to do. And the reason that I’m so excited about this is because we see the beginnings of this. Right? Today, as we talk about voice, this is the beginning. It’s not always just gonna be about voice, but it’s about the technology that powers all of it. We always say at Amazon that it’s day one. Every single day is day one, and that’s not necessarily wrong. If you think about building skills for Alexa, it is day one. We’re still really thinking about where is all this stuff gonna go and how is it gonna work.

Bradley was talking about voice payments, and how is that… how that problem is gonna be growing and growing and growing over time. But it all comes back to that story about trust. Do people trust to make purchases with their voice? We’re starting to see a change. It’s good. But I want that whole world in my hands. I want access to all of that technology. I wanna set it up once and forget it so that I can go do the things that I wanna do. So I can go walk on the beach later this afternoon without having to worry about a bunch of other stuff. That’s the future that I’m excited about, and I hope that I get to see all of you there too. So with that, thank you. As as Bradley is making his way up, I am literally Jeff Blankenberg everywhere. I mean, everywhere including, like, TikTok. So if you guys wanna find me, if you wanna connect with me, I would love to meet every single one of you while we’re here, and I wanna thank you guys for all of your attention. One last thing I’ll mention as you come up, we do have a community of people that are thinking about technology just like this. And if you’d like to join us, it is very Alexa-centric for the most part, but people are talking about the big issues. You can join us at alexa dot design slash slack. This is a Slack community for people just like all of us to talk about what’s going on, how to build things, how to interact, and I think it’s a place that all of you would be welcome as well. So thank you. Certainly.

If you have any feedback about this post, or anything else around Deepgram, we'd love to hear from you. Please let us know in our GitHub discussions .