Robert Scoble - Futurist UploadVR at CopenX, September 2016

intellegence. What a great day. We are about to enter the fourth transformation of the PC world. The first transformation was getting the PC itself and getting a character mode interface where we typed commands to the personal computer and that was about it, right, when we had an Apple II and a PC. The second transformation was when the GUI came along and we clicked on icons and dragged down menus and got the what you see is what you get interfaces that we still use today on our Macintoshes and our PCs. When the second transformation arrived, big companies, Borland and WordPerfect, went away. That's a lesson for all of us when these new paradigm shifts or transformations come. The third transformation was getting a mobile phone and especially getting a touch interface. This radically changed the world as you all know. And more big companies went away. Borland and Nokia really are not what they were ten years ago. And today we're getting a new kind of interface, an interface that puts the computing onto the world itself. And so let's dive in and have a little journey into this fourth transformation. I work at UploadVR and hopefully this all works. And at UploadVR we have a green screen and a bright, we do media about the virtual reality world. I don't hear audio and I bet there's some audio on that clip but that's okay. You notice this is in the Google Tilt Brush and the user interface is on the world. You're walking around the user interface and you're seeing the user interface respond to the environment. If you play music you'll see the things shaking as the music goes along. You might have to hit plus on the audio levels. It doesn't work. It will. That's going to be a problem. This is my office. We do a lot of training. The first hackathon, the HoloLens hackathon in San Francisco was in my office. Philip Rosedale introduced his product in my office. I get a first-hand look on a lot of the things that are about to come and it's a really big honor to represent UploadVR here. This is my son who just turned nine yesterday. He loves VR. The kids take to this world instantly and don't ask questions. It's very different than previous transformations. I have a 22-year-old when he learned the Macintosh or learned the iPhone, we were first in line to get the iPhone in Palo Alto, he asked a lot of questions about how it worked, how to set the settings, how to move things around, how to manipulate the computer or the phone. That's not true in VR. They pick it up instantly and they don't need to ask questions because it works the way that the world works. We learn to pick up a bottle or touch people or hit things and it transfers instantly into this new world. I wanted to talk about the primitives because if I was introducing Windows 95 to you back in 1994, I'd need to talk about the mouse move messages, I'd need to talk about the APIs and the DLLs, the way that you talk to the program. Let's discover some of these primitives. I'm having trouble going forward. Can you hit forward on the Macintosh, please? Thank you. Hit forward again. Oops. This is the Google Tango phone that's coming out from Lenovo. The Google Tango sensors are building a 3D map of the world and as they do, they'll see all these chairs, they'll see you, they'll see the floor, and it builds a polygon map on top of these surfaces that then virtual items, holograms, can be put on top of or the surfaces themselves can be changed. That's what we talk about when we talk about augmented reality or mixed reality. This is a new capability that's coming and coming quickly to us. It's escaping the R&D labs and being productized and consumers are starting to expect this. We're training our consumers with apps like Pokemon Go, right? Soon, in fact, next month, Google's going to have a lot more announcements about its Daydream initiatives and hopefully show continued support for the Tango sensor system and the augmented reality that it brings. But you're going to be able to soon build new kinds of applications for mobile phones and within two or three years for mixed reality glasses, although we already have the HoloLens today, but the HoloLens is too big to wear around the world full time. I think when we come back in five years, we're wearing glasses, we're not using phones anymore, and there's a variety of reasons why I believe that's true. Let's dig into what Tango does. There you go. Tango sensors give developers new capabilities. You can map out the world in a new way and put user interface elements on top of that world or sense the world in a new way. Retailers are really excited by this because you're going to walk into a shopping mall and the phone is going to know exactly where it is in that shopping mall and it's also going to be able to see patterns in the shopping mall and put things on top of the walls, right? You're going to be able to build new kinds of applications that know elevation changes and going upstairs and going around roller coasters and doing other kinds of movements in the world that are just really impossible to do with today's GPUs and motion sensors that are in current phones. This is going to open up new kinds of capabilities and new kinds of experiences for us to all build with phones. Like a previous speaker said, I believe mobile phones are going to be how most people in the world experience virtual reality because I don't think most people in the world have $2,000 to buy an HTC Vive like I did and a PC. It's too nerdy to put lighthouses on the walls like we all do in our offices right now. The Tango sensors and the Prime Sense on Apple will let the phone know exactly where it is in this three-dimensional space and therefore we can do six degrees of freedom kinds of things coming soon. It's going to be really interesting to see this. This is one of the four problems with mobile VR and if Google solves this this year, that's going to really help this industry out. I'm going to skip a little ahead because you can watch these on YouTube, but there's a variety of new kinds of games that you're going to build, new kinds of apps, new kinds of things like being able to take a scan of your room just by doing that with your phone. And it knows the distances of the walls pretty precisely, precisely enough to then lay in furniture and have it accurately displayed to know that the furniture will actually fit in your real house when you get that new furniture. And of course, you're going to be able to lay new kinds of information lines like navigation onto the floor like a blue line that will tell me where to go for my next meeting or whatnot like that. So let's talk a little bit about mobile VR. I mentioned there's four problems with mobile VR that we experience in the world. And I don't know what's coming up next. One of them is this four, six degrees of freedom. Today with Gear VR, I can only turn my head like this. I can't walk around an item. I can't walk around a house. I can't walk around a house. I can't walk around an item. I can't walk around the world. And I need to to experience VR the way we all do with our HTC Vive and soon the Oculus Rift. This is a Zed camera. It has two cameras on it. It's using differences between the two cameras to build a 3D model of the world. And because it builds a 3D model of the world, it knows where that device is. And so we can now add six degrees of freedom to devices like the Samsung Gear VR. The camera is still, I think, too big and too expensive. But this is the thing that time solves. These sensors are going to be built into new phones really soon, within the next 18 months for sure. And can you get the audio working on my machine? Yeah. So this is the picture of the polygon map that the Zed camera causes to happen underneath. And if you've played with a HoloLens and turned on the developer mode, you know it does a similar thing as you walk around the world with a HoloLens. It's putting these maps on all the surfaces so then virtual items can be laid on top, virtual holograms and other things can be laid on top. So let's talk about mixed reality because this is where I'm really excited. I agree with Tim Cook who said yesterday that, yes, virtual reality is very interesting, but mixed reality or augmented reality is much more interesting than virtual reality, in part because it will do VR as well. So it will incorporate all the work that we're doing right now in the virtual world to come in. So customers already can do things like this. I saw this kind of demo four years ago at Matio, which Apple showed, right? And people are already getting magazines and they aim the camera at the magazine, and it does this kind of stuff. You can turn that down a bit so I can talk over it. Vuforia came out of Qualcomm, and Qualcomm designed this kind of virtual reality, but back then the phones were underpowered and the tablets were underpowered, and the use case just wasn't quite there. I think when we put it into glasses, this kind of use case is going to explode and really change our world dramatically. This guy is Rony who runs Magic Leap, and if you haven't heard of Magic Leap, I don't know what you're doing in this room. But Magic Leap got $1.3 billion of investment without having a product and without having a customer. So you know that those little glass chips do something to your brain to get you to pull all the money out of your wallet and hand it to this guy. Magic Leap... You're looking through Magic Leap's objects right now. Magic Leap is actually six techniques. You're looking through Magic Leap's objects right now. Magic Leap is actually six technologies wrapped into a glass. Let's just watch this a second. Notice some of the things that are really amazing here. First of all, the images are properly occluded. So when a R2D2 goes behind a table, it knows that it needs to go behind a physical thing. It also knows that this surface is a table because it's using artificial intelligence to categorize everything in the world that it looks at. So it knows this is a floor, this is a table, this is a glass, a bottle. That's a human, human, human, human, human, human, right? Floor. And it is able to put things where they're supposed to be. Because if I had a zombie attack coming down the aisle right now, it needs to go on the floor, not on the humans, right? So it needs to know a lot of things. This new world coming at us is already letting developers build magical new experiences. This is in the HoloLens. And let's just watch this. Hey guys, it's Paul. I'm here with the HoloHeroes again today for what we're going to kind of consider an app preview. So this app's not big enough to really do a review. It's just kind of a cool idea that someone did with the technology, kind of to give you guys an idea of what you can do with the technology. So I'm just going to bloom. I'm going to go ahead and do a little bit of a preview. So I'm going to go ahead and do a little bit of a preview. So I'm going to go ahead and do a little bit of a preview. So I'm just going to bloom. Bloom is a gesture that you do in front of the teleportation door. That brings up a menu. So I'm just going to place it. OK. So it was made in Unity, much like everything with HoloLens right now. OK, so you can kind of hear it. It's pretty loud music. So I'm going to say Play Store. So I'm going to place it right in front of you. You can see the polygons that the HoloLens is seeing in the world. So now it's on the ground. It's a little short door, but it's pretty cool. So now we're just going to say open door. Open the door. Come on. There we go. So now you can see there's actually a world in there that it's showing me. So I'm going to get in there. And yeah, it's just like a cool little world someone made in Unity, where if you walk through that door that I just walked through right there, you can see it's kind of like just showing what you can do with augmented reality. This is really mixed reality. It's pretty cool. In my opinion, this will kill. Once the graphics on this stuff gets better, this will kill VR. And it's just incredible because really good AR will definitely beat VR. So I don't know. It's just a really cool little app that kind of shows the capabilities. So now after we're done looking around, I'm just going to head on out. And yeah, let's do that. I'm walking through the terrain kind of here, but you get the idea. Then walk out the door. No holograms. Turn around. Close the door. Close door. There we go. The door is closed. And yeah, thanks for checking it out. So I visited a startup in Seattle called Look with three O's. And they work in a room like this, but they have no furniture other than a table with a Bluetooth keyboard on it. And as you put, you know, it's an empty room. You put on the HoloLens and all of a sudden there's 50 virtual light-ups all over the room. There's dancers in the corner. There's things on the wall. In fact, there was a window on the wall, virtual light-ups. And you look around and it's, wow, that's really cool. It's locked to the wall, right? And it's properly locked and it stays there, right? If you go there right now, it's still on the wall. It hasn't moved. It is geo-located onto the physical surface of that wall. And he said, click on it. So I took my hand and I clicked on it. And zombies started coming through the window and crawling into the space and started crawling toward me. It freaked me out. And I was like, oh, I'm going to die. And I looked at the mirror and I looked at the face and started crawling toward me. It freaked me out. Because it's, if your mind buys into the idea that this is actually something in the real world that's a threat to me, it's very interesting. So mixed reality is actually six technologies that are all melding together. And they're all evolving very quickly. The optics that Rony was holding up, there are trillions of micro-mirrors. If a mirror turns like this, you turn black, right? Because it's blocking the light from coming to my eye. If it turns this way, it lets the real world go to my eye so I can still see you. A second set of mirrors, or the same mirror, reflects virtual light into my eye. So all of a sudden you're SpongeBob. Hi, SpongeBob. Nice SpongeBob outfit. It has really high-speed wireless network. I just went to China last week to meet with Huawei. And they say LTE 5 is going to come around the world in around 2020. So we're four years away from really high-speed wireless. And LTE 5 has a few attributes. You can have many, many more devices today connected to a cell tower than today. So Internet of Things devices. If we have clothing that's talking to the cloud, or glasses and shoes and cars and thermostats and lights that are all talking to the cloud, the cell networks can handle that. It has lower latency, which you know latency is a killer because we want to play ping pong with each other over the Internet. Lower latency matters to make it more realistic. It has eye tracking, and we'll see that in a second. And you saw a competitor of what I'm going to show you this morning here in this room. It has room tracking. So HoloLens has four sensors that map out this world in three dimensions. And once it maps it out, it remembers that. And it remembers the space of that so that you can put a hologram on top of that floor. And in a year, you come back to this room, and that hologram is still there unless somebody has moved it. It has augmented audio. And audio is something that I think nobody is expecting what's coming because there's a variety of companies that are doing things like what we saw earlier with spatial audio. That's cool. But at Coachella, a big music festival, they sold me some Doppler Labs headphones. These headphones block the audio from getting to your ear, sort of like Magic Leap's optics, block the light from getting to my eye. And it has a microphone, and it has processing. And the processing has gotten small enough and cheap enough to put in an earbud now. Why did Apple get rid of the cord? This is why. It's coming. It's just a little expensive. It cost $200 for my earbuds from Doppler Labs, the Hear Active Listening System. And it has a speaker. So when I was at Coachella, I was at a music festival with $2 million of audio equipment aimed at my head. Pretty nice setup for listening to bands and listening to performers. But the sound from the headphones was better. It let me put the performer into a cathedral or a small bar or a sporting stadium because the processing can change how the audio is being processed into my ear. I could turn up the bass. I could turn down the volume if it was too loud. I can change the processing so I can do echo and all sorts of different effects to make the audio better. I showed this at a music festival in Alabama to about 20 people randomly, and they all said this is absolutely mind-blowing. They had no idea this was possible. And you can control it on your iPhone. The other technology is categorization. The artificial intelligence that's going to tell that that's a chair, that's a Macintosh, that's a camera, that's a human, that's a shirt, on and on. The more we train these systems to do, the better these systems are going to be able to mix our reality as we walk around the world. The artificial intelligence is also going to do a lot of fun things. If I had 70,000 images of badges, for instance, I could run that into Intel's artificial intelligence system and train it on how to recognize badges. So I'm going to walk up to you at a conference, and it's going to go, oh, there's a badge. Oh, there's a name. Oh, there's a title. Oh, let's go to LinkedIn. Why didn't Microsoft buy LinkedIn? For this reason. We're going to have this on our face, and we're going to walk around at conferences, and it's going to recognize your badge, and it's going to do a LinkedIn lookup, and all of a sudden all of your LinkedIn information is going to be sitting right next to you. I won't have this stupid thing that happened to me at Davos. I was talking to Peter Pyatt. I could see his name. He was at the Bill and Melinda Gates Foundation. It was on his badge, and I knew he was scientific and some badass, right? Because you don't go to Davos if you're not a badass, and all the other badasses in the room were listening to him and talking with scientific language. I knew he wasn't one of me and one of my community. I couldn't Google him at that point because there was no connectivity. When I could, Wikipedia was the first answer, and the first line in Wikipedia was, Discover of the Ebola virus. I wish it had told me that when I was talking to him, because it would have changed my conversation deeply. So let's talk about eye sensors. And this is Jim Margraff. He's the founder of EyeFluence. He's a competitor to the EyeTribe that you saw earlier. We're going to see which one wins. I don't know. But watch the screen as he talks. If I want to, I can change pages. Here's another close-up of some electronics in our office, and I can go home when I want to. Over here is a medical application. All with your eyes. All with my eyes. So I'm doing this solely with my eyes as fast as I can look. I'm not waiting. I'm not winking. I'm just looking. And here I've got the patient. I've got some allergy record, protocols, insurance, confidential information. Current conditions, why is he here? Well, he tells me he's got a pain in his foot. Notice I'm looking at this, but there's nothing happening on the screen. But when I decide, for instance, that I want to check out his X-ray, there it is. And now I want to go back because a couple of screens ago, there was some confidential information, which is here. And now it's going to take a picture of my eye. It grabs it, says, oh, who is that? Confirms that it's me. And in a moment, you'll see that it'll give me access. I'm Jim, head of CEO and founder of EyeFluence. And there it is. I've got confidential information. When I want to, I can return home as fast as that. That's amazing. All of my eyes. That's amazing. He only let me film about one-tenth of the demo. Afterwards, he pulled out a newer pair of glasses with better sensors. And he said, pull something out of your pocket. And I pulled out my iPhone. He said, look at it. And I looked at it, and a menu popped off the iPhone. And the menu said, would you like information or would you like to buy this? I went, I'd like to buy this. It took a picture of the iPhone, put it to Google Image Search, figured out it was an iPhone 6S Plus, went to Amazon, found the latest price, and it introduced that to me in my eyes. And I was able to say, yeah, buy one of those and have it delivered to my usual account, all with my eyes. So think about what's coming in a short period of time. These are the sensors. So you can see it's a real product. It's coming. These guys who are here in Copenhagen, same thing. They wouldn't let me take pictures of what they're doing, but they're productizing this very soon. They've told me that nobody understands how fast these glasses are coming. They're coming in two years to consumers. I think it's four to five. We'll see. But it's clear this industry has keyed up billions of dollars of R&D on a variety of glasses. There's nine glasses under development that I know of and probably a few that I don't know of. So there's a big competition. There's a big war. And there is billions and billions of dollars being invested in bringing these to the world. So let's see next to your favorite player. You look to your right, you see them and you look ahead. And the coaches there firing up the team. Hold on. You're sitting next to your favorite player. Actiongram is an app that allows you to bring holograms into the real world and create and share videos. Being able to incorporate visual effects into live action right there on set is a lot of fun. It gets your brain going. I was doing this earlier. It's great to work with holograms in the first place. That's already cool. But the fact that there are these iconic characters makes it even cooler. Action. Welcome to Pudgeley. He's sitting in an 8i camera system. 40 cameras are around him. This camera captures him in a volumetric way. And I see somebody there in an orange sweat. That's me. This is fascinating. We talked about holograms on Star Trek. And to actually be doing that in reality today, it was a lot of fun. Technology and being a part of it at the cutting edge is a thrill that very few human beings can feel. So it was really fun creating videos with the George D.K. holograms because he's just hilarious. Oh no she didn't. You get this really high bar of talent that you normally wouldn't get to work with. Love that. It's not every day that you get a chance to work with George D.K. Live action holograms that are three dimensional and function within your actual three dimensional space was mind blowing. I think it's a really exciting time for video creation and storytelling. You're now able to capture with these 8i cameras volumetric performances. The performances are so amazing. I wore a meta glass a couple of weeks ago and saw one of these volumetric captures. And it was like the dancers right here on stage with me. And you can walk around the dancer and you can walk right up to the dancer. The effect of the dancer falls apart when you put your hand through her. So we have a lot of haptics to come to make it possible to touch people and touch objects and all that. It's a 40 year industry. We just got the Apple II of this industry this summer, the HTC Vive. And it's 1977. It's 1977. And we just got a personal computer. This is what we just got this year. It's a 40 year industry. It's going to improve for 40 years. We're going to come back in 40 years and these things are still going to be improving just the same way our PCs are still improving after 40 years. So marketing. We heard a talk earlier today about some of the things that marketers are doing with virtual reality. Ford is taking pictures of their high end cars and letting you experience those. But Ford has a room in their R&D lab in Silicon Valley, a motion capture room. And they're designing the new cars in virtual reality. So everything is designed in virtual reality and you will experience it for the first time in virtual reality. Disney's new theme park in Shanghai was designed in virtual reality. And the guy who runs the ride design teams there told me they saved millions of dollars by doing that. And they could walk around the park and say, is this hamburger stand in the right place? If it's not, just grab it and we'll see a demo of one of those rooms later. Absolute Vodka told me that they did a music festival in virtual reality where they gave away 5,000 headsets. And they said the average time spent was 19 minutes in a Google Cardboard. So they said that was extraordinary for the amount of money that they had spent for this campaign. And they're going to be doing a lot more in the future. Snapchat. We've started seeing rumors of the new Snapchat camera. They bought this guy's company, Epiphany Eyewear, a Virgin's Eyewear. And two years ago, these pictures are two years old. He had a 1080p camera with a Wi-Fi radio that was a popsicle stick and it was stuck inside the eyeglass frame. That was two years ago. So how big is it today? I think it's a sugar cube, a sugar cube that you put on your shirt or your hat or your glasses and probably have a little screen on it as well. That's low cost because Snapchat's aimed at teenagers. You can't have something that's $1,500 or $3,000. But it's coming. The strategist at Snapchat said they're going to ship this this year. Now maybe he was a little aggressive, so next year. But it's coming in the next six months. Medical. I visited Cheryl Lee Calder down in South Africa. She works with the number one athletes, the top athletes around the world. The South African cycling team's owner told me that she took his worst rider in terms of falls per race. And today that rider is the best on his team at falls per race. How? She figured out how to hack our perception system. She's a former athlete and she figured out some tricks to make her eyes work better than they used to. And she put those into a computer program. Right now you would do two pieces of this with her. She'll do a test. So you sit in front of a screen and you follow things on the screen. And you have to remember numbers and type them in and do a variety of other tasks on the screen. She finds holes in your perception system and figures out your weaknesses. I'm 39 percent of her average clientele at perception. So I'm way behind some of the professional athletes, which makes sense. If I could hit a ball, I probably wouldn't be here. Right. She then has an app where you play it for 10 minutes a day. And she's building virtual reality to do this that exercises those weaknesses in your perception system and makes them better. She took the number one rugby player in the world who kept dropping balls coming at an angle about this way because he had a weakness in his perception system. He was best in the world at everything else. But that one test he was weak at. He now is number one in the league at that one task by doing her program for 10 minutes a day. So think about some of the medical advances that are coming that we haven't even started figuring out how the brain works and how to hack the brain. We know that virtual reality already is way better at pain relief than morphine. Way better. Not just a little better. Ten times better than morphine. So if you have some pain from a surgery, don't ask for drugs. Ask for VR. You're sitting next to your favorite player. You look to your right, you see them. Sporting is about to radically change because we're going to wear these glasses at the NFL or at the NASCAR and we're going to see things on the field. So let's just watch a little bit of us. Sell that same seat for $10 a ticket to 100,000 people. You can create an avatar of your friend and they'll be in that experience with you. But it's their real voice and it's the real movement. Now the VR has done well. There's no technology. There's no gadgets. It's not about pixels or refresh rate. It's an experience. There's complete mental transportation. You forget that you're even wearing the helmet. You are just mentally transported somewhere else. I do believe fans will be able to put on the head mounted display and have a highlight reel and get to experience a play and feel what it's like to be someone else for a minute. We like to say the brain has not yet evolved to differentiate a compelling virtual reality experience from a physical one. Soon this fantasy will be in households across the globe. Will it change the way people interact, the way people watch football? Will virtual reality change the world for the better? So right now people are watching sports on the screen, but VR turns the screen into the world all around them. What you can see in the stadium potentially could be what you see in your living room through a hologram. This is NFL Next. I visited NASCAR because I have a friend there who's an executive and she took me to a race. And if you sit in the pits, they have hundreds of sensors on the car. They know the heat of the tires on the track as Danica Patrick is doing laps. So they know in the pits how aggressive she's being with the car. So let's talk about some of the sensors that are coming to sporting that's going to let us watch sport in a whole new way. Statistical analysis and data collection have become commonplace in sports like baseball. Technology is driving new types of data collection and consumption, creating new possibilities for all sports. In football, we call this next-gen stats. We've always thought about football in a three-dimensional sense. In the future, there's a fourth dimension that we see. That dimension is information. We're in the initial stages of what we like to say is revolutionizing the game. Player tracking is going to bring new insights to the game never seen or never even imagined. Possibilities for the NFL's future are endless, and they begin with next-gen stats. This is NFL Next. Those of you that have been here before know that we like to open our meetings with a little icebreaker. H-TAC is a club that pursues the quantitative analysis of sports strategy and management. As Harvard students, we're not going to be the ones providing the best qualitative analysis, but we can try to answer some quantitative questions. We're in discussions with the New York Giants about real-time data. Stats and sports actually go very well together. Taking a more statistical approach can lead to gains of 5, 10, 15 percent. They have these computer chips in their physical pads. Through this, you can measure things like acceleration, deceleration, things like that. In order to understand the meaning of next-gen stats, we need to understand how and what kind of data we are collecting. The NFL, along with Zebra, are at the forefront of this revolutionary technology. Every player has two tags on them, one on each shoulder pad. 12 to 15 times a second, those tags on the player's shoulders are blinking. The sensors in the stadium are looking for those blanks, and as they see them, they calculate the position of the player by the number of receivers that see each of the tags. The stats that people are used to consuming for football, yards gained, yards passing, those are really scratching the surface of what a player does during the game. So next-gen stats is meant to measure all the other things. It's how far a player runs, how fast they run, how fast they run compared to earlier in the game, earlier in their career, earlier in that season. We're looking at a whole new future that largely centers around data. In the manner of printing the ball in the future, you'll be able to integrate microcircuitry to monitor the statistics and help an athlete understand his performance better by measuring spiral quality, measure things like throwing velocity and how well he throws a football, or even the timing sequence of him throwing a football. Data becomes a part of our lives, and being able to measure things that are going on with the technique of the athlete, feed that information back to him, is a port de mention that we see as a huge area for innovation for us, for our company. So what does all this data mean? Many tech companies are hoping it means new ways to exploit and enhance the game-dealing experience. The way we integrate next-gen stats into the broadcast is basically by following along the storylines of the game. Giants in Washington getting set to start on Thursday night football. We look to find big-time gains with running backs or wide receivers. We look to say separation between defensive backs and wide receivers. Manning back to throw, steps up, deep ball. Adele Buckham Jr. on the post. Now someone can say, is 22 miles an hour fast? You bet it is. It's number one in the league this week. Jumps up, makes the catch, touchdown Giants! In addition to CBS, Microsoft has been using the NFL's next-gen stats data in an even more unique way. What really got us excited about working with the NFL was trying to figure out ways where Microsoft's technology could kind of take the NFL experience to the next level. With one button pressed, I am now going to get the next-gen view. You kind of get that ah moment of like, hey, this looks like a game of Madden. But then you realize when watching the video highlight that they were actually tied and they're directly synced. We're getting all this data in real time. We actually kind of have a tachometer where we can see his speed during the play and then we'll actually stamp where he hit the top speed. He's probably in one of our top playmakers this week. I absolutely think these next-gen stats are going to change everything. We're starting to see things in a new light and the whole explosion of data and information that's being created. Tracking data is certainly the future of any professional sport. There can always be technology added to the NFL and the cool thing about that is sometimes you don't even know what's coming. We're still exploring how to best utilize it, but I think we're giving fans a better perspective with these data points. At all times I can look over to the right and I can see that play with a 3D representation of the play on my coffee table. I mean, that takes the game to a whole other level. Technology continues to provide new data and new ways to exploit it. We will soon see how this will affect coaching, players, and outcomes for professional sports. Next-gen stats and statistical analysis are here to stay. If you go to the 49er Stadium in Santa Clara, California, they have 2,000 beacons at the stadium. They're not just tracking the football players. They're tracking everybody. They know where you are within a meter of where you are full-time the minute you enter the stadium. They know where you are and that lets them add features for us. They can bring a hot dog right to you. They can tell you where the bathroom with the shortest line is because they have a volumetric sensor on the bathroom. They know how long the lines are in the bathroom. When you pull out your app, it works today. It worked at the Super Bowl. You can say where's the closest bathroom with the shortest line and it routes you there. It tells you how to get there. If you have the Tango sensors, there will be a blue line on the stadium floor that will tell you how to get the things. This is already happening at airports around the world. American Airlines and United Airlines apps now show you where you are in the stadium. The guy who built this app for the airport said, we know everything about the airport. We know where every power outlet is in some airports. Think about getting into an airport and needing some power before you go on a trip. You can just ask your HoloLens or your MagicLate, hey, direct me to a power outlet. A blue line will appear on the floor and it will take you right to the power outlet, which is right back there. This is mind-blowing stuff, but it's here. Let's talk about art. You already know if you have an HTC Vive that tilt brush is changing how we're doing art. It is enabling a new kind of design that is impossible to do in the real world. You cannot put a piece of paint in the air and have it stick there. Right? We know this. This is our world. We love this world. But this is a new spatial computing that's arriving. It's teaching us that we can do things in the world and we can walk around them. We can manipulate them with our hands and they're stuck to where we put them. Right? If I paint a line here and I go away for an hour, that line will be here in an hour. It's a new kind of user interface. And you see there's user interfaces that get onto the hands as well. This is a radically changing world that's coming at us. And now we can take these things and print them on a 3D printer. And we would not physically be able to make this any other way. Right? Let's just watch some of the creations that were posted last week from tilt brush. And notice that the media that we draw in these worlds can have properties that are physically impossible. The lines can change in response to my voice. And soon we're going to be able to grab that, put it into databases and share it with the world. And soon we're going to be able to grab that, put it into databases and share it with other developers who are going to take these art and mix them up and mash them up and do things in our own apps. A new medium is coming. And it's quite exciting. Let's talk about real estate. In Seattle there's a company called VR arcade that is selling condos using a motion capture room and VR. Why do that? Because I can see the real estate market. And I can see the real estate market. And I can see the real estate market. And I can send you a condo. You know, we have an art room and VR. Why do that? Because I can sell you a condo before it's built and I can have you design it to make you emotionally feel like it's your condo. Let's just watch as he manipulates this room. Just change the floor. An example of why VR and later AR is going to be so amazing, because you're going to be able to walk around the world with AR glasses eventually and change your entire world. You might think you're walking on a gray floor, but no, you're actually walking on a beautiful hotel floor. The picture was shot by a drone. In a boring place, right? And the mind could come up with infinite things for us to do in boring places. Really cool. So the picture was shot by a drone, so he upsells his customers. He says, okay, your view on the bottom floor is this, but if you want the penthouse suite for 500,000 more, it'll give you this view, right? And people emotionally can decide whether it's worth the extra money. We're doing new kinds of architecture. This was just posted on a website. You can turn this down so I can talk a little bit. So we're using HoloLens already to design things. I visited Saab down in Australia, and he's the defense contractor part of Saab, is using it to not just design new kinds of ships and new kinds of aircraft carriers, but they're rethinking everything about the aircraft carrier. I visited an aircraft carrier, and the admiral or the captain at the front now has multiple big screens, which show them the battlefield, show them an F-16 flying by, and who is that F-16, and what country does that F-16 belong to, and whatnot. You're going to be able to put these things on and realize the world in a whole new way, and this is just a little taste of what's coming to architecture. Some of this sounds a little science fiction and a little high-end, but many businesses are popping up around the world. This one is in Paris. This guy just takes 360 cameras into houses, records the houses, and then sits down with his real estate customers and says, here's 30 houses we can visit this afternoon, and you can experience what the house looks like in VR, and then we can pick three or four and be efficient about it. We don't have to go to 20 houses and waste your time and see a house that you might not like. This is what he's building, Virtual Reality has said. If you go to the Eiffel Tower in Paris, there's a new business right at the base of the Eiffel Tower where you can rent a VR headset and get a VR tour of Paris. Now, why would you want a VR tour of Paris while you're up in the Eiffel Tower and you can see Paris, right? But it's pretty cool, because you go up in the top of the tower and you see Paris. You see the Louvre is over there and Arc de Triomphe is there and Notre Dame is over here. But then you put on the headset and you click on or gaze at the different tourist attractions, and it takes you into the Louvre and lets you experience what it's like to visit the Louvre. So if you only have one day in Paris to visit tourist spots, you can only visit two tourist spots then, because it takes two hours to wait in line at the Louvre to get in the Louvre sometimes. So you can experience what the Louvre is like, right? From the headset costs 50 euros and there's lines, people waiting to rent these headsets. It's a simple business when you see it. It's not rocket science. Any of you could design the app in Unity to do this and do the 360 video with a camera like this Ricoh Theta, but it opens up new business opportunities, right? And some of us are starting to think, hey, there's a new way to make money with these things. There's new business opportunities that are sitting right here that aren't, you know, building the new HoloLens, right? Music. This is Soundstage. You saw a taste of it earlier. But this app lets a professional musician create new kinds of music virtually without owning any of the physical equipment. It is a representation of the real instruments that you would really play if you could afford a $2,000 synthesizer. But you can now play this in the HTC Vive and create radical new kinds of music as you play with the virtual instruments here. You can also design the lighting show for the LED screens that are going to be around you in a nightclub or at Oculus, Coachella or at a music festival. And you can design the lighting show in reaction to the music you're building on the instruments in Soundstage. It's a pretty cool app. It's a lot of fun to play with. Education is about to radically change. And I think this is one area where AR is not understood how deeply it's going to change. At Caterpillar, you already put on ODG glasses or HoloLenses, and it teaches you on top of the tractor how to fix the tractor. So you view. It shows you exactly what to do on the tractor and how to actually change the oil pump or change the air filters or change the different things that are wrong. And it teaches you. So you this is how we're going to save people's jobs after we lay them off with self-driving trucks and self-driving cars. We're going to re-educate people with these glasses to do new kinds of tasks that the computer isn't good at and that we need humans to do. And here's another view of HoloLens. And you can start to understand how education is about to radically change because groups of us can get together in a space like this and work on a virtual item or virtual surgery. See a virtual surgery and practice it over and over again until we get it right. And we can talk about it before we actually do the real surgery. We do the virtual surgery. And we can talk about it before we actually do the real surgery. We know that this works because Olympic athletes do this kind of visual visualization in their head. And this is going to assist us to do this kind of visualization and practice of things before we get and actually physically do them. Think about learning how to cook a new meal with these mixed reality glasses. It's going to have a professional chef show you how to make a meal and exactly how much salt to put in the meal. And then the 3D sensors are going to watch to make sure you do it right. And it's going to warn you too much salt, right? Because it's going to see the amount of things in front of you. It knows everything about the world around you sooner or later. And here you get an idea of how education is going to radically change. I mean, imagine being a teacher up here on stage with a hologram teaching an audience about a new topic and having the students wearing a HoloLens or a Magic Leap or whatever Apple comes out with or whatever WeChat comes out with and being able to experience a new kind of teaching that just isn't possible today. Social is one of the more exciting things about VR for me. I'm not a big gamer, so I didn't come out of the gaming industry like many of you in this room did. I don't enjoy playing a lot of video games the way my sons do. But with social VR, that has radically changed in my life. Because it's no longer a game that I'm playing with a computer. I'm just throwing frisbee with my friends who have an HTC Vive, right? And let's get a look at some of the things that are coming later this year with Oculus. This is a new version of the toy box. I hear there's even a newer version with even more mind-blowing things that they haven't shown us, but this gives us a taste. We're standing in a VR testbed that we've put together to test various experiments that we've put together to see how can interact together in VR. Let me show you a couple here. So this is a 360 photo like you were just talking about, but it's in a sphere. So why don't you grab that and slide it right onto your face. Whoa! There you go. Welcome to St. Pancras Station, London. We can go in there and get it. Yeah. Whoa, check out that clock. It's a really sweet clock. I've never noticed that before. Yeah, it's a fancy clock made by the Dent Company. They've been doing that for like 300 years. The same clock as in Big Ben. Let's keep going. Let's try someplace else here. I don't know about... Okay. Piccadilly Circus. You know, why do they call it a circus? Yeah, not so much in the way of elephants and clouds. They call it a circus because it's a circle of buildings. Ah, okay. Hey, it's one of those double-decker buses. You want to take a look? Yeah. This is really amazing. This is awesome. Okay. Hey, wait, wait. I recognize this. This is a place you can't otherwise easily get to. This is a killer. Whoa! Wow! Yeah, check this out. Look how big this is. You're just going to sense all the way here, from my right side... By the way, this is a plane that Facebook built to fly over our heads and bring us ultra-high-speed wireless for very low cost. We can put a communications bay in the middle here, and then we've got another... You told me ten years ago that Facebook would be building a plane. I would have said, what kind of drugs are you smoking? I had no idea. It was so huge. And what's really cool is I've never been here. So I feel like you've taken me to your seat lab. One more place to show you the other. So let's take a look at this here. Okay. That looks like Big Ben to me. And there's Big Ben. That's the Parliament. Where I knew is the one that up. Where is that? Oh, there it is. Right back there. The one and on. That's pretty sweet. Pretty sweet. Cool. Hey, they're getting a selfie in front of Big Ben. Oh, yeah. You know, I never really got a selfie in front of Big Ben before. I might have a solution for that. Another experiment we've been trying is the virtual side. Hey, you've got this sweet avatar, and I'm just like the blue band group over here. What's going on? Well, so using these, another experiment, these pins, you can draw on your avatar and customize yourself. But we've already prepared one for you. Ah, perfect. So I just put this on? Yes, a little bit on like a mask. All right, let's do this. Woohoo! Selfie! No, I think we need one more thing. Hold on one second here. Okay. Like this. Huh? Like this. So let me pull this in a little bit like that. Okay, and one more detail here. All right. What are you doing? Oh, it's great. Can I put that on? Yeah, hold still. All right. There you go. Okay. What have we got? Oh, sweet. Although, you know, this is a formal occasion. I think we need to hook you up too. So let's, uh, not a pencil, not yellow. Let's go with red or top. That's what I'm thinking. What do you think, Mr. Booth? So I said it's classy. I like it. Class this photo up a little bit. All right. Get rid of that. All right. Hold still, please. Perfect. All right. We can do this, right? All right. Woohoo! Then selfie. Yeah, all right. Selfie! Woohoo! Got it in here? All right. So when Mark Zuckerberg says this is the most social media he's ever seen, this is what he's talking about. Let's see what else we can do in this new virtual reality world coming soon. We can soon play... I already can play with my HTC Vive, and anybody who has a Vive knows this, right? You can play Rec Room, and you can play basketball and frisbee and mini golf. In other games, you can play Ski Ball. In other games, I can shoot you in the face. And on and on soon, I'm going to be able to go ski jumping with you, right? On and on, we're going to be able to do things with each other over the internet in a magical way. This is something, if you don't have a Vive at home, and you've only had a 15-minute demo out here, you just don't get. This is radically changing how we're going to play and work with each other, right? So thank you very much. Thank you very much. It's really an honor to be here at the home of Unity, where you guys are building the world. So thank you so much. I can't wait to see what you guys do next. Wow. Thanks for a great keynote. Just one question. I know a lot of you have a lot of questions for you, and we would love to make it possible, but I think we'll do it in the panel, because we have so much more time, so we'll have to do this. But just one question. You talked about we are in 1977. This is Apple II. Yeah, I remember getting my first Apple II at school. I went to school in Cupertino, a mile from Apple Computer, and it didn't have a hard drive. It had a tape drive, right? It was a software. There was no software you could buy for this thing when it first came out. We had to write our own software. That's why many of the people who got an Apple II are programmers today, because they learned that if they wanted to store some recipe data or make a contact list, they had to program that, and they learned the skill to do that. But like we say, that's 40 years ago. The companies right now, Apple is big, Microsoft, stuff like that. Where do you see the companies, the big ones right now, in let's just say five years from now? I don't know who's going to win. Magic Leap has an advantage with the optics. They figured out how to build an optic chip that blocks light from my eye and then puts virtual light into my eye, right? So it makes you black and then puts SpongeBob or whatever on top of you, and there's magical new things coming, and we're getting tastes of this with HoloLens and other things. But you mentioned Apple, and you mentioned WeChat. I know of nine companies around the world who are working on them, from Tesla, which is an unexpected one, to Snapchat, to WeChat, Alibaba just started a $200 million fund for VR and AR. In Israel, I know of two or three companies who are working on these glasses, and many companies who are working on the technologies that go into them, whether they be sensors of different kinds or different optical systems. I don't know who wins. That's what makes this interesting. But Microsoft alone, I have friends who still work there. I worked there for three years, and they said that Satya is betting the company on HoloLens. The whole company. He knows that the $14 billion businesses at Microsoft are going to depend on HoloLens in the future. We're not going to use a physical laptop in five years, or even a phone. We're going to have virtual screens all around us. The company I visited in Seattle, Look, was started by two guys who started the HoloLens team, and their desk, he has five screens like this sitting on his desk when you use the HoloLens. He's coding in the HoloLens. If you visit Meta, Meta is getting rid of the physical monitors in their office. They're just giving people the Meta glasses, and the coder for Meta is coding inside Meta, right? Because he can have five screens right here. And keep in mind, when you have virtual screens, it changes what software developers can do. So they're using a new piece of software that lets them see their code in a different way. So you click on a variable name, and it pops up five screens where it shows the code that uses that variable name. So now you can experience a new system in a way you can't with a physical monitor. You're not going to use physical monitors in a few years, and you're certainly not going to use cell phones, because why would you use a cell phone if you have this kind of computing all around you? It's a new frontier. It is. Robert Scoble. Thank you. Give a big hand, big applause.