Virtual Reality Technologies

Okay, let's try again. So we are technology. So just a few words about the place I'm from. It's called the Alexander Institute. We're a GTS institute. It means we're a privately owned but non-profit company. And our mission is to get new technology research into application in Danish industry. And it means that we're here for the companies to bring new knowledge to them. But we're actually also here for the researchers so that their research can be brought into industry and they can get new cases. And it's all because we want to give companies several years of head start. And we're here in Aarhus and we're also in Copenhagen. When you work with us, it's either because you buy something on a commercial basis, it's because we do this wide knowledge exchange, or because we do applied research development and innovation. And we have a broad set of expertise within different areas. So I'm head of the visual computing lab. So our expertise is very much within the visualization simulation but also interaction of course. And the reason I'm interested in all of this is because since I can remember I've played around with computer games. I love playing them. And my mind was really opened back in the late 80s when I was old enough to buy the first set of computers. So I started out with the Commodore 16 and couldn't do any kind of 3D graphics. But the Commodore 64 could do some very, very simple graphics. It took ages for each of these images to be rendered. This was kind of an adventure game, the Driller from 1987. I also bought the Amiga at some point and I was blown away by 3D graphics actually being interactive. And I could drive around in this case as a stuck car race driver. And so I was amazed by these essentially virtual worlds that I could experience as a child. And so I couldn't let go and I continued playing. So I played Doom of course and Kouhenshi on the PC. And each of these games pushed the boundaries of what you could do. This was back in the days when 3D graphics was all about programming something in software. So the guys who did the development here had to be very creative to actually push this kind of interactive graphics at interactive rates so that I could experience virtual worlds. And I still couldn't let go. So back when I did my education I became a computer scientist. And I just had to do virtual surgery. So again, building virtual worlds. But now not only that. Not only for entertainment but for something that has to do with life and death. And just to make a good story out of it. It had to do with children and malformations in the heart. And everybody can understand the soft story of children and something being wrong and we needing to fix it. And in this case we wanted to build better tools for surgeons so that they could be better prepared for actually doing the surgical procedure. And so back in the days when I did my PhD. We also experimented with some of the first head mounted displays that you could get at that time. And also some of these either active or passive stereo screens. Big screens or small screens. And we even ported the surgical simulator to this augmented reality experience. Unfortunately I don't have any images from that. But that was back then. And so in around the 2000 or 2006 the technology still wasn't mature enough. So it was these big clunky things. You had to have on your head. There was no commercial drive. And there was nothing really pushing the boundaries. And you've already seen these pictures. So Matt showed them to you. But didn't have the year. So back in 1956 Morten Heilig was one of the first inventors of something we could call virtual reality with the centorama. So this was kind of this thing where you could put your head into it. And you could then sense something. So unfortunately I haven't tried it. Of course I was born way later than that. But as the story goes it was with smell and with the visuals. And you get the auditive as well. One of the first head mounted displays was by Ivan Sutherland. So one of the great guys in computer science. But back then the head mounted display was something that had to be ceiling mounted. So it was extremely heavy. And it was heavy for a number of years. So this definitely wasn't the area where things really came to be. But it was something. But it was the area where things began. VR has been here almost before. So from what I can remember we had this hype around the 90s. Where you had the Sega VR. So it was almost a product. It was shown at big product launch events. But it didn't become a product. The other two, the Virtuality 1000 CS was driven by an Amiga 3000. And it was actually a thing. That you bought for big arcade halls. And you could try it. And you had this relatively simple games with of course very simple polygonal models. But you could shoot. And you could do stuff. But you had this quite narrow field of view. And you had slow updates of your head movements. So you got quite nauseous trying it. Also you had this Victor Max Cyber Max model. That you could actually buy as a consumer. It had a very short span or a lifespan unfortunately. Because well for some reason the hardware was just not mature enough at that point. So you got really sick. You had a really low field of view. And low resolution. So people have had the idea of virtual reality for a long time. It's almost been there a number of times. And of course everyone here hopes that this is the time when virtual reality really will kick in. Now why is it that it does kick in? It all has to do with the mobile revolution. So the reason we can now have screens of a very high resolution. That can be put into the virtual reality equipment. And the reason we can have accurate tracking. So you have the gyroscopes. You have the accelerometers. So that you can actually track the head movement. It's all because of the mobile revolution. If we didn't have these mobile phones. We wouldn't have the virtual reality equipment today. And one of the initial companies to see this is of course Oculus. And you've heard the word Oculus said a number of times today. But just to give you a bit of the back story. So Oculus did a Kickstarter in 2012. They were the first mover in this market. But they've only just released their consumer product this March 2016. But there's been a lot of delay on their products. We still haven't received our version of the consumer version of the Oculus Rift. But I was lucky enough to try the Crescent Bay. So the developer version that was just prior to the Oculus Rift consumer version. And that one is incredible. So I mean everyone is hoping for the consumer version of Oculus Rift to be great. For some reason HGIV that was co-developed with one of the big game companies, Valve. Has come for us at least, has come before. So back at the Switch Box in Aarhus Noor. My version of the HTC Vive is just waiting for me to pick up. So I'm so excited. I'm almost going to run out the room to get it. But of course I'm going to stay and talk. Don't worry. So the reason AR and VR is important to us is that as a GTS institute we have these strategic projects every three years. Where we're sponsored by the Danish government. It's called for us a GTS contract. It's like a research contract where we promise to bring in new technology. And to in essence give it out to all the Danish companies. Or at least give knowledge out to the Danish companies. I'm yelling so loud. Everyone can hear me. Anyway. Okay. Yeah. So we're going to focus on augmented reality and virtual reality for industrial training and instruction for the next three years. And we think this is an important field because gaming as a market is going to drive the selling of virtual reality equipment. And maybe actually spectators or spectating of either sport events or e-sport events is the thing that's going to drive it. But there are many exciting applications within the industry as well. So we have this strategic focus. And I'm going to end up showing you a slide where I will invite you guys to be part of this project as either a participant in our steering committee or as a case member. But please come up and talk to me if this is something that has your interest. We did some initial very short experiments. For example with the GONFOS. Oh God. Oh. Oh. And I can't for some reason turn off the sound. Thank you. So I mean this is one of the very simple cases that you can already realize today without having any kind of advanced equipment. So in this case I have to put this cooling tape on some specific equipment and they need to be guided. So part of talking about virtual reality and augmented reality is also about how do you train or how can you use this kind of equipment to easily train people and students. And in this case a simple iPad can do it. And there is a bunch of standard equipment that you can use to get this kind of experience. Just looking a bit abroad. I brought, so this is not our work. This is work by one of my international colleagues called Didier Strecke. He's from a place called the DFTI in Germany. And it was just to give you a broader view of what does it mean to use augmented reality. Augmented reality or virtual reality is training. So one of the huge issues that you may have kind of gotten from the talks today is about content creation. And how do you actually produce the content without being a 3D graphics artist. And they solved it quite elegantly in the sense that you can see this kind of overlay that is on the instruction that you get right now. He's supposed to switch out the RAM on this notebook. And everything here is just a video recording. Overlaid on then an augmented reality experience. So you don't need to do anything 3D in essence to get training instruction from augmented reality. And I think this is one of the huge things that the community needs to do something about the content creation. And how do you do easy content creation? Who can do the content creation? So that's going to be an issue. So technology, when we talk about virtual reality, has also to do with what kind of visualization equipment or screen reader. What kind of visualization equipment or screens do we use? It's all about tracking and interaction for tracking. It's definitely about visuals. And you've seen a lot of visuals today. Also about the content creation here. And then I put in the parenthesis the sound. So sound should be important. Not many people talk about it. I'm not an expert in sound in VR. But it is going to be one of the important things so that you actually feel you're in it. So if you don't already look into sound, please do. So VR, of course, can be the helmets you wear. But just to broaden your perspective, I forgot who, but someone said, oh, yeah, Jesper, Bendix, talked about cave. So you may not know what a cave is. It's this often rectangular room where you have projections on all the walls. And then you have these special glasses on so you get a stereoscopic or 3D effect. So that's virtual reality as well. That was the old style virtual reality where you had these centers where people need to go to get a virtual reality experience. We also have various virtual reality centers. And then you have these glasses that can give you a virtual reality experience. The two, bottom one up there are the active or passive stereo glasses that you can use if you have a cave wall or you just have a screen or you know it from the movie theaters, of course. But you also have the anaclip stereo where you filter out the right and left eye with colors instead. And then one exciting thing you also want to think about when you think about VR technology is projection technology. So you can actually mix the 3D experience from a real world experience. With the projection of images on top of that and get some incredible experiences from that as well. And I think I'm going to due to time I'm going to not show you the video. But I am going to show you just this video. A small crazy example that they did at Aarhus University. That also has to do with projecting onto stuff. But in this case what they did is they projected onto the wall. And they projected onto something that can change its shape. And so they did it because they imagine this future where your technology, your phone for example may change its shape. But I think it's interesting just because they project onto something and they know what it can look like or what the configuration of it is. And so this is another kind of augmented or virtual reality and another kind of medium. Even though I know everyone here is probably excited about the virtual reality from augmented reality. From Oculus and Samsung. Another thing you want to consider is the 3D effect you get in TVs and in the movie theaters. And as you may know all the big manufacturers of televisions are going to drop the 3D support. Unfortunately. So LG is dropping it. Samsung is dropping it. So it just didn't catch on. And that's a big fright for everyone probably in here and new here. We included who are nerds about VR is that I'm actually scared this may not catch on right now. I really, really want it to catch on. But I mean 3D in televisions definitely didn't. It's still a thing in the movie theaters. We haven't seen this big drop. But for some reason that was a fad. It was just a thing. It didn't really catch on. So let's hope it catches on. VR is also about tracking. So not many people today talked about interaction as a high level. It's a hardware thing. But it's a big thing in VR because I can't use my keyboard anymore. I can't find it. And so Oculus is shipping with the, you saw it over there, with the Xbox controller. But then that's also a strange thing to put into a consumer's or a customer's hands. So they need to learn to control it. So interaction is difficult in VR. There are a number of options here of course. Many of you may have tried the deep motion that can track your hand movements. That may work. It may be more intuitive when we talk about interacting in VR. But you still can feel anything and you need to be very close to the deep motion sensor. You can also go for something like the Phantom Omni device. It's actually a device that's been there for many years. I used it in my PhD back then. It's this haptic device where you can actually get, you can feel the resistance of something so that you can pull or you can feel the surface. So that's a very nice small piece of equipment. Unfortunately, there is no big market behind that one. So it hasn't been developed for a number of years. Rokoko down there is something I find quite exciting. It's a Danish, I forget if it's a kickstarter. It may be a kickstarter project where you have this full bodysuit that is not visually tracked. That's what you normally do in the visual effects industry. But it's tracked by sensors instead. So that may also be part of the interaction that you may want to use for future virtual reality equipment. That could be an exciting one. Then of course, you have the PlayStation Move. So these gesture devices. You have the Wii Move, the Wii Mode. That is also in itself very exciting. Again, I just want to show you this video. So an early one, before we talked about VR. This incredible guy called Johnny Lee. He's now employed at Google, working on their Google Tango phone. He did some very early strange stuff. He did some very strange VR where he mounted a B-mode. And in such a way that he could track the head movement. And of course, this being a video, being flat, it's kind of cheating because you didn't have the stereo effect. But having tried this, you actually get a large part of the effect still from this depth into the screen effect. So this is like an early VR display. But without the stereo. And maybe it's augmented reality. Because it's actually in real life. But it's on a screen, of course. So there's been a lot of experiments in different kinds of VR equipment that goes beyond this screen that we put on our head right now. Now this one I'm very excited about. So I'm not going to show you this video. But just to explain that it's this mount that you have on your fingers. So that you can get haptic feedback through each of your fingers. And the exciting thing here is that this is a new... I don't think it's really a product. But they try and launch it like a product. They want your money, of course. But back in the days when I did my first experiments into virtual surgery, this was also a thing. So we had these crazy Japanese researchers who do these big mounts where you had the mounting on each of the fingers. The intra-big rack mount behind you. And you could feel stuff. So all of the things that we've been dreaming up as researchers is kind of coming into life now. Because interaction in VR and also AR is going to be the big difficult thing. So Oculus is trying to solve it. HCI with their Vive head-mounted displays is trying to solve it. And right now their best answer is on the right side you can see the standard equipment that comes with your HTC Vive head-mounted display. And on the left side you can see the expected device that comes with the Oculus at a later stage. Right now of course it's the Xbox controller. So these ones will give you something that can track in 3D with rotation. You'll have lots of buttons you can press. And we hope that this is going to be the standard interaction device for any kind of application in virtual reality. But in essence we don't know if it's enough or if it's good or if it even applies to whatever you guys might want to do. I'm going to skip a few slides here. But I'm just going to... I'm going to point out the fact that what we've heard today is also about content creation. And one thing is to do with real-time content creation tools could be either the Unreal Engine that hasn't been mentioned today that can also export to VR. It has very nice visual effects. Does anyone use the Unreal Engine? No. One. Yes. Very good. And then of course you have Unity that for many people is a standard way of doing interactive virtual experiences. And then you've also heard about... The content production for movies or still images. So either Autodesk 3D Studio Max or Maya. They each have a set of renderers that you can use to create this content. And you've also heard about the 360 video where you need a special video camera to record this kind of equipment either in mono or in stereo. The other thing that I want to show you just for a comparison chart is a set of headsets. This isn't every one of them because there are a lot of headsets out there. But this is the most important ones. And I included what I call the DK1 and DK2. So the developer versions of Oculus just so you can see and compare some of the numbers here. And one of the very important things that you may want to see on the third row is the increase in resolution. So that was one of the major things that the first DK1 was lacking. That was the resolution. But one of the things that isn't mentioned in the chart such as this one is the fact that what really works with the VR equipment today. Is a very short latency in movement to something that comes into your eye. So a photon coming into your eye. And also this quite large field of view. So the fourth one says that we can actually see a lot of the virtual world. So that was part of what was wrong with the virtual equipment that came before this. Is that you had this narrow field of view. So it felt like this small monitor in front of you. And that is supposed to also be the problem of course with the Microsoft HoloLens. That you have this very small. Very small field of view. So you don't really get the immersive experience. But many of these are available. I included one that is not available. The PlayStation VR. Previous called Morpheus. So that one I'm looking very much forward to. Because I have a PlayStation at home. So I want to play games. Right. You've also heard that there are a number of mobile phone variants out there. And so you have the Gear VR of course being one of them. But there are a number of. A number of mobile phone variants that you can buy. The merger VR up there is exciting because it has a hole for the camera. So you can actually do this cheap augmented reality experience with mobile phones. The LG one for LG phones is special in the sense that it has a built in display. It doesn't reuse the display of the mobile phone. So that's kind of either strange or exciting depending on your point of view. But one thing that is exciting I think is the potential to do eye tracking in virtual reality. So we did some early experiments. You can see that we destroyed one of our developer kits for the Oculus Rift to include eye tracking. And we used some research based eye tracking software. Included it and did a small project together with Adlinko. They do beautiful designs. But they want to know how people experience design. Where do they look when they see something? And in virtual reality we can track where people look. And we can map it onto the 3D object. And so you get additional data when you inspect new design in virtual reality. And not because of us but because the rest of the industry sees this as well. For example the full VR is going to come out at some point. We don't really know. Maybe in May. That's what they promised at some point. It's going to include tracking of your eye movement. So that can be one way of interacting with your content. In that you look somewhere. That's going to be what you select or where you go. So is VR going to catch on? Well I found this survey from NewSoup. They do surveys for gaming and eSports. And they asked an online community. And what I don't know is if it's just the broader online community. Or an online community that has to do with games. I hope it's a broader online community. People will buy the equipment when it comes out. And it varies very much from country to country if they are going to buy it or not. So we need these people to buy it. So that the industry can come running up with new VR equipment. And buying it has very much to do with the cost of it. So this is another part of their investigation. Is how many people actually have money? actually have money enough to buy the high-end VR equipment. And it seems that 28% of the online community in Canada has enough money, so more than $600, so that they can actually buy the equipment. But in my mind, that's actually a very low number. So who is going to buy it? And are they going to buy it just because it's VR, or does it need to be included in their existing hardware budget? We don't know yet, because it's only just launching this year right now. But we hope it will catch on. So I almost used up all my time because I had the initial beginning issues. So I'm not going to say very much about AR, but I'm just going to agree that AR is going to be the next big thing after VR. But it's not right here right now. It's not even in the next three, four, or five years in my mind, because there are so many issues you need to solve if you really want a head-mounted display for AR. And it definitely doesn't... It shouldn't come too early, because people are going to be very disappointed about it. But it's extremely interesting, nonetheless. I'm just going to show you this last one slide so that you can come to lunch. So we're going to look more into augmented reality, virtual reality. And we want you guys to get involved. And so we have a number of choices to get involved. You can be part of our advisory group. So point us towards the real world. You can either write opportunities for the broad industry, or you can join us for some of the cases that we're going to build. We're going to do it both within broad industry, but also within some of our specialist areas, such as the medical industry. And we are, of course, going to do a lot of workshop talks, conferences. We promised that as part of the application. So, I mean, the evil empire... No, I mean, no. We want you to join us. And you can write me if you have any questions or if you're interested. Thank you.