Immersive Technologies and Virtual Reality
As immersive technologies are gaining popularity in the mass market, there is an on going discussion on how these technologies will affect our everyday life. From how we interact with the world - to how we play our video games. In this section, through short stories, case studies and technology/product reviews, I will offer my vision of the future.
Virtual Reality in Gaming -- Game Mechanics Review
With the rising popularity of the 3 main commercial VR systems (PSVR/HTC VIVE/OCULUS RIFT) several games that utilize the platforms are already out in the market, with more currently in development. As developers are struggling to learn the new language of VR-interactions, we are witnessing a very interesting period in gaming that moves far beyond the comfort zone of traditional video game production companies.
In this post I will try to identify which games have been successful (or lets say – positively viewed /rated by gamers) and break down some of their success reasons. In the process I will identify a few key points that game studios need to take into consideration while developing their games.
Navigation
No genre caused as much excitement to gamers as FPS when they were originally released. Wolf3d, Doom, Quake, were games that asked the player to become the faceless protagonist. You were not assuming the role of another person anymore, instead you were the hero. The genre promised – and did delivery up to a point - a much more immersive experience, but more importantly, it shaped the imagination of the players and created a vision for what could follow. First, we will see through the eyes of the player, then we will hold a gun in our hands and we will point and shoot, then slowly our whole body will be tracked and we will run and crawl and jump and shoot and BE in the virtual space. Low and behold, those same gamers that cried of joy when they killed Hitler at the final stage of Wolf3d, are probably some of the developers creating the first VR FPS games of today.
When the first FPS games where released in VR, developers tried to follow the formula they know. Place a virtual character inside a huge world. They quickly realised however that there is a big problem. When you attempt – and partly succeed – to convince your eyes that you are in a castle running away from Nazi soldiers, your body expects to be there too! If your eyes tell you one thing, and your body something else, then players quickly get the effects of motion sickness.
When game developers discovered this, they started experimentations with alternative ways to navigate big environments. With the introduction of Roomscale tracking, virtual game areas could be mapped to the physical space, allowing gamers to physically walk around. You want to grab a weapon which is 2 meters away from you on the floor? Walk there, lean down and pick it up. Roomscale though can only map an area of 3m x 3m and that is provided you have the physical space in your living room! But what happens when you want to move beyond the 3m? This is where the teleporter system comes along.
One of the first games that uses this system really well is Budget Cuts a stealth game that has the players infiltrating an office environment to approve their own job application.
Newer games like Arizona Sunshine, Raw Data and Vanishing Realms all use the teleportation system, which essentially allows you to teleport from one part of the world to the next. When this system is combined with the roomscale feature of the HTC Vive and the Oculus it offers a quite satisfactory solution at these early stages of the VR evolution.
Another very clever implementation of moving through different areas can be seen in GORN. In this brutal first person arena fighting game, just before you step out to fight, you can navigate through 3 different spaces by using an elevator system. The three spaces are lined up vertically, one on top of the other, and by going to the edge of the space you can use an elevator to travel upwards or downwards to visit the other areas.
During the latest 2017 E3, some newer games showed a number of innovating ways to travel greater distances. Sprint Vector asked the player to move the Vive controllers in a sprinting motion in order to propel through an obstacle course. Reviewers and journalists who tried the demo, commented favourably, so it will be interesting to see how this will turn out.
Another interesting example is Lone Echo, which places the player in zero gravity. In order to travel around, you need to pull and push around the environment using only your hands.
Beyond an FPS - Staging the World
In order to explore other game genres, firstly we must ask a simple question. If a VR experience needs the active use of our physical body – can we really make a VR game where the user sees and assumes the control of other bodies? Or in other words, will we ever get the chance to play, for example, Lara Croft in VR?
I believe the answer is yes, however the game needs to follow specific guidelines which are again related to the human body.
As humans, we are used to interact with physical objects in close proximity to us. Furthermore, every action we perform, is closely linked to our sense of proprioception and to an internal cognitive “map” of the space we are currently in. If somehow we lose these senses, then our body gets confused. Game designers have figured out a very clever way to use these facts in order to create 3d person experiences – essentially tricking our mind that we are not trying to “inhabit” the body we are seeing in front of us but rather "interact" with it, the same way we can pick up and move a toy soldier.
In order for this to work, the game world needs to be “framed” inside a space. This creates what I call the “table effect” as essentially the game tricks our body into thinking that all we see in front of us are spread out on a table – just like a board game. This gives the ability to our brain to satisfy its inherit need to create a cognitive map of the space.
Perhaps the best example to demonstrate this, is by looking at Air Mech Command – an RTS I will come back to a bit later on in this article:
Another very clever implementation of moving through different areas can be seen in GORN. In this brutal first person arena fighting game, just before you step out to fight, you can navigate through 3 different spaces by using an elevator system. The three spaces are lined up vertically, one on top of the other, and by going to the edge of the space you can use an elevator to travel upwards or downwards to visit the other areas.
During the latest 2017 E3, some newer games showed a number of innovating ways to travel greater distances. Sprint Vector asked the player to move the Vive controllers in a sprinting motion in order to propel through an obstacle course. Reviewers and journalists who tried the demo, commented favourably, so it will be interesting to see how this will turn out.
Another interesting example is Lone Echo, which places the player in zero gravity. In order to travel around, you need to pull and push around the environment using only your hands.
Beyond an FPS - Staging the World
In order to explore other game genres, firstly we must ask a simple question. If a VR experience needs the active use of our physical body – can we really make a VR game where the user sees and assumes the control of other bodies? Or in other words, will we ever get the chance to play, for example, Lara Croft in VR?
I believe the answer is yes, however the game needs to follow specific guidelines which are again related to the human body.
As humans, we are used to interact with physical objects in close proximity to us. Furthermore, every action we perform, is closely linked to our sense of proprioception and to an internal cognitive “map” of the space we are currently in. If somehow we lose these senses, then our body gets confused. Game designers have figured out a very clever way to use these facts in order to create 3d person experiences – essentially tricking our mind that we are not trying to “inhabit” the body we are seeing in front of us but rather "interact" with it, the same way we can pick up and move a toy soldier.
In order for this to work, the game world needs to be “framed” inside a space. This creates what I call the “table effect” as essentially the game tricks our body into thinking that all we see in front of us are spread out on a table – just like a board game. This gives the ability to our brain to satisfy its inherit need to create a cognitive map of the space.
Perhaps the best example to demonstrate this, is by looking at Air Mech Command – an RTS I will come back to a bit later on in this article:
As you can see, the game world is placed on a barrel like structure, which allows the player to walk around and interact with the objects.
The following screenshot, is by another well know 3d person VR game called Lucky’s tale. This is actually a zoomed out view of one of the stages of the game. As you can see, this time is not a table, however it still “frames” the game around a lake to allow the creation of a cognitive map:
The following screenshot, is by another well know 3d person VR game called Lucky’s tale. This is actually a zoomed out view of one of the stages of the game. As you can see, this time is not a table, however it still “frames” the game around a lake to allow the creation of a cognitive map:
Finally, here is another image of the yet another 3d person VR game, called Blaze Bowl that shows a similar space arrangement.
Now, you might be wondering, so what will the effects be if you do not “frame” the game world?
There are not a lot of games out there in order to test this hypothesis, however I would urge you to watch the following youtube video of a player testing out Edge of Nowhere, one of the available games that does not follow these mechanics (jump to 3.40 – no need to watch the intro of the game):
There are not a lot of games out there in order to test this hypothesis, however I would urge you to watch the following youtube video of a player testing out Edge of Nowhere, one of the available games that does not follow these mechanics (jump to 3.40 – no need to watch the intro of the game):
Interesting eh?
Immersion Techniques
One of the most immersive and highly praised applications is Google's Tilt Brush. This award-winning application (Proto Awards 2014 / Unity Awards 2015) users roomscale to create an empty 3d canvas - where you can go in and create art. Your main hand is dedicated to drawing lines of varying styles and thickness and your off hand, is used to select tools, colors, effects and brushes.
I did extensive research with this tool (Please refer to the Academic Work - Student's Projects) and the results we got from students where amazing. With only one training session of 15 mins and a further allowed time of 45 minutes of experimentation, students where able to create very impressive pieces of work:
Immersion Techniques
One of the most immersive and highly praised applications is Google's Tilt Brush. This award-winning application (Proto Awards 2014 / Unity Awards 2015) users roomscale to create an empty 3d canvas - where you can go in and create art. Your main hand is dedicated to drawing lines of varying styles and thickness and your off hand, is used to select tools, colors, effects and brushes.
I did extensive research with this tool (Please refer to the Academic Work - Student's Projects) and the results we got from students where amazing. With only one training session of 15 mins and a further allowed time of 45 minutes of experimentation, students where able to create very impressive pieces of work:
Now, why does it feel so natural to students?
The answer lies again in the human body. The player in Tilt brush holds a brush and pallet. Just like a painter in real life. It feels natural. It is natural. Google has replicated an action from real life.
The answer lies again in the human body. The player in Tilt brush holds a brush and pallet. Just like a painter in real life. It feels natural. It is natural. Google has replicated an action from real life.
|
|
Anything that feels familiar to the players can immediately create a greater sense of immersion. The player will pick it up quickly - which is really essential when trying to attract new audiences or trying to break into a new industry.
Superhot VR has developed a really interesting game mechanic. Time only moves, when you do. And this is probably the closest you will ever get to feel like Keanu Reeves in the Matrix. This mechanic promotes experimentation by manipulation something very familiar to the user. Movement of the their own bodies. It is challenging. It is exciting. It is something different and it makes you feel like a bad ass.
Since we are on the subject of Superhot VR, there is one more issue worth discussing here. In the game, you can pick up a number of objects which you can throw at enemies. As you hold the controllers, they turn into virtual hands in the VR space - just like in countless games. The natural thing would be to place your hands over the virtual object and then press and HOLD the trigger in order to pick up the object. When the trigger was released - the object would fall. You do apply pressure with your hands when you lift a physical object after all, right?
When the game was originally released, this was the mechanic that the developers implemented. However, these caused further issues as players complained that their fingers were getting tired holding down the trigger. So they had to change it - now you press one of the triggers to pick up the object and then press it again to release.
TO BE CONTINUED...
Superhot VR has developed a really interesting game mechanic. Time only moves, when you do. And this is probably the closest you will ever get to feel like Keanu Reeves in the Matrix. This mechanic promotes experimentation by manipulation something very familiar to the user. Movement of the their own bodies. It is challenging. It is exciting. It is something different and it makes you feel like a bad ass.
Since we are on the subject of Superhot VR, there is one more issue worth discussing here. In the game, you can pick up a number of objects which you can throw at enemies. As you hold the controllers, they turn into virtual hands in the VR space - just like in countless games. The natural thing would be to place your hands over the virtual object and then press and HOLD the trigger in order to pick up the object. When the trigger was released - the object would fall. You do apply pressure with your hands when you lift a physical object after all, right?
When the game was originally released, this was the mechanic that the developers implemented. However, these caused further issues as players complained that their fingers were getting tired holding down the trigger. So they had to change it - now you press one of the triggers to pick up the object and then press it again to release.
TO BE CONTINUED...
The Three Realities (and our body) -- Speculative
Let me start with this: No matter what you hear, no matter what popular movies (like the matrix and avatar) promise you - you will never be able to get rid of your human body. You are stuck with it - so I highly suggest you take care of it.
So what does this leave us?
If we can not get "plugged in" into the matrix, if we can not "download" our consciousness into another body, how are we suppose to use our own body to interact with worlds beyond our own?
As immersive technologies evolve and personal headset, glasses and VR/AR contact lenses become available, people will move away from the screens and their mobile phones and we will integrate into an ecosystem with 3 different states of existence:
1) The real, technologically unmediated reality
2) Mixed Reality - Reality augmented with a layer of digital information
3) Personal Reality - An isolation space, existing and adapting in real time to the dimensions of the real space, where a user will interact with personal data and facilitate social interactions
As part of our everyday life, there will be a quick and fluid movement from one state to the next.
The best way to explain the concept, is probably with a couple of user scenarios:
User scenario 1:
A night in
Wade is sitting comfortably in his living room, along with his girlfriend Molly. They are holding hands and they are staring intensely at the wall. The living room is very minimalistic. The space is not very large, its rectangular in shape with clean flat white walls and a clatter free wooden floor. They are sitting on a white sofa, which along with the two side tables at each side, they are the only furniture in the space. Wade prefers it like this. It allows for more customisation.
Wade and Molly are watching a movie. There is no physical TV in the living room, just a set of speakers embedded in the walls. They are both wearing their ACLs – a set of Augmented Contact Lenses that allows them to shift between their 3 different “realities”. Currently they are in the Mixed Reality mode. They can see the physical world superimposed with digital information. Even though they are sitting in a empty space, what they see is completely different. Molly has decorated the space with virtual objects. Opposite them, on the big flat wall, there is huge TV. On their left, there a big world map, with little post it pins on the places they would like to visit. Next to that is their calendar and on the other side of the room, Molly has Café Terrace at Night hanging by the windows. Wade always hated that painting, and even though Molly doesn’t know it, Wade has “hacked” the painting and instead of Van Gogh’s masterpiece he now sees an image of his favourite video game character.
Molly suddenly gets up. She needs to take a bathroom break. They could of course continue the movie – Molly could undock an instance of the TV, place it on the top right of her viewport and take it with her to the bathroom – however Wade prefers to take this opportunity to take a break. With a simple twist of his index finger, a customized gesture he chose which is elegantly tracked by the ring he is wearing, he is suddenly taken to his “personal reality”. This is his own personal space. His command centre. His social space. His game space. Two of his friends are currently logged in, sitting around a table discussing the latest on the sport they follow. His eyes drift to the empty chair. He focuses and blinks his eyes. His is teleported on the chair and joins the conversation. He quickly loses interest though because the space is full of distractions. Avatars of off-line friends are trying to attract his attention with messages. Game characters urging him to return to the game and annoying salesmen parading to sell him products. As Wade was making a mental note to update his ad-blocker, Molly returns back to the room. Wade sees an outline of her body, in his personal reality. Her avatar’s head pop in: “Come out come out, I wanna see what happens in the film!”
User Scenario 2:
House of Eternity
Enzo is sitting in his childhood bedroom, shifting through the dates slider. Ethereal figures of his father, mother and sister flicker in front of his eyes. He goes all the way back to the year he turned 13 and finds Christmas Day. He remembers – probably from the last time he played the experience, not from the actual day – that his father was especially happy that day. It was one of the few occasions that the whole family was home. He minimizes the UI and gets up.
He takes a quick look around his bedroom, the bed and cupboards are still in the same space – he left them intentionally like this after all. His eye hover over the adolescence posters on the walls, which he swiftly ignores and walks out of the door. As he passes through the long corridor connecting the bedrooms to the main house, he notices his little sister running towards him. “Wow.. she was so young..”. Before he can move out of the way, his sister passes through him and disappears into her bedroom. He keeps walking down the long corridor and turns right into the kitchen. Immediately he notices a number of red 3-dimensional areas superimposed on his vision. This is from the system warning him that physical objects are actually there right now. The kitchen has been renovated since then and furniture have been moved around. He carefully navigates the space and approaches his mum, who is currently leaning over the stove and tasting the casserole dish. His grandparents are also present, sitting on the opposite side, deep in a cheerful discussion. Enzo quickly realises that he doesn’t have audio. He checks his pockets for his headphones and puts them over his ears. His mum is singing – oh how he missed her voice.
Enzo fast forwards the day until the moment his dad comes home. He walks through the door, cheerful as always, with little Enzo following behind. He always found it strange seeing himself in a simulation. He noticed all the little imperfections. The funny way he walked, the way he talked, the way he sat on the table.
“Dinner is ready people!” says the mum, inviting everyone to sit around the table.
Enzo searches around for the green highlighted chair. The green chair is not part of the simulation. It is a physical chair he previously tagged in the system commonly know as the “Travellers chair”. A comfortable sitting space, from where he can fully enjoy the experience. He walks over and sits down.
Enzo decided, long ago, that he will never sell his parents house. Most people have "downloaded" the experience on their home units. They can relive every moment like a computer game, from anywhere in the world. Not by walking around the physical space but by using a simple gestures controls to move from room to room. He does not like that. After all, having them move around the house helps him to feel less alone. Even though his family has passed, they will always be there. Even after he is gone.
House of Eternity
Enzo is sitting in his childhood bedroom, shifting through the dates slider. Ethereal figures of his father, mother and sister flicker in front of his eyes. He goes all the way back to the year he turned 13 and finds Christmas Day. He remembers – probably from the last time he played the experience, not from the actual day – that his father was especially happy that day. It was one of the few occasions that the whole family was home. He minimizes the UI and gets up.
He takes a quick look around his bedroom, the bed and cupboards are still in the same space – he left them intentionally like this after all. His eye hover over the adolescence posters on the walls, which he swiftly ignores and walks out of the door. As he passes through the long corridor connecting the bedrooms to the main house, he notices his little sister running towards him. “Wow.. she was so young..”. Before he can move out of the way, his sister passes through him and disappears into her bedroom. He keeps walking down the long corridor and turns right into the kitchen. Immediately he notices a number of red 3-dimensional areas superimposed on his vision. This is from the system warning him that physical objects are actually there right now. The kitchen has been renovated since then and furniture have been moved around. He carefully navigates the space and approaches his mum, who is currently leaning over the stove and tasting the casserole dish. His grandparents are also present, sitting on the opposite side, deep in a cheerful discussion. Enzo quickly realises that he doesn’t have audio. He checks his pockets for his headphones and puts them over his ears. His mum is singing – oh how he missed her voice.
Enzo fast forwards the day until the moment his dad comes home. He walks through the door, cheerful as always, with little Enzo following behind. He always found it strange seeing himself in a simulation. He noticed all the little imperfections. The funny way he walked, the way he talked, the way he sat on the table.
“Dinner is ready people!” says the mum, inviting everyone to sit around the table.
Enzo searches around for the green highlighted chair. The green chair is not part of the simulation. It is a physical chair he previously tagged in the system commonly know as the “Travellers chair”. A comfortable sitting space, from where he can fully enjoy the experience. He walks over and sits down.
Enzo decided, long ago, that he will never sell his parents house. Most people have "downloaded" the experience on their home units. They can relive every moment like a computer game, from anywhere in the world. Not by walking around the physical space but by using a simple gestures controls to move from room to room. He does not like that. After all, having them move around the house helps him to feel less alone. Even though his family has passed, they will always be there. Even after he is gone.