Mixed Reality is an exciting, relatively new technology that merges Augmented Reality and Augmented Virtuality to create a new world where the real and the virtual co-exist in harmonious totality. This is a new hybrid reality, with many great possibilities the likes of which we have never seen before and what’s even more exciting is that this technology is already here.
Mixed Reality: An experience of two different worlds.
In order to be able to understand what Mixed Reality (MR) is, we first need to understand both VR and AR. Articles on Virtual reality (VR) can be found here and Augmented reality (AR) here and these can serve as a general introduction to the subject of Mixed reality (MR).
In a nutshell, in Virtual Reality we are immersed into a virtual, computer-generated world. In Augmented Reality, on the other hand, objects are overlaid into the real world with the possibility of spatial registration (spatial registration is the use of coordinates when imposing images, thus achieving a sense of reality or in some cases scale in the virtual objects). In Mixed Reality, however, objects are overlaid and anchored onto the real world, meaning that any virtual objects generated by the computer are aware of the objects that reside in reality, thus attempting to blur the lines between what is real and what is virtual.
We said earlier in the introduction that MR combines Augmented Reality and Augmented Virtuality to create a mixed reality. Let us now examine how these two different, but complementary technologies, contribute to MR.
Whilst AR is the augmentation of reality, AV is the augmentation of a virtual objects. In AR, one sees virtual elements superimposed on a real, physical reality. In comparison, in AV one sees real, physical elements in a virtual representation. Confused? Let’s use an example.
Imagine an airplane that develops a fault mid-flight. Nothing serious, don’t worry, but worthy of attention, nevertheless. Using AV, an engineer in an office would be able to view a virtual representation of the actual engine that is airborne, and thus be able to visually examine the fault without leaving his or her office. This will allow the engineer to assess the damage and determine if any corrective action is required or not for the safety of those onboard.
In Mixed Reality, what we essentially get is the benefits of AR and AV mixed into one technology. Sounds too science fiction? I’m happy to tell you that this technology is far closer than you think. Companies such as Microsoft and BIS are already in their second or third iteration of such technologies including HoloLens2 and Virtual Battlespace 3 respectively.
To see HoloLens in action, a TED talk by Alex Kipman showcases Microsoft’s technology using special cameras that allows the audience and you to see what its like behind their coveted headset.
In MR computers understand and interpret what’s around them; whether it’s, the environment, or objects residing in that environment including people and other physical objects. This can be accomplished in a number of different ways such as with the use of cameras, biometric sensors, heat sensors and speech recognition to name but a few, technology that already exists in most smartphones today. It is then able to output information into reality, or at least in the user’s visual field. Through the use of HMDs (Head Mounted Displays), MR-technology can generate holograms that understand the physical world around them and become responsive to the changes in the environment or objects’ movements. Moreover, interaction can happen using technology such as haptic feedback that can allow a user in an MR experience to touch and feel the heat of a virtual object.
The applications of Mixed Reality continue to evolve as technology continues to advance and computing power becomes ever so more ubiquitous. More than that, these advances are of extreme importance from allowing scientists to examine the Martian surface from data collected by rovers to ultimately freeing us from the confines of screens and keyboards that are so prevalent in the way we live our lives today. Literature over vastly different fields proclaim that we are at the cusp of a technological revolution and with technologies such as Mixed Reality becoming ever more real, one can not only understand why, but how.