The Science of Virtual Reality

Your Brain in a Virtual World
Virtual reality (VR) technologies play with our senses to transport us to any world that we can imagine. How do VR environments convince your brain to take you to these different places? How does your brain, in turn, react as you explore a virtual world?

Creating a Virtual Environment
Your brain builds on your past experience to develop “rules” by which to interpret the world. For example, the sky tells you which way is up. Shadows tell you where light is coming from. The relative size of things tells you which one is farther away. These rules help your brain operate more efficiently.

VR developers take these rules and try to provide the same information for your brain in the virtual world. In an effective virtual environment, moving objects should follow your expectations of the laws of physics. Shading and texture should allow you to determine depth and distance. Sometimes, when the virtual cues don’t quite match your brain’s expectations, you can feel disoriented or nauseated. Because the human brain is much more complex than even the most sophisticated computer, scientists are still trying to understand which cues are most important to prioritize in VR.

The Next Wave: Multisensory Virtual Reality
VR technology is also revealing new insights into how the brain works. When you navigate through space, your brain creates a mental map using an “inner GPS”—a discovery that was awarded the Nobel Prize in 2014. However, recent studies with rats in virtual environments show that their brains don’t create the same detailed map as in a real physical space. Visual processing is just a subset of the rich multisensory integration taking place in your brain all the time. As new VR technologies start to engage more of our senses, their effects may be even more compelling.

Applications for Health
Today, the interaction between VR and the brain has already led to applications in health and medicine, including treatment of post-traumatic stress disorder, surgical training, and physical therapy. Scientists are even exploring whether VR can change social attitudes by helping people see the world from a different person’s point of view.