2017 NAB SHOW Special Edition Submission by Ian Spohr
The introduction of depth to a filmmakers’ toolkit is a powerful paradigm shift. It adds an additional layer to how media interacts with our sensory perception. With the interactivity and ability for audiences to compose their own perspective on a story, this is the pinnacle of so many things we love about modern storytelling.
Early on, it was tempting to view the emergence of VR as a gimmick. However, the lineage of motion pictures owes a small chunk of its heritage to cheap vaudevillian parlor tricks. After more than 100 years, motion pictures have grown to be the most widely consumed type of media on the planet. VR is the next progressive step in this lineage of visual media.
Looking back at art history, photography played a contributing factor to why painting moved into impressionism, and later surrealist periods, because it changed the dominant medium of portraits. Not only did the new medium adapt, painters felt freed to look past early formats and find new means of expression. Motion pictures had a similar effect on still photography; then 3D stereoscopic filmmaking introduced audiences to depth, and the feelings of inertia and movement.
The mind doesn’t have frame lines, and allowing audiences to choose where they look allows for additional sensory information. VR is the next adaptation in the current media landscape. Every new visual medium piggybacks on the one before it, and allows its predecessor to explore new territory in a harmonious way. This is where the story of VR starts.
The goal of any media project, whether a motion picture, commercial or VR, is to connect emotionally to the viewer, and achieve a response. So our goal is simple; make content that keeps viewers invested in the medium. Volumetric capture is the first chance for VR to step out and guide the viewer through a truly immersive experience.
Imagine if everything around you could be captured – a moment in time frozen and preserved in perpetuity. For example, you could stand at the 50-yard line of the Super Bowl next to your favorite quarterback as he throws the final pass to win the game or share heart-wrenching moments with actors from your favorite shows while feeling like you’re in the same room.
Sight, smell, touch and hearing all offer our brains sensory information based on depth. This is where the storytelling power of VR becomes apparent. The modern brain is craving media that can branch across more of its senses. Psychologists have been documenting this trend in modern populations for years to gauge effects on motor activity. The creative and technical applications to achieve these emotional experiences could be literally endless. Volumetric capture allows this.
So how do we get there? How do we let VR achieve its storytelling potential? With the advent of algorithmic stitching and light field capture we have the ability to gather information, along with images. Think metadata on steroids. These depth maps can create virtual stereoscopic camera arrays, and in addition, be translated into cloud points within a game engine such as Unity. We now have the chrominance, luminance and density of every point of light that is captured, but more importantly, we have the distance. We can now triangulate, much more accurately, any assigned pixel value from its nodal reference to the camera when captured or created. This means more than just better translation between the live action and computer generated information. This is the ability to merge them.
As the technology develops, these maps will increase in resolution and provide an unprecedented amount of digital information to craft an authentic experience. Using technologies like Lidar, Matterport or Depth Kit to map real spaces and blend them with imaginary ones, the possibilities are limitless. From simple paint work to Sci-Fi blockbusters, every part of our industry has the potential to benefit from these advancements.
Virtual Reality is an exciting platform and we look forward to seeing how far-reaching its applications can be. Already, through recent campaigns, our team has helped a paraplegic experience her dream of becoming a champion rower despite her physical limitations, brought viewers courtside with the Golden State Warriors, allowed military families to reunite and share experiences while countries apart, and gave viewers the thrill of being able to hit the track and go off-roading with Kawasaki professional riders.
The technical innovation required to develop and sustain this field is bringing together specialists from various areas, pulling their insights, and using this to inspire innovation in others. Our company, The Astronauts Guild, grew out of the need to provide technical solutions in stereoscopic filmmaking. Stereoscopic filmmaking brought together a peer group of smart, talented technicians who started jumping the hurdles that would send VR off and running. These strides increased productivity, reduced costs, and laid the groundwork for a race in this new medium.
To truly have longevity, we need to grow the VR industry, we’ll need fearless creative experimentation and we need to make lots and lots of mistakes. Most importantly, along the way, we need to make sure the content we create keeps audiences, and capital. This will determine the future of VR and its success.
That makes “now,” a very exciting time. We’ve already seen the marketplace quickly shift from monoscopic 360° video to stereo and on to volumetric capture. Next stop, a more compelling use of depth will improve the cinematic quality of VR. From our experience with cinematography and immersive technology we do not forsee VR replacing traditional filmmaking, but look forward to exploring more with varied audiences as we continue to pioneer and explore through VR.
Ian Spohr, Co-founder of The Astronauts Guild
The Astronauts Guild is a collective of professionals skilled in advanced technology and the development of immersive experiences through virtual reality, augmented reality, live streaming, 360 video and stereo 3D for narrative filmmaking and brands – for more information visit, www.theastronautsguild.com.