After all the effort put into transforming the world from a spherical capture to a flat image, presenting your film requires that process to be reversed. The simplest player for VR-video is simply a way to convert from latlong encoded picture to a (limited) normal projection. Limited meaning that only a portion of the image will be seen at any one time, and which portion is being viewed is controlled by the audience, either through traditional input like a mouse or a touchscreen, or some integrated input method like the gyroscope in a mobile phone or head-mounted display. The process is essentially identical to those used in post-production, albeit that they now need to function reliably in real time.
A straightforward method to display a panoramic video is to map it to a sphere and have a playback camera in the center. The field of view of the camera becomes the view of your audience. Whether this mapping is literal, and done inside a 3D application like Unity or computational, through a mathematical formula, the results are the visually identical.
For players on a head mounted display, like the oculus, or cases for mobile phones like the Samsung Gear VR and Google Cardboard, the view needs to be duplicated for each eye. Head mounted displays offer the ability to show content in stereo, much like 3D cinema. Stereo productions are many times more complicated and are discussed in the Advanced topics chapter.
Technically, the virtual cameras for head mounted displays should be offset a small amount, to capture a little more of the world on their respective sides, as your eyes do away from your nose. In practice however, this fact is usually ignored in favour of simplicity. Many players designed for head mounted displays do add an amount of barrel distortion along with a heavy vignette around the view of each eye. The barrel distortion counteracting any distortion caused by the eye-lenses in the display, whilst the vignette softens the edges so the pictures more gradually fades away towards the periphery.
The audience needs to have some way of interacting with the player, and may be as simple as clicking and dragging inside the video to change the view. When used on a mobile phone, the phone’s orientation can be linked to the video and people can look around the video as though their phone were a window into another world.
A more sophisticated or memory efficient way of distribution may involve converting from a latlong to a cube map, thereby creating a more “computer-friendly” file, most players however, including youtube and facebook, still expect latlong files.
FRAMERATES – MORE IS MORE
There are two distinctly separate frame rates to consider when talking about virtual reality. The first being the frame rate of the video being played, the other is the rate at which the player updates its view of that video. It is important to discuss them separately and a good player will reflect this whereas a bad one will lock these two together.
While 24 frames per second may be sufficient for persistence of vision, if the player does not update it’s view at least twice as often, the video may appear to stutter, thereby undermining the immersive nature of the content. Moreover, a stuttering player on a head-mounted display can cause motion sickness. The jury is still out on which magic number the world settles on, but suffice it to say that it will be a rather large number. The Sony Morpheus for example, is slated to refresh at least 90 times per second.
At the moment, the limiting factor for the frame rate of the player is the display itself. Most modern displays have a maximum refresh rate, the speed at which a picture can be redrawn per second, of between 60 and 75Hz. Mobile phones too, have the same limit – generally 60 Hz and so even the best player cannot draw the scene more than 60 times per second. Fortunately, for all but the most extreme head-movements, this is adequate.
The creator of the experience may decide on the frame rate of the content, and I encourage experimentation in this regard. Some fast moving actions may not appear as smoothly in 360 compared to normal video, but this may very well create an interesting effect or be exploited to create a mood for the piece. Very high frame rate video may alleviate this problem, but under many circumstances this only adds to the processing overhead whilst doing little for bettering the experience.
Determining the View of your Audience
The focal length (field of view) of the camera becomes the view the audience can move around. For films that aim to recreate a “normal” view on the world, like a documentary for example, the lensing of the camera should be set close to the natural field of view of human vision, somewhere between 120 and 180 degrees. Though a 180-degree view is unattainable as a flat projection and will look wildly distorted.
The field of view of the human eye also stretches into the periphery, an area not present for a screen mounted flat in front of your eyes. For most viewers the field of view is similar to viewing the world through a rectangular hole cut from a piece of paper.