The VR of the 1990s was a certified gimmick ; technology wasn’t there for the most part and a simple system then by today’s standards was worth millions of dollars. Fast forward to 2011 when an inventive young man “only 18 years old then” Palmer Luckey built the first prototype of a VR headset in his parents garage. By the 6th generation prototype he named it the Rift and had a Kickstarter project to be able to launch to the gadget gamer community “raising US $2.4 million or 974% of its original target”.
John Carmack the genius programmer of ID software fame had an Oculus prototype headset and was blown away by it; he modded it to better performance and was able to run a Doom 3 edition on the headset in 2012. Carmack is now the Chief technology officer CFO at Oculus VR. The interest in VR intensified and blew up right after the adoption of the technology by some of the best minds in the gaming industry. Mark Zuckerberg was so into it and the possibilities it could bring to the future of social media that Facebook acquired Oculus VR for 2 Billion USD in March 2014. Talk about a successful meteoric rise for such a young company!
John Carmack Discusses application and future development on the Oculus Rift / Gear VR Headsets:
The Final consumer version of the Oculus VR after a DK1 and DK2 developer prototypes will ship in Q1 2016 for the masses which will need an Oculus certified computer to be able to work properly. The specs of such a computer are very high end but will be less so as technology progresses. There is also the Samsung – Oculus Gear VR headset which supports the Note 4, Galaxy S6 and now in November 2015 the S6, S6 Edge, Note 5 and S6 Edge + phones. It gives an entry VR experience for the masses with a completely portable component. Competitors like Valve and Google Cardboard are getting products out as-well which is intensifying the competition in the space which brings us to the most important part…
VR Content:
There has been an explosion in VR content as of late from panoramas to full 360 degree 4k Footage, cinema experiences, Netflix app and a few cool games; some have been great and others are just a boring demo but there is no turning back. Once you have tested VR when it works right and it gives a sense of presence you will never feel the same about entertainment content on a flat screen.
One thing that has been clear is that in order to avoid motion sickness and image judder when moving your view/head you need a high refresh rate. Oculus feels that 90fps is the ideal step forward to approach a realistic experience and avoid the problems with tricking the brain into an alternate reality.
This is where high speed frame rates in video come in. While some Oculus demos run at 60fps it is far short of the 90+ fps needed for the VR sweet-spot. Added to this is the need for ever higher resolution footage in order to fill the ever increasing pixel counts needed for the VR headset screens. Early VR prototypes used 720p or less resolution screens for both eyes but today the Gear VR with a Note 5 for example uses a 1440×2560 QHD Amoled Display. This is 4 times the pixel count of 720p or 2x the resolution. Even with this it is apparent that the screen-door effect is still a real problem when trying to suspend disbelief and offer a true VR experience.
A QHD display still has to split the screen resolution into each eye lowering resolution even further to something more like 1000px/eye which is far from ideal. Even if we increase the resolution to a 4k UHD display which will happen as soon as Christmas in large smartphones; the resolution has to be diced for two eyes again giving something just slightly above 1200px per eye.
So for video to truly be immersive in VR it will need to be 4k or above ideally 8k at 90+fps. The 4k VR footage when recorded in two or more cameras will be good enough for most people as the resolution will be less of a problem but 1080p footage on each eye is not going to cut it. The bandwidth goes up considerably as-well as now you need a dual or quad 4k stream playing back at 90fps with no lag on a portable device. It will take considerable time for hardware to support this but the content will have to be recorded like this regardless. In order to feed what will be an explosion of VR consumption the amount of footage on offer will have to grow exponentially.
Cameras like the RED Epic already can shoot 120fps at 4k-5k and cheaper options like the fps1000 4k when shipped will support up to 240fps at 4k. It will probably be standard to record at 96fps or the like in 4k for VR in the near future so you will have to double your camera investment to record in 3D for each eye. And if you want a full 360 degree experience you will need even more cameras and expensive rigs. So high frame rates will need to go up in tandem with resolutions for VR production and these will usually not be for showing slow motion in itself but for capturing content real time at a high enough refresh rate.
Unreal Engine 4 VR and Slow Motion Test Use VR Headset:
https://www.youtube.com/watch?v=HpGpSg7MWaY
But what about Regular Slowmo?
You could still play back high speed frame rates at 24p, 30p 48p or 60p and get away with a pretty good effect if not much movement happens in the users VR headset/visor. However in order to improve the VR experience we need 90fps+ to get the fluidity when the vision field moves. So you will have to place 90fps as the base playback frame instead of 24p or 30p rate for slow motion which will make shooting higher frame rates a complete problem for slow motion cameras. If you shoot 4k at 240fps it would only be 2.66x slowed down when played back. You will need 900fps to have a 10x slowdown in 4k that has no judder when moving about. So basically you will need to nearly quadruple 3.75x your frame rate to get the same slowmo result as 24p playback. Suddenly things start to become problematic in storage and camera specs land. Things will have to improve speedily in higher frame rates in order to support it.
Simulation: VR headset with 4K per eye (4K) by eVRydayVR:
HSC: The video above compares a 1080p Oculus Headset Witha hypothetical 4k per eye headset of the future at 75fps. It is clear that VR needs a very high resolution for it to become the next big thing in entertainment. Thanks to eVRydayVR for providing such excellent content on VR!
But what about all those 30p and 60p VR footage samples?
It is of course possible to watch any video even 24p on the VR headsets with pretty good results but this is because the experience is not immersive enough. In order to feel like you are some place else you will need higher frame rates at very high resolutions. However low frame rate footage even at 4k will continue to be the VR standard for a while until specs in storage, capture and delivery catch up.
So now what?
One thing is for sure, VR is here now and it will only get better, this is no 3D TV gimmickry, this is the real deal; go to a Best Buy and test out the Gear VR today or the Oculus Rift, Valve’s Vive when available. It is nothing but astonishing what can be done with current technology which will only get better.
As far as VR footage it really needs to be recorded in higher resolutions 4k and up and same goes for slow motion. It is just a matter of a couple of years for Phone screens to be 4k in the high end as standard with Sony already there with the Xperia Z5 Premium. It is the logical progression if Phone VR is to evolve further. Now headsets will also have to keep evolving and optimizing the pixels by customizing screens for VR only. This could concentrate more and more pixels in smaller panels that are positioned for each eye much like the Oculus Consumer version is going to release but with much more resolution in the future.
The field of VR video is growing and as video professionals you have to stay ahead of the curve, shooting in 3D will no longer be an avoidance as it will be a necessity for VR and it will bring a whole new plethora of competition. Should we budget for dual cameras now? It seems we may have to to produce this type of content.
4K walk in the souk of Bethlehem 360 degree VR video by Wide Web:
Camera manufacturers could also prop things up by offering dual stereoscopic 4k recording in a single camera in the near future but that is best left for hardware makers to figure out as 360 degree is the best use of this tech not 180. The widespread need for higher frame rate 4k cameras is here to stay thanks to VR as that will be needed for normal use at least from 75 to 90fps. Thanks for reading-HSC!
Some thoughts on the future of VR by Michael Abrash below, highly recommended to watch!
Oculus Connect 2 Keynote with Programming Genius Michael Abrash: