Techcrunch is covering a session that in their words, Facebook’s Engineering Wizardry Makes Even VR Videos Load Fast, is coverage of Scale @Video by Facebook Engineers about 360 degree video and virtual reality uploads. In the video below they describe the different variations that have been used to encode 360 degree video compared to virtual reality. It all has to do with your field of vision, video rendering, and current point of view.
These technical hurdles are magnified when we want to play 360 video in VR. The maximum visual resolution in the GearVR, for example, is 6K — and that’s just for one eye. A 6K stereo video at 60 fps is roughly 20x larger than a full HD video, with an average bit rate of 245 Mbps. However, most mobile hardware can play only 4K video.
This is just after our recent post Facebook Stadium Collaboration before Virtual Reality?. The amount of engineering might being put towards virtual reality is astounding. While many of us think of virtual reality in high level terms, there’s engineers working every day to bring this dream to, well, our reality.
By using pyramid encoding, which builds upon cube encoding, they furthered the goal of view dependent streaming:
Unlike in cube maps, where every face is treated equally, in pyramids only the viewport is rendered in full resolution. So when the viewer shifts perspective, instead of looking at a different face of the pyramid, he or she hops into a new one. In total, there are 30 viewports covering the sphere, separated by about 30 degrees.
Encoding video for virtual reality at the scale Facebook operates at is something we still try to wrap our heads around at TRS. Facebook using their Streaming Video Engine is creating new ways to consume real world based VR. With each of these releases we get more excited! Spoiler, this is technical, if you want the magic to remain do not watch.
Take a look at their post, Next-generation video encoding techniques for 360 video and VR if you are looking for additional in depth technical coverage.