Gamasutra: The Art & Business of Making Gamesspacer
Do we need 4K resolution for immersive VR?
Printer-Friendly VersionPrinter-Friendly Version
arrowPress Releases
April 18, 2014
PR Newswire
View All
View All     Submit Event





If you enjoy reading this site, you might also want to check out these UBM TechWeb sites:


 
Do we need 4K resolution for immersive VR?
by Ivan Blaustein on 10/29/13 03:51:00 pm   Featured Blogs

The following blog post, unless otherwise noted, was written by a member of Gamasutra’s community.
The thoughts and opinions expressed are those of the writer and not Gamasutra or its parent company.

 

Oculus HD and Beyond

Shortly after shipping 10,000 developer kits, the folks at Oculus announced that they had made an HD Prototype of the Rift that has a 5″ screen and a resolution of 1920 x 1080. Oculus set up a booth at the IndieCade conference this month to let patrons try out the HD model on the winners of the VR Jam. This was my first opportunity to test out the prototype and in both of my experiences with it (playing Virtual Internet Hacker and Dumpy), I did not notice the decreased screen size (from 7 to 5in), but definitely noticed the increased number of pixels. The edges of objects were still rough and I would not call the image crisp, but without a visible grid of pixels, there was no screen door effect that you get with the developer kits. 

Overall, the HD prototypes provided a more pleasant VR experience than the developer kits because of the increased pixel density and decreased size, but they were still nowhere close to the quality necessary for a truly immersive experience. They need a significantly higher resolution in a screen so close to your face... and they know it. They recently announced that they are working on a version of the Rift with a 4K (3840 x 2160) resolution display, but will that be enough?

Pixels Per Inch

PPI - Pixels Per Inch

As screens get bigger, the pixels that make up the image get bigger. With the same resolution, the pixels that make up a 50″ TV’s image will be much larger than that of a 32″ TV’s image. So when Oculus showcased their 5″ prototype with a higher resolution than the 7″ model, the pixels are dramatically smaller, and the pixelation effect is far less apparent. The standard measure of pixel density is pixels per inch (PPI).

The folks at ifixit took apart a developer kit to see what it was made of and found that it was using a “Innolux HJ070IA-02D 7″ LCD.” This 7″ display has a resolution of 1280 x 800 and provides 215 PPI. The Oculus splits the screen in half so each eye is only seeing 640 x 800. The HD prototype has a 5″ screen with a resolution of 1920 x 1080 (960 x 1080 per eye) for a PPI of 440. So in just a year, they have more than doubled the PPI of the device by decreasing the screen size and increasing the resolution. It was still not nearly enough to remove the pixelation effect, but it shows they are moving in the right direction.

One big advantage that the virtual reality peripheral manufacturers have is being able to share R&D efforts with smartphone and tablet manufacturers. Everybody is trying to put more pixels into smaller packages. Right now, there is a lot of talk about the resolution of handheld display resolutions and the marginal benefit of “retina displays”. Some argue that we do not need to keep increasing the resolution of these devices because the human eye cannot distinguish additional pixels at the distance that most people use their cell phones (about 12-18in), while others welcome any increase in resolution. Either way, all of the major manufacturers are pushing for more pixel density, and that is great news for fans of virtual reality.

8K Per Eye

Unlike large television sets many feet away, with a screen only inches from your face, any additional pixels make a dramatic difference to the quality of the image and the immersive effect. To answer the title of this post, Do We Need 4K Resolution For Immersive Virtual Reality? No. For a truly immersive virtual reality experiencewe need much more than 4K resolution!

Palmer Luckey, the founder of Oculus, made a comment that he thinks the pixelation effect will not go away until the resolution reaches beyond 8K (7680 x 4320) per eye. From an Arstechina article:

“To get to the point where you can’t see pixels, I think some of the speculation is you need about 8K per eye in our current field of view [for the Rift],” he said. “And to get to the point where you couldn’t see any more improvements, you’d need several times that. It sounds ridiculous, but HDTVs have been out there for maybe a decade in the consumer space, and now we’re having phones and tablets that are past the resolution of those TVs. So if you go 10 years from now, 8K in a [head-mounted display] does not seem ridiculous at all.“

I agree. With the current rate of progression for screen resolution (with 4K coming to tablets and 8K TVs on the immediate horizon) the idea of a very small 8K screen is not far away. For a more detailed look at the current status of the Oculus screen resolution, there is a post on the oculus developer forum that goes very deep into the current and optimal resolution for perfect virtual reality with the Oculus. While some of his analysis is difficult to follow, the author uses Field of Vision (FOV) and Angular Pixel Density (APD) measurements to conclude that:

“If we want the “perfect” Oculus Rift, with 180 degrees FOV and an APD of 45, we’ll need a screen with a resolution of 8000 x 4500″

This matches up very well with Luckey’s comment of 8K per eye. Using a PPI calculator, a 7″ screen with resolution of 8000 x 4500 would require 1311 PPI. The highest PPI of any display developed so far is 651 PPI on a 2″ screen by a company in Japan. So if you are looking for a truly immersive virtual reality experience, give it a few years.

For more information on gaming in 4K Ultra HD and beyond, check out my blog at 4KGamer.com.


Related Jobs

Mapzen
Mapzen — New York (or San Francisco), New York, United States
[04.18.14]

Mobile Graphics Engineer
The Workshop
The Workshop — Marina del Rey, California, United States
[04.18.14]

Sr. Programmer (Generalist)
The Workshop
The Workshop — Marina del Rey, California, United States
[04.18.14]

Senior Graphics Programmer
Machine Zone
Machine Zone — Palo Alto, California, United States
[04.18.14]

Senior Software Engineer (Server-Side)






Comments


Katy Smith
profile image
How are processors and memory holding up to the display increases? I will admit to being out of the loop on hardware improvements for a little while now, but I remember that for every doubling of a screen dimension, you need 4 times the memory. This was a pain in the butt when the iPod touch 4 came out :) how much memory and processing power would be needed to display Oculus games in 4k?

Catalin Zima Zegreanu
profile image
Very good point, Katy.

A minimum setup of a framebuffer for the final image (4 bytes per pixel) + a depth buffer (4 bytes per pixel) + an HDR rendertarget (16 bytes per pixel) (you'll want this for proper colors lighting and effects) leads to ~190MB for 4K and a whooping 760MB for 8K. Double that for two eyes, and this is just for the final image composition, ignoring all textures, vertex buffers and whatnot.

But as long as you have the Occulus connected to a PC/laptop this should be ok, since these are more easily upgradeable. Much less of an issue that iPod quadrupling the resolution but leaving the CPU the same...

Daniel Boy
profile image
Concerning GPU power:
At NV Montreal Tim Sweeney talked about 4K gaming: It makes sense for him with a gpu that hits around 20 TFLOPS. Taking his advice 8K per eye equals 160 TFLOPS. A GTX 780 is capable of ~4 TFLOPS. The PowerVR 543MP3 in the iPhone 5S has a theoretical peak of ~75 GFLOPS (0.075 TFLOPS).
Doubling the performance every other year (hasn't happened in years, closer to 25% per year) the desktop would reach 160 TFLOPS in around 11 years (let's say 15) and doubling the performance every year (5 -> 5S was almost 3x) mobiles would reach parity to the 780 in 6 years.

Vincent Hyne
profile image
It's highly likely we will move over to graphene in the next ten years, as we will have just about exhausted what we can do with silicone by 2019, and once that happens, 8000x4500 won't be that hard to achieve.

MR FRENCH
profile image
Much as I'd like to see this happen, the engineering cost of getting to 1080p displays only 4-5inch diagonal has been met by the mobile companies' pursuit of "Retina" resolution. I would suggest that the mobile industry's needs are now met, with little for them to gain from further resolution increases (and a big battery life penalty too!).

My guess is that the 1311ppi objective would require very significant development investment- and apart from gaming VR- I'm not sure there's another market driver to pay for this yet?

Going to LCOS might be easier- and have lower NRE costs- but that's likely to be a big piece of wafer for each eye so it's not going to be cheap to buy either.

So I'd suggest that the article's suggestion that, for a truly immersive experience, we need to "give it a few years" might prove to be optimistic.

Jennis Kartens
profile image
Great article. Thanks!



I personally wish for better ordinary displays first... at least higher PPI, but also reduction in input lag and all the problems we face currently.

Ben Lewis-Evans
profile image
I am not sure that really high def screens are needed for immersion for a large proportion of the population. There are plenty of low fidelity simulators used in research and such that are not graphically stunning but are very immersive. I have personally seen participants throw their hands up and scream in anticipation of a crash in a low fidelity driving sim, and that isn't even with a HMD (which is generally more immersive just by their nature).

Latency is probably more important for HMD's simply because of its implications for gameplay and simulation sickness (resolution quality doesn't seem to impact much here, although it does help reduce eye strain and resulting headaches).

On the other hand graphics are very pretty.

TC Weidner
profile image
Sure it would be nice, and that might be a nice yard stick to shoot for, but as far as "needing" it to be immersive? I dont think so. Hell we all have been immersed in games with a lot less tech than this. On 800x600 2 d screens, and worse. The mind is a fascinating and powerful tool for gaming as well, grab its focus and graphics become secondary. As the old text adventures proved many years ago.

Daneel Filimonov
profile image
That's a good point, but the difference between 800x600 resolution immersion and HMD immersion is very much existent (though quite subtle). When you're wearing a HMD, you expect to "feel" as if you're inside the experience. In other words; the immersion is deeper, psychologically because the visual border is (hopefully, if done right) non-existent and your sense of awareness and perception is altered. While with, perhaps, a 800x600 (or any resolution of the past decade) display you're not so much immersed since you can clearly see the outlines of your monitor and what ever is behind it (well, depending on your monitor size I guess :P). The same sort of immersion isn't there. Yes, you may feel as if you're the character or part of the game, but you can look away if something scary happens or turn off the monitor. With HMD it's a little different.

Carter Gabriel
profile image
Immersion is Immersion.

It doesn't really matter if it's in a book, playing a retro game, or in full virtual reality.

HMD would probably just allow people to more EASILY get immersed. Not everyone can become immersed in video games, or even books.

Of course, I don't think even VR can match the immersion of a book. It's far away, if even possible. We do not know what it will be like, until we get there. Crazy psychological effects could happen, like disconnecting from reality to the point of VR being a drug, Total Recall or BTL chips (see: Shadowrun), or the opposite effect of our brains always realizing it's fake somehow and we realize it's impossible to beat the full immersion of a well written book. If you don't believe me that it might be the opposite of our brains failing to realize we're in VR, look at dreaming. People have both vivid dreams where they claim "I did not know I was dreaming." and lucid dreaming where they claim "I am fully aware I am dreaming."

We won't know until we get there, but immersion is immersion. There is a reason people always say "The book was better than the movie." and why no video game even comes close to the immersion of a good book.

The scariest change is when VR becomes so realistic, you can't tell the difference between game graphics and real life graphics. Unless our brains protect us, there could be some SERIOUS social consequences to too high a quality of immersion.

Daneel Filimonov
profile image
If I may, I shall dissect this:



"Immersion is Immersion.

It doesn't really matter if it's in a book, playing a retro game, or in full virtual reality."

Immersion comes in different forms, to compensate for one thing or another. It does indeed matter what media you talk about. You can essentially say that reality itself is a form of immersion because you can become - say - visually immersed with reality, compensating for other senses like smell or hearing which are not as pronounced as in, for example, a dog. And so, the dog is "immersed" equally in the opposite direction because it doesn't have a visual spectrum as refined as ours but can smell much more.

The same can be said for media such as books or games. Games immerse you visually (VR further immerses you subconsciously and on a visual level as well because in part, you're tricked into thinking you are actually part of the experience you are participating; theoretically if done right, that is), while a book would immerse you thoughtfully as you read the words off the page and imagine its descriptions in your own interpretation.

"Of course, I don't think even VR can match the immersion of a book. It's far away, if even possible. We do not know what it will be like, until we get there. Crazy psychological effects could happen, like disconnecting from reality to the point of VR being a drug, Total Recall or BTL chips (see: Shadowrun), or the opposite effect of our brains always realizing it's fake somehow and we realize it's impossible to beat the full immersion of a well written book."

You seem to be confused by what VR is capable of and what books are capable of.

While a book can immerse you in what ever story it has going for it, it does not have the elements intrinsic to most modern (and even some arcade and retro) games that help sell their immersion much more deeply than a book ever could. So to say "VR can['t] match the immersion of a book" is to intentionally ignore the fact that the past 20 or so years of games have done exactly that, some have even surpassed such immersion (though, this is more subjective).

Also, VR has been a reality for far longer than when the Oculus had been conceived. The only difference is that the Oculus is the first successfully marketed and financed version of VR, partially because todays' technology allowed it and due to proper funding. As for "crazy psychological effects", Our brain is wired in such a way that despite what happens in front of us visually (and even audibly), our other senses are also subconsciously taking into consideration our environment. If the senses do not match up, then we automatically assume it's fake. That's it. No 'waking up not remembering who you are, on a vacation on Mars' sot of thing. This isn't science fiction! Even books can't go that deep.

Immersion or not, we will always know what is reality and what is not (not taking into consideration altered state of mind) due to perception of our senses and ultimately hindsight (if you look back at an event, you can tell whether it was real or not).

"If you don't believe me that it might be the opposite of our brains failing to realize we're in VR, look at dreaming. People have both vivid dreams where they claim "I did not know I was dreaming." and lucid dreaming where they claim "I am fully aware I am dreaming."

Vivid dreams and lucid dreams do not fundamentally fall under immersion in the same vain as media such as books or games. This is because such dreams are conceived within the brain, and are abstractions of our perception of our reality. Again, hindsight takes care of this, as you have even proved in your own reply "I did not know I was _dreaming_" which implies that afterwards they did in fact know. Then again, you cannot always rely on word-of-mouth to be fact. And as well, the whole dream bit is irrelevant in context.

"We won't know until we get there, but immersion is immersion. There is a reason people always say "The book was better than the movie." and why no video game even comes close to the immersion of a good book."

We have been there, and we are in fact there now. The technical difference between VR now and VR 15 years in the future is resolution. Considering anything of the future is arbitrary because technology of the future will all be matched (better visuals = better audio = better input, etc.) due to the standards being met and how things are moving along as we see them today.

Finally, the reason people say "the book is better than the movie" is not because of immersion, that is totally missing the point. The reason people say this is because the movie is based on one person (or a groups') interpretation of the book. Because the book itself doesn't have any of these elements, the reader is left with their imagination to figure some things out. In summation, it's a false argument to say that a book can and is more immersive than games, whether using VR or not. It's simply just not true given the reality of the situation.

*wipes brow*


none
 
Comment: