It`s been a while already since game studios try to replicate lens and film camera effects to enhance the visual fidelity for their games. Most of the time it does not make much sense to see them from the perspective of the player, but nonetheless it does look nice if done right.
I am talking about lens flares, vignetting and chromatic aberration, depth of field, bloom effects and film noise/grain. In photography or film they are considered as visual artifacts caused by imperfections and properties of film and lenses and mainly - the physics behind optics. Most of photo/cinemato-graphers do their best to avoid them by using better lenses and lens hoods to reduce vignetting, lens flares and chromatic aberration and use different sensitivity films for appropriate scenes to reduce image noisiness so in the end they get as clear image as possible that is usable for editing - tracking, compositing or whatever they do.. In the gaming industry it is pretty opposite - renderer outputs a perfectly clear image and artists do whatever they can to make it imperfect.
Few years ago these effects were done with some moving view-aligned textures, but now with clever shader tricks we can fairly accurate simulate them as an image post-process effects.
I have re-created most of these effects already and posted results in my blog. Links:
lens blur with bokeh v1
dof with bokeh v2.1
And here are very nice results from other guys
Lens Flare by Max Planck Institut Informatik
Lens Flare by John Chapman
DoF with bokeh by Epic Games
DoF with bokeh by Matt Pettineo (MJP)
..is the reason why I am writing this post here right now.
Film grain is a texture on the photographic film caused by an emulsion containing photon sensitive silver halide salt crystals. They are sort of pixels of the photographic film. Depending of the size of the crystals varies the resolution and sensitivity of the film. Bigger the particle (higher ISO number), higher the light sensitivity, but the image is less detailed. Unlike the digital image sensor where light sensing pixels are arranged in a regular grid, film crystals are jittered randomly over the film which gives an image more pleasing for the human eye. The possible reason for that might be the fact that the film grain does resemble the pattern how the photoreceptors are arranged in the retina of the human eye. Awesome, right?
photoreceptors in human eye (source http://www.sciencecodex.com):
an extreme case of color film grain (source Wikipedia.com):
Now in the digital era of cinematography most movies are shot digitally, that`s why often the film grain effect is artificially added on digital image. Many of the Blu-ray movies have a very distinct graininess, which actually gives a nice high-def cinematic feeling.
So exactly the same should apply to computer games.
Film Grain does seem to be the least difficult to simulate compared to other lens effects, but surprisingly I have not yet seen any convincing real-time implementation that does resemble, for example, the nice granularity look of 35mm film. Well the best examples of film grain effect I found used in games are by Crytek and Valve software - Crysis Warhead/Crysis2 and L4D series to be specific. Unfortunately the effect can hardly be called "film grain", it is rather an overlay noise that makes the appearance of slight jitter. What they did right is the elusiveness of the effect. Increase the grain amount a little bit more and it gets annoying. For many game titles film grain effect bothered me so much I disabled it completely (if there was an option at all). Mass Effect was one of them, in fact in darker scenes it looked nice, but in highlights made the image look dull and dirty. It did a good job covering up some less detailed textures and models though.
The possible reason why film grain does not seem to have advanced over the years like other effects might be that the grain filter is so subtle the developers do not bring much attention to it. And why waste time, money and precious milliseconds of computing time to create an effect most gamers would never notice. Well, here exactly lies the problem - overdone effects. Most notorious are Bloom, SSAO, DoF and yeah.. Film Grain and they are certainly very easy to overdo.
Personally I love subtle details. For me the best effects are the ones that enhances the visual quality while not bothering me with its presence during gameplay.
And there is another point - high ISO film photograph can get very grainy but the image looks nice and natural, while a bit overdone film grain effect in game ruins any viewing pleasure.. So the problem lies in the technique which generates the grain pattern and mixes it with the scene image.
The common methods of simulating it in games are either using a real grain texture or computing a noise pattern procedurally in runtime and then mixing them with the rendered image.
Pre-computed texture approach might make the best results as you can use actual grain texture taken on real film (filming against a grey background) and then overlaying this image to game scene. the downside is that it needs to be different in every frame. You can offset it a little and tile it, but a trained eye will always notice the tiling and this can make the effect very annoying. The approach does work best for still shots.
Procedural approach will ensure the grain will never tile, but the result is actually a noise and does not resemble the granularity of film grain at all. It does look more like a digital sensor noise. One might actually misinterpret it for an interference of the video signal to the monitor.
My take on Film Grain effect
All the time I have been using the procedural noise approach which is fast and gives nice results if the noise amount is very little, but as I mentioned it looks more like an interference in video signal.
After doing some research in this matter I collected bunch of reference images taken with film camera. I extracted the noise pattern from them and compared it to the noise shader I have been using. As you see it is lacking the graininess of the real thing.
I opened the image of my noise shader in Photoshop and tried to replicate the real grain texture. Blurring a little and adding 3 passes of sharpening did actually the trick. Unfortunately doing that for full-screen texture in shader would significantly slow the process and make it unusable in real-time.
So now I had to find a procedural or semi-procedural approach to simulate the same pattern in the shader without having to blur or rendering it in multiple passes.
Few years ago I did a vertex displacement mapping experiments with a procedural Simplex noise algorithm which created a nice grainy pattern. I resurrected the shader and applied it to a grey color and here is the result:
That is a really pleasant uniform grain. It is lacking a bit of the randomness but otherwise I call this a success!
Comparison shots of noise and grain applied to a gradient.
There is yet long way to go for more accurate results. It could be extended to include the real science behind it like varying graininess based on camera`s relative aperture. But I am only an artist so I go by whatever looks the best :)
This whole thing is a work in process, but I feel that even at this stage it looks already better than many other real-time film grain shaders. As you see the grain is not making the image look dull. It is less visible at bright areas of the image, but more noticeable in darker shades. An option to change the grain size is yet to be done.
Perlin noise grain applied in CryEngine (big images):
100% crops from the shots above:
today during further grain shader development I tried to replicate the grain in the image with the rally car. I had to de-noise the original image with resulted in some loss of detail. Then I applied the new grain shader on it. Here is the result:
de-noised and artificially added film grain shader:
real film grain compared to new film grain shader:
There are some significant changes from the original shader:
- user variable grain size
- added varying coordinate orientation for noise pass to eliminate any directional artifacts
- option to reduce grain based on luminance value
- added more tweakables for color noise
v1.0 (old one):
HERE you can find the GLSL shader file.
- noise algorithm I copied from HERE by toneburst, but original implementation comes from Stefan Gustavson found HERE
Wonderful! I will be diving into this!ReplyDelete
One thought. Should not the grain be animated?ReplyDelete
it is animated of course. It is hard to show that in screenshots :)Delete
Oh! I used it as a 2dfilter, maybe it is supposed to be applied to a plane in front of the camera?Delete
it works best as a 2D filter, yeah. For grain animation you have to add a timer property to the object that has the 2Dfilter actuator.Delete
Great as always. I'm watching the 2 images with the vehicle (the "natural" and the shaded one) and I observe that the grain have something to do with the dof. Grain seems to be sharper in focused area and blury when it's out of focus. Perhaps the best way to proceed is to apply noise and then blur it with dof in a second pass.Maybe its not dof but just depth. Other thing, it can be possible to mutiply the noise with the z-buffer cause the noise of the distant area seems to have less difference in luminosity. But it's a lot of post-processing. Nice to you to share your work and your ideas.ReplyDelete
thanks for the comment. Well the grain indeed is more noticeable in blurry areas than sharp ones, but I guess that is only an optical trick. I cannot think of a plausible way how in real life DoF would change the graininess - grain is basically a uniformly distributed particles on a film plane so they do not really change intensity based on depth.
This comment has been removed by a blog administrator.ReplyDelete
That is pretty awesome. Your shaders for the BGE are quite something. I'm trying to get the math behind it all but since I've just started digging into shaders it's rather insane. xDReplyDelete
If you have any tips for me I'd be delighted.
Btw even that de-noise effect has some uses, I really love the effect filters have on a fast paced game (Working on a rather fast one atm) but how far can you go with them before you'd have to start worrying about performance?
I guess what I mean, can I do particle-ish effects with shaders? That'd be awesome.
about particles - with shader you control how the particle will look like, not really the behaviors and movement of particle itself.
But I guess with some clever tricks you could make particle animation in vertex shader aswell.
Usually I start worrying about performance when I want to share it with others :) Then comes the cleaning and optimization.
Wonder ful blog very nice blog more infromation our sites Post processingReplyDelete
Hi, I am using this shader in my engine https://github.com/nem0/LumixEngineReplyDelete
Hi Mikulas, I am happy to see it being used :)Delete
You may be interested that I converted the 1.1 effect to ShaderToy. You can see the effect running here: https://www.shadertoy.com/view/Mly3RwReplyDelete
Feedback welcome :-)
I'm glad I found this web site, I couldn't find any knowledge on this matter prior to.Also operate a site and if you are ever interested in doing some visitor writing for me if possible feel free to let me know, im always look for people to check out my web site.ReplyDelete
Escaping the Prison Game
This is exactly the thing I was hoping to find, unfortunately the download links no longer work. If you or any random person reading this could send me the .glsl v1.1 shader file, that would be amazing! Mail is email@example.comReplyDelete
hi, you can find it here:Delete
but there are even better samples when you search for "grain" in shadertoy:
Ah, thank you!Delete
ProSlideshow Particular was professionally intended to work flawlessly within Polished product Genius X. As a generator and change impact, each ProSlideshow Secluded preset can be moved onto media and reviewed progressively. With the distributed parameters found in the FCPX Plugins investigator, clients can make facilitate alterations with only a couple of snaps of a mouse.ReplyDelete
Anyone can use image-editing software. Even the more complicated programs do not require the user to have computer programming knowledge. The best ones offer plenty of opportunities for photographers, graphic artists, and other creative types to express their ideas using images.image background removalReplyDelete
Kivalo Photography believes in love and the importance of photos to remember moments in time. Specializing in wedding photography Kivalo photographs weddings throughout Maine and all of the New England states. Kivalo Photography is largely photojournalistic but endeavors to capture not only the special wedding day moments but also the wedding details. wedding photographerReplyDelete