December 19, 2011

GLSL depth of field with bokeh v2.4

here is a new update on the depth of field shader.
changes:
• again totally redone the depth of field calculation with 2 possible options:

- Physically accurate DoF simulation calculated from "focalDepth" ,"focalLength", "f-stop" and "CoC" parameters.
- Manual - artist controlled DoF simulation calculated only from "focalDepth" and individual controls for near and far blur planes.

• added "circe of confusion" (CoC) parameter in mm to accurately simulate DoF with different camera sensor or film sizes.
• optical lens vignetting factor based on f-stop value and user defined properties.
• cleaned up the code
• some optimization

Shader is HERE
Blend is HERE

controls:
Mouse+WASD - look/movement
LMB - drag the focal point object
1 - enables DoF rendering
spacebar - toggles debug view
Up/Down arrowkeys - zoom in/out (changes focal length)
Left/Right arrowkeys - changes f-stop value

22 comments:

  1. digitalspineDecember 19, 2011 at 12:07 PM

    Hi Martinsh..

    First of all. Amazing work! Love it! Thanks for sharing.

    Copied 2.4 shader to 2.3 .blend. 2.4 bokeh seems disabled while moving, bokeh reappeared during stop. 2.3 bokeh works nicely while moving and stop (ATI4650).

    Thanks.

    ReplyDelete
  2. martinshDecember 19, 2011 at 12:36 PM

    Hey, I have updated the blend also. Yeah the python file has also changed significantly.

    ReplyDelete
  3. digitalspineDecember 19, 2011 at 7:28 PM

    Now its working properly (ATI HD4650). Thanks man. :D

    ReplyDelete
  4. Leonard RitterJuly 9, 2012 at 3:17 AM

    Amazing work Martins, thank you very much for sharing!

    ReplyDelete
  5. UnknownApril 13, 2013 at 2:36 AM

    Hey Martin, just questions:

    focalDepth
    focalLength
    fstop

    what range of these values ? what are they equal to ?

    ReplyDelete
    Replies
    1. UnknownApril 13, 2013 at 3:20 AM

      i start game in blender and do not see any dof also

      Delete
    2. UnknownApril 13, 2013 at 4:00 AM

      and why *1000?

      and which pdf did you read to achieve this dof ?

      Delete
    3. martinshApril 13, 2013 at 4:56 AM

      hi Andrey,
      so it does not work when you press "1" in Blender, does it show anything in console?

      - focalDepth is the focal point or actually focal plane in meters from the camera.
      in other words it is the area where will be no blur. range can be from (0.0 - to infinite)

      - focalLength is sort of the optical power of the lens which changes the field of view (zoom of lens). In shader it will not zoom in or out but this value changes the depth of field (blur) significantly. The value is in milimeters and range depends on the lens you are trying to simulate. For example I have a lens on my camera of Focal Length ranging from 18mm to 200mm, basically this means that with 18mm I can make a wide angle photo but depth of field will be minimal, but 200mm will zoom in alot and depth of field will be very noticeable.

      fStop (or "f-number" or "relative aperture") is maybe the most important value that will affect the DoF. It sort of simulates the diameter of the aperture - entrance pupil. Smaller the diameter, bigger the fStop value - less depth of field.

      You can read these basic concepts of photography in wikipedia. And from there I have based my implementation.
      http://en.wikipedia.org/wiki/F-number
      http://en.wikipedia.org/wiki/Depth_of_focus
      https://en.wikipedia.org/wiki/Focal_length

      Delete
    4. UnknownApril 14, 2013 at 2:51 AM

      This comment has been removed by the author.

      Delete
    5. UnknownApril 14, 2013 at 3:50 AM

      This comment has been removed by the author.

      Delete
    6. UnknownApril 14, 2013 at 5:25 AM

      This comment has been removed by the author.

      Delete
    7. UnknownApril 14, 2013 at 7:16 AM

      thx

      1) do i calculate values right ? it seems ok.... like in blender

      /////////////////////////////////////////////////////////////////////

      camera_s *cam = world->current_cam;
      float zfar = cam->zfar;
      float znear = cam->znear;

      ///////////////////////////////// calc ////////////////////////////////

      // project mouseXY along Z axis
      vec3_t start,end;
      Screen2World3DRay(cam,
      //TSystem::Mouse[0],TSystem::Mouse[1],
      TSystem::tvars.v_width->fvalue/2,TSystem::tvars.v_height->fvalue/2,
      start,end,0x7FFFFFFF);

      // find intersection point with scene from mouseVector in world coords
      trees_collider trace;
      line_visible(start,end,trace);

      //DrawGlSphere(trace.end,12);

      // project to 2d
      float focalDepth = 0; // m - focus point depth (focal plane origin)... is ignored if 'autofocus' is on
      vec3_t screenPosition;

      if(render->Project(trace.end[0],trace.end[1],trace.end[2],
      cam->Rotation4x4GL,
      cam->Projection4x4GL,
      cam->Viewport4GL,&screenPosition[0],&screenPosition[1],&screenPosition[2]))
      {
      screenPosition[1] = TSystem::tvars.v_height->fvalue-screenPosition[1];
      AdjustTo640(&screenPosition[0],&screenPosition[1],0,0);

      //world->over.Quad(ga.data.materials.lord_inventory,screenPosition[0],screenPosition[1],32,32);

      // linearize depth
      focalDepth = -zfar * znear / (screenPosition[2] * (zfar - znear) - zfar);
      }
      //

      // it must change FOV but does not affect...
      static float focalLUserVal = 20;
      if(input->GetKeyState(ID_KEYBOARD, KEY_DELETE)== KeyPressed)
      focalLUserVal+=1;
      if(input->GetKeyState(ID_KEYBOARD, KEY_INSERT)== KeyPressed)
      focalLUserVal-=1;
      float focalLength = _clampf2(focalLUserVal,6,600)*1000; // mm - default is 20 (userval)

      //
      static float stopUserVal = 1; // default is 1 (userval)
      if(input->GetKeyState(ID_KEYBOARD, KEY_ADD)== KeyPressed)
      stopUserVal+=1;
      if(input->GetKeyState(ID_KEYBOARD, KEY_SUBTRACT)== KeyPressed)
      stopUserVal-=1;

      float stop = _clampf2(stopUserVal,-1,16);
      float fStop = powf(sqrtf(2.0),stop) * 1000; // mm - no phyton round here...

      /////////////////////////////////////////////////////////////////////

      shader.SetUniform1f(shader.m_paramsHash["focalDepth"],focalDepth);
      shader.SetUniform1f(shader.m_paramsHash["focalLength"],focalLength);
      shader.SetUniform1f(shader.m_paramsHash["fstop"],fStop);

      ///////////////////////////////////////////////////////////////////////

      if i need to make different dof variants (to blur near/ not to blur far ; to blur near only; to blur far ony) ..
      i need to tweak these values,i guess

      2)
      one more problem here:
      float penta(vec2 coords) //pentagonal shape
      this causes crash on geforce gtx580....

      3)
      vignetting:
      what is in gl_TexCoord[3].xy ? ))

      ..........................
      i just mainly try to adapt shader into my app,not blender

      Delete
    8. martinshApril 14, 2013 at 7:55 AM

      Hi, :)
      Okay I will answer the easier parts first:

      "if i need to make different dof variants (to blur near/ not to blur far ; to blur near only; to blur far ony) ..
      i need to tweak these values,i guess"

      For that case play with "manualdof":
      - first set "bool manualdof" to "true".

      then if you want near dof only:

      float ndofstart = 0.0; <-- near dof will start from focal plane
      float ndofdist = 2.0; <-- near dof will falloff for 2 meters

      float fdofstart = 999.0; <-- far dof will start somewhere in infinity..
      float fdofdist = 999.0; <-- far dof will extend even more distant

      - I have to make it more intuitive.. like adding "bool neardof" and "bool fardof" to enable/disable near/far dof completely. And I actually have to write a document of the shader and general use :)

      2) Well it was experimental anyway, so better to delete the pentagonal part completely. I can do that for you and make more simplified shader.

      3) gl_TexCoord[3] was the canvas coordinates in Blender, but now it is the same as gl_TexCoord[0], I have to fix that.

      Delete
    9. UnknownApril 14, 2013 at 8:02 AM

      i had not understood about manualDof also/.
      how to calculate those values

      Delete
    10. UnknownApril 14, 2013 at 10:23 AM

      ok cool )))) thx.... i played with shader and tried to optimize that, so check this out:

      https://dl.dropboxusercontent.com/u/5862637/dof/DoFbokeh.glsl

      i guess i made all that right...what do u think ?

      ...................... app code part ......................

      ////////////////////
      CVector3f DepthBlurOffsetKernel[9]; // xy - offset; z - kernel

      float dbsize = 1.25; // depthblursize
      CVector2f texel(1.0/(float)postRender2dMapRes,1.0/(float)postRender2dMapRes);
      CVector2f wh = texel * dbsize;

      DepthBlurOffsetKernel[0].SetXY(-wh[0],-wh[1]);
      DepthBlurOffsetKernel[1].SetXY( 0.0, -wh[1]);
      DepthBlurOffsetKernel[2].SetXY( wh[0], -wh[1]);

      DepthBlurOffsetKernel[3].SetXY(-wh[0], 0.0);
      DepthBlurOffsetKernel[4].SetXY( 0.0, 0.0);
      DepthBlurOffsetKernel[5].SetXY( wh[0], 0.0);

      DepthBlurOffsetKernel[6].SetXY(-wh[0], wh[1]);
      DepthBlurOffsetKernel[7].SetXY( 0.0, wh[1]);
      DepthBlurOffsetKernel[8].SetXY( wh[0], wh[1]);

      DepthBlurOffsetKernel[0][2] = 1.0/16.0; DepthBlurOffsetKernel[1][2] = 2.0/16.0; DepthBlurOffsetKernel[2][2] = 1.0/16.0;
      DepthBlurOffsetKernel[3][2] = 2.0/16.0; DepthBlurOffsetKernel[4][2] = 4.0/16.0; DepthBlurOffsetKernel[5][2] = 2.0/16.0;
      DepthBlurOffsetKernel[6][2] = 1.0/16.0; DepthBlurOffsetKernel[7][2] = 2.0/16.0; DepthBlurOffsetKernel[8][2] = 1.0/16.0;

      for(int i=0;i<9;i++)
      shader.SetUniform3fv(shader.m_paramsHash["DepthBlurOffsetKernel"]+i,DepthBlurOffsetKernel[i]);
      ////////////////////////////

      Delete
    11. martinshApril 14, 2013 at 2:45 PM

      yup that last code seems to be correct, but it is not vital for DoF, it just blurs depth map a little.

      here is my simplified GLSL shader for DoF:
      https://dl.dropboxusercontent.com/u/11542084/dof_bokeh_2.4.simple

      also in the python file "focus.py" there is a lot of stuff that is confusing and unnecessary :P

      So more about the DoF values you need to get:

      you only need 3 - focalDepth, focalLength and fStop

      - As I understand for the "focalDepth" you will read a pixel from rendered depth map, right?
      That is how I do the autofocus in shader - I just get middle pixel of the screen and focus on it.


      - "focalLength" should also change together with your camera lens.
      In Blender it is simple - camera.lens = focalLength
      I guess in your implementation you need to specify FOV right? Or change the frustum matrix?
      anyway here is the math to get FOV in degrees from focalLength:

      x = 35.0; //35mm camera film/sensor
      FOV = (2 * atan(x / (2 * focalLength)) * 180 / PI);


      fStop - it ranges from 0.7 to 256.0
      if you don`t need physically correct steps, just use any value in range of it.

      btw. there are many much more simple depth of field shaders out there :D
      This one I have made is physically correct though.

      Delete
    12. UnknownApril 14, 2013 at 11:13 PM

      physically correct - it is big cool thing)

      i thought,that it is not correct to allow postfx to change camera matrix, i will think over fov changing, but thanks. probably i will left fov untouched (but term will be added)

      thanks, for feedback - i will play with that.. you are a guru
      )

      Delete
    13. UnknownApril 15, 2013 at 4:32 AM

      hm....
      seems
      ndofstart
      ndofdist

      does not change anything.... manualdof is on

      Delete
  6. UnknownApril 14, 2013 at 5:24 AM

    This comment has been removed by the author.

    ReplyDelete
  7. relgukxilefJune 16, 2013 at 3:24 AM

    Hi there!
    Did you know you can improve the quality of the blur by mip-mapping the screen texture? I wonder if this is possible with a python script.
    It is possible when you have access to the OpenGL functions so maybe you can simply implement this into the Candy branch?

    ReplyDelete
  8. UnknownApril 13, 2016 at 3:59 PM

    Does not appear to be working on BGE 2.77

    ReplyDelete
  9. UnknownDecember 26, 2017 at 5:33 PM

    Hello,
    The shader and blend link are disappear.
    It is possible that updating it?
    Thank you very much.

    ReplyDelete