Monday, April 22, 2013

OpenGl Transition Effects with the Stencil Buffer

Swapping between objects or even entire 3D scenes is something that graphics programmers will always have to deal with.  Deciding exactly how to accomplish such a task leaves room for innovation from not only a visual standpoint but a technical one as well.  Screen fades, wipes, and dissolve effects are all common forms of transitions used in graphical programming.  For this project I chose to recreate a dissolve effect and wipe transition in C++ OpenGL using the Stencil Buffer to control it.
The Stencil Buffer is an interesting tool that most modern graphics hardware takes advantage of, it opens the door to a wide variety of effects like the ones I chose to create as well as much more. The Stencil buffer acts very similarly like an actual stencil in real world would.  OpenGL allows us to directly interact with the Stencil Buffer manipulating how each pixel show within the effect area will be affected by it.  What this means for us is we can literally draw out a patter to be displayed through the Stencil Buffer. But how do we accomplish such a task? It is not as hard as you might think, but there are definitely a few things to make sure are turned on before you start to battle with the Stencil Buffer.

By default the Stencil Buffer is not being used so in order to start manipulating it need to enable it, by simply calling glEnable(GL_STENCIL_TEST);  now that we are using the Stencil Buffer we need to make sure to clear it out each frame otherwise it will overwrite itself.  By calling glClear(GL_STENCIL_BUFFER_BIT); we also want to turn off depth and color mask otherwise the Stencil buffer will interfere with those values when and in this case we don’t want that to happen so similarly disable them by passing in GL_FALSE to both. 

Now this is where the fun starts, We want to set up the pattern we want to be seen through the stencil buffer, so if we were doing a transition the first frame would basically be empty, but as the transition progressed more and more of the screens design would be drawn until the other object is fully visible and the first one is gone.  We do this by setting the stencil buffer to always fail everywhere we draw a pixel.  We then draw whatever pattern we want displayed through the stencil buffer.  An example would be in a shooter, when a bullet it’s a wall they don’t replace the entire texture on the building or even the model with one that has a bullet hole, no they use the stencil buffer and choose a few pixels on the wall to reveal the bullet hole texture through it.  We can also use this to draw interesting shapes and patters like spirals or zig zags.  Once we are satisfied with the pattern we drawn either by pixel using something like glDrawPixels() or possibly a whole object (to make a star shape or something drawing as you would with any other model) we then turn off writing to the stencil buffer.  We now set the stencil buffer to pass wherever it did not previously fail as well as turning back on our depth and color masks so our display objects are drawn correctly.  We now draw our first object, the one we want to be displayed everywhere the stencil pattern isn’t.  Once that image has been drawn we set the stencil buffer to only draw where the it previously failed meaning the second object will only appear where the stencil pattern was drawn.

 The image below is a perfect example of what we are trying to do.  The 1’s are where the stencil buffer was set to fail and the blank spaces is where it passed.  When we draw the objects over one another we can see based on the value different parts of the red and green rectangles are revealed. 

Congratulations you are now using the stencil buffer from here, each frame you can change the stencil buffer pattern to allow more and more of the second object to be seen or hidden based on what you are trying to accomplish. Those are the basics of how to use the stencil buffer to switch between, see through, and draw over objects in specific locations.  When creating my two effects (wipe and dissolve) I used the same basic technique but handled drawing to my stencil buffer differently.


For the wipe effect I picked a starting location in this specific example I used bottom left and slowly drew a larger and larger square expanding outwards until the stencil pattern completely covered the screen revealing the second object underneath.  I could then reverse the process to re reveal the first object.
The dissolve effect was a little bit more difficult, I had to gain access to different parts of the screen and randomly draw pixels into the stencil buffer thus causing random bits of the object to change until finally the entire screen was covered in stencil buffer pixels and the second object is revealed.  This gives the impression that the object is dissolving into the second object as you saw chunks of it slowly switch over until the new object is visible. 

Overall this process allowed me to gain a much better understanding of how the stencil buffer is used and what types of effects can be accomplished with them like I mentioned above.  It allows us to represent other objects, color, or patterns without changing textures or swapping out whole models and makes for great for screen transitions.  The Stencil Buffer is a powerful tool which I will definitely be using more of, and I hope you will too.

1.       Advanced Graphics Programming Using OpenGL By Tom McRynolds, David Blyth (Pages 196-199)

Used for basic tutorial of how to use a Stencil Buffer.


How to use glDrawPixels.


How to use glRasterPos


More Stencil Buffer How To.