If you are creating a 3D game which gameplay requires the player to know the exact position of the character, then you have to think about what to do when the model is behind other objects.
You can either test whether the 3D object is occluded by other objects and then make the objects where this is the case transparent.
Or, you can save yourself the trouble of performing a raycast and use another trick which uses mostly GPU functionality to render an outline for concealed objects. Many games, like Torchlight 2, use this method.
I wanted to implement the same mechanic for Rubberband Racing in Away3D, since the new track I’ve implemented has a lot of objects that can appear in front of the car.
Utilizing The Depth Buffer
The idea depends heavily on the z-buffer, or depth buffer, that can be found on all modern graphics cards.
All pixels rendered by the GPU not only have a color value, which is visible onscreen, but additionally get a depth value assigned.
If a 3D object is drawn behind another, already drawn object, a test is performed for each pixel to see whether it is occluded or not. If a pixel about to be drawn has a bigger depth value than the one already present, then it is further away from the camera and will be discarded.
This test, however, can be switched around, so that only pixels that are behind already rendered geometry will be accepted.
To better understand the effect, I looked at the rendering process of Torchlight 2.
For this, I used the software PIX for Windows. This program lets you inspect the API calls of games that use Direct3D to render their content.
PIX was available in the now discontinued DirectX SDK. The software has recently been integrated into Visual Studio, but since I don’t use that IDE, I’m still using the old stand-alone version of PIX from the SDK.
To implement the silhouette effect, you have to basically perform three steps:
- Render the silhouette
You have to switch the z-buffer comparision mode to accepting pixels which are behind the already rendered geometry.
In Direct3D 9, you would do this by setting the D3DRS_ZFUNC render state to D3DCMP_GREATER.
Also, for this step the GPU should only render color values into the backbuffer. No values should be saved in the depth buffer. This prevents render glitches later on.
This would be done by setting the D3DRS_ZWRITEENABLE render state to 0.
Once you have set up these steps, all you have to do is render the geometry of the character with any shader you see fit.
- Render the character
Switch back to the default GPU settings. The graphics card should only render pixels which are in front of the geometry and the depth buffer should be written to.
Then, you simply render the character a second time with the normal textures and shaders.
This way, you don’t have to worry about raycasts or similar things and still can show your player exactly where the character is.
Implementing Silhouettes for Concealed Objects in Away3D
Set Z-Buffer Comparision Mode
The most important, and luckily very straightforward step is to set the comparision mode of the depth buffer.
You do this by setting the Context3DCompareMode at the respective places.
Away3D implements a scene graph and doesn’t give you the option to manually draw a certain object a second time. That means you have to create a second 3D object that will be responsible for drawing the silhouette.
This object receives the same geometry and has to copy the position and rotation from the original for each frame.
This second object receives another material, on which you set the depthCompareMode property to Context3DCompareMode.GREATER.
Set Up The Correct Rendering Order
Rendering the objects in the scene in the correct order, as described above, is very important for this technique.
Away3D currently doesn’t give you any direct influence over the render order of the scene, unfortunately. Neither can you explicitly assign objects an order, nor are they drawn in the order in which they are added to the scene.
To ensure the correct render order, you have to set up separate views in Away3D. The concept behind this is described in this tutorial about how to set up Away3D and Starling.
The difference is that we don’t set up an Away3D view and one (or sevaral) Starling views, but two Away3D views.
To have two separate Away3D views run at the same time, you need to create a Stage3DProxy first, that both will share.
_stage3DManager = Stage3DManager.getInstance(stage);
_stage3DProxy = _stage3DManager.getFreeStage3DProxy();
Once the CONTEXT3D_CREATED event has been thrown, you can set up the View3D objects themselves. It’s important to pass them the Stage3DProxy object and to set the shareContext property to true.
_view3d = new View3D();
_view3d.stage3DProxy = _stage3DProxy;
_view3d.shareContext = true;
_view3d2 = new View3D();
_view3d2.stage3DProxy = _stage3DProxy;
_view3d2.shareContext = true;
Now you have the control over the order in which the View3D objects in your application are rendered. The render code for each frame would then look like this:
You can now add the static level geometry to the Scene3D object in _view3d and the objects responsible for the character and the silhouette in _view3d2 to ensure that the level is rendered first.
We now have set up the correct z-buffer compare mode and the correct render order. However, if we render our scene now, it will probably look something like this:
Problem: Stage3D Always Writes Into the Depth Buffer
As you can see, on the left side, where only the silhouette should be visible, there are other lines and shapes which are caused by rendering the car with the normal textures. This is because the silhoutte is writing values in the depth buffer as well, messing with drawing the actual car.
If we were using Direct3D, we would prevent the silhouette to put any values into the z-buffer at this point. The problem is, Flash doesn’t let us do that.
You can use Context3D‘s configureBackBuffer method, but only to enable or discard the depth buffer entirely. This doesn’t help due to the impact on performance and the fact that the depth values of the rest of the scene would be lost as well.
So we could either just settle for the effect we have now (which is kind of ugly) or we can find a workaround.
The render glitch is caused by the z-values of the silhouette and the actual car influencing each other.
The way I solved this problem is by flattening the geometry of the silhouette each frame in order to create a billboard I could put in front of the actual car geometry.
In order to do this, we need to scale the geometry down along the view axis of the camera. This requires
- Measuring the angle between the y-axis and the view axis of the camera.
- Rotating the object by that value around a rotation axis which is perpendicular to the y-axis and the view axis away from the camera.
- Scale the object along the y-axis to a very small value.
- Rotate the object back
This can be done with the respective matrix multiplications.
Granted, this method isn’t appropiate for all situations, but it serves me well in Rubberband Racing.