Quote: "You can sort of only render shadows in view by putting the shadow camera in front of the main camera, then moving it in the direction of the light source"
That sounds reasonable, however wouldn't that call for an orthographic camera as well? Otherwise the shadows cast would be huge / all-encompassing if the camera managed to end up just behind some object. Even if in theory the main camera is also in shadow in that case, light would realistically seep through to light up areas further in front of it if the light source was "infinitely" far away.
I just can't figure out how to position these ortho cameras in 3d space; essentially you can set how many units across and down will be seen by it and that's it. I'm trying out the more advanced D3DXMatrixOrthoOffCenterLH function at the moment, which sort of seems to allow movement along the X and Y axis but it is irradic at best... I don't suppose you happen to know anything about how to manage that?
Anyway, thanks for the suggestion; I will keep tinkering with it to see if I can come up with anything
@Green Gandalf: I was not aware of the border attribute for samplers. I've been doing this for now:
float shadowFactor = IN.lightPos.x < -1.0f || IN.lightPos.x > 1.0f ||
IN.lightPos.y < -1.0f || IN.lightPos.y > 1.0f ||
IN.lightPos.z < 0.0f || IN.lightPos.z > 1.0f;
But that sounds like it could be both more readable and efficient; I'll switch my custom check for that, thanks.
Quote: "Is it possible to use the depth map directly when rendering the shadow camera? I saw a "D32" image format and you can use the ": DEPTH" semantic rather than ": COLOR" in the pixel shader."
I tried that and no, it didn't work (actually it crashed the engine).
Using the D3DFMT_R32F format and only writing to the red channel works swell however.
"Why do programmers get Halloween and Christmas mixed up?" Because Oct(31) = Dec(25)