I've been using this method for a few years now, It just took a lot longer to figure out how to get DBP to work with it. I actually used something very similar when I designed the terrain technology for Wrath-Unleashed, which has just been released by Lucasarts on Xbox and PS2(plug), take a look you'll notice the similarity (and the same for my art style), but I do think this method is quite different and has far better possibilites and wider optimizations so Lucas can't sue me
****************************************
So here is perhaps a better description to how it works and how to build it:
Plase Note: Some of my methods listed below are workarounds (until I can get a .x exporter that exports RGBA verts without corruption), however doing it this way also helps to expose other creation methods.
1. I created a regular mesh in Max with premapped UV's (no texture) and shaped it into a terrain. The cool thing about doing this in Max is that I could also include overhangs, a proper edge to the world and even caves and retain the contiguous mapping. So forget the matrix method, that is too restrictive.
2.Once shaped, I optimized the mesh (retaining UV's) so I could keep all the details where I needed them and not waste any poly's where it was flatter. Bringing the overall mesh down from around 64k tris to 16k tris or there abouts
3. Then I painted vertex color striations and baked(blended)the lights and shadows on top.
4. Exported this as a .X
5. Loaded this into DBP as a mesh (this is important to retain the vert colors)
6. Created a memblock from the mesh
7. Converted this to an object and textured it with my base grass (Scottish of course)
8. Went back to max and reset ALL of the vert colors on my original mesh to white (very important)
9. Painted vert colors (this time just greyscale) which would be used to show where the next texture layer would blend and by how much. In this case I wanted to use rock - which is actually part of an ancient gravestone from my home town of Dundee - and basically airbrush where I wanted it to appear.
11. I then isolated all the verts that had remained untouched (the pure white ones).
12. Inverted the selection and grew it by 2 steps to make sure that the blend had a clean edge all around
13. Inverted the selection again and deleted the verts so I now had an exact copy of the original mesh but just the alpha parts for this layer. (This normally appears as a cluster of fragmanted parts) It came out at around 4k in tris (Do not move or optimize any of these verts - it has to hug the original landscape perfectly for the vertex color compare process AND the to blend)
14. Exported this layer as .X
*** This is where it gets a little funky
Now that I had a layer (without alpha, just vert colot) I needed to bring this into the scene and do 3 things to it (a)Convert the grey colored verts to alpha and (b) ADD the original vert colors from the primary mesh (the main grassy land mass) and append this mesh with RGBA values that would match the primary layers vert colors exactly -(if you leave this part out the alpha layer will look very odd sitting on a landscape that has vert colors, especially in shadow). (c) Export the layer so it can be reloaded later without the processing stage.
The tricky thing now was that the alpha mesh had far fewer verts and faces than the original mesh so all the vert indices would be in a different order so a compare was needed.
15. So I then loaded the alpha layer into a memblock
16. Compared EVERY vert in the primary mesh against each vert index in the alpha layer and when I got an exact xyz match I grabbed the primary vert color, inverted the grey scale vert value of the alpha vert and combined this with the vert color to create a 32bit color value that I then used to replace the original grey scale value of the vert
17. Saved out this new mesh layer from the memblock to a .x file
18. Loaded it back in, offset it on the y axis by 0.1 units and hey presto! I now had my very first alpha layer working in DBP and it blended perfectly and has the added advantage of also reacting to dynamic lights - you would never know it was there.
19. repeated this several times for the other layers (rough grass, dirt etc.)
20. chopped up the layers a little for better culling
22. Added collision detection on just the primary land mass
21. ran around and took some screenshots.
Oh BTW I've started a forum section on my website
http://www.leadartist.com/forum/ so if you want to chat about any of this over there or anything else for that matter then please feel free. I'll be checking my own pages more regularly than trawling through here (this place is getting so busy nowadays - its cool but I ususally only have the time to drop in at the weekends)
I've also spoken to Andy over at Panda soft and he has sent me a few beta's of his .x max exporter that supports alpha verts, but it's not working that well yet for Max 5, all the vert alphas come out in the wrong order (the Max 6 version is a different build so that may work, I'm not sure), but when it does work perfectly then this process will be a lot easier (although isolating alpha verts in max for a selection is still not possible so I would still paint the vert colors WITH alpha and use the vert colors for seperating the terrain parts).
But in our world, we all know there are is more than one way to skin a cat, boil a fish, kick a goat.
Well I hope this helped. I'll have a demo for you all soon enough, but in the meantime, give this a shot and if you need a hand then let me know.
Asylum Hunter.