Sorry your browser is not supported!

You are using an outdated browser that does not support modern web technologies, in order to use this site please update to a new browser.

Browsers supported include Chrome, FireFox, Safari, Opera, Internet Explorer 10+ or Microsoft Edge.

Work in Progress / BlitzTerrain

Author
Message
kaedroho
16
Years of Service
User Offline
Joined: 21st Aug 2007
Location: Oxford,UK
Posted: 9th Oct 2009 01:13 Edited at: 9th Oct 2009 01:32
First render is here! [image attached]. This is the very first successful render of the new engine. I very quickly whipped up a simple render loop to render them. I got so excited, I took this image and posted it on here before I fixed any bugs.

The 12 in the top left is the poly count. As this is a completely separate system, DBPro no longer is aware of what I am drawing, except for a little box I decided to create.

No performance tests yet, I shall give you them in a few secconds. (Cant get GDK to get rid of the vsync).

[EDIT] the bug in the screenshot is fixed.

Alot of my work over the next few days would be getting this to render like DBPRo. and allow the user to have the control they had before in DBPro. Also a few random functions like get poly count and get draw count to allow people to still find polygon counts.

Attachments

Login to view attachments
kaedroho
16
Years of Service
User Offline
Joined: 21st Aug 2007
Location: Oxford,UK
Posted: 9th Oct 2009 03:17 Edited at: 9th Oct 2009 03:24
Some new features:

No pinholes (they somehow disappeared when i made the rendering engine)
Faster terrain rendering
3 render modes, Solid, wire frame and points

A few more things to come.

You will need to watch where you call the BT renderterrain command. If you render it after 2D commands the terrain will render on top.

I also can now fully confirm that blitzterrain exclusion also excludes vertices!

BMacZero
18
Years of Service
User Offline
Joined: 30th Dec 2005
Location: E:/ NA / USA
Posted: 9th Oct 2009 04:19
[/b]

kaedroho
16
Years of Service
User Offline
Joined: 21st Aug 2007
Location: Oxford,UK
Posted: 9th Oct 2009 13:11 Edited at: 9th Oct 2009 13:15
I can also confirm that the render speed is about 50% faster than before!

Also, loading speed has dropped down to 130ms instead of 270ms. That's less than half the loading time!!!!!

kaedroho
16
Years of Service
User Offline
Joined: 21st Aug 2007
Location: Oxford,UK
Posted: 9th Oct 2009 14:06 Edited at: 9th Oct 2009 14:07
Some of the most amazing results have just come through!

Just decided to use a 2048x2048 heightmap, with a 32x32 split.

Loading speed 8.3 secconds

FPS: 32 when rendering the full 8 million polys.


This test was done on a single nvidia geforce 9600 GS
loading speed does not include loading images. It is only the time since the make terrain function all the way to when it finnishes building.

revenant chaos
Valued Member
17
Years of Service
User Offline
Joined: 21st Mar 2007
Location: Robbinsdale, MN
Posted: 9th Oct 2009 14:25
Nice work kaedroho!
kaedroho
16
Years of Service
User Offline
Joined: 21st Aug 2007
Location: Oxford,UK
Posted: 9th Oct 2009 15:23 Edited at: 9th Oct 2009 15:23
Thanks

More testing. using a 256x256 terrain this time.

Run speed 400FPS (no frustum culling or LOD).

Locking, setting and unlocking every vertex and index on a terrain, every loop. 100FPS.

Regenerating mesh data from quadmap for every part of the terrain and locking, setting and unlocking every vertex every loop. 70FPS.

kaedroho
16
Years of Service
User Offline
Joined: 21st Aug 2007
Location: Oxford,UK
Posted: 9th Oct 2009 17:03 Edited at: 9th Oct 2009 17:42
Working on LOD, frustum culling and the render loop... well its not really a loop any more. The rendering will be done in a recursive function. This is so quadtrees run faster. LOD will also be done on the quadtree. So draw calls will be reduced too.

After that I will add loading and saving. As duffer has been waiting patiently for months for that feature.

kaedroho
16
Years of Service
User Offline
Joined: 21st Aug 2007
Location: Oxford,UK
Posted: 9th Oct 2009 18:49
BT InitialiseTerrain has now been removed. All this done is called exclude object off for every sector. It is no longer needed.

'BT EnableAutoRecovery enabled' has been added. If enabled Blitzterrain will detect when the device gets reset and automatically recover any lost terrains.

kaedroho
16
Years of Service
User Offline
Joined: 21st Aug 2007
Location: Oxford,UK
Posted: 10th Oct 2009 16:08
Quadtree frustum culling is almost done! The system is now rendering from quadtrees! Just got to make my frustum culling system detect intersection as well as in frustum and out of frustum checks.

kaedroho
16
Years of Service
User Offline
Joined: 21st Aug 2007
Location: Oxford,UK
Posted: 11th Oct 2009 13:59
The new max lod levels is:


Blitzterrain now generates all the LOD levels. I been spending hours clearing up all the bugs. I'm pretty confident that there are isnt any more in the LOD generation part.

kaedroho
16
Years of Service
User Offline
Joined: 21st Aug 2007
Location: Oxford,UK
Posted: 17th Oct 2009 14:21 Edited at: 17th Oct 2009 14:32
Ive been spending ages rewriting some parts of the rendering, also researching into ways of improving FPS.

I also discovered that when I added a texture, the pinholes came back. They never went, it was testing a light terrain on a light background so pinholes were extremely difficult to see if not impossible. I came up with a way to solve it. Just draw lines on the very edge of each sector, they go instantly. I haven't tested that idea, all I've done is render a wire frame on top of a solid terrain and it worked. If I can get the lines in the right place, then no one has to worry about pinholes any more.

I haven't looked into this very much, but I had an idea to make an object which Blitzterrain will copy render data from. You can set it shaders, textures, etc. And Blitzterrain will use its render data to render the terrain. The saves me having to recreate every Basic 3D command in DBPro, and allows people to treat the blitzterrain as a normal object. The only difference is that this object wont have any vertexdata so it may cause confusion if someone tries to change its vertexdata or use the object in physics.

kaedroho
16
Years of Service
User Offline
Joined: 21st Aug 2007
Location: Oxford,UK
Posted: 17th Oct 2009 17:15
More updates!

- Textures now work on Blitzterrain objects.
- Anisotropic filtering has been added into the rendering engine (and works)
- The blitzterrain main objects World matrix now gets multiplied into the terrains World matrix (for people who are unfamiliar with matrix math, this means that terrains can now be rotated, positioned and scaled with the basic 3D commands).
- BT set terrain wireframe removed. Use the basic 3D version instead

Zeus
17
Years of Service
User Offline
Joined: 8th Jul 2006
Location: Atop Mount Olympus
Posted: 17th Oct 2009 17:55
Great, this is awesome!

kaedroho
16
Years of Service
User Offline
Joined: 21st Aug 2007
Location: Oxford,UK
Posted: 17th Oct 2009 20:47
Thanks

I managed to put the anisotropic filtering into a dll. I have just released it to program announcements. So if you want to use anisotropic filtering in your games, get it here:

http://forum.thegamecreators.com/?m=forum_view&t=159383&b=5

kaedroho
16
Years of Service
User Offline
Joined: 21st Aug 2007
Location: Oxford,UK
Posted: 18th Oct 2009 02:09
I just made an 8 million poly terrain (using a 2048x2048 heightmap) rendered at 20 - 30 FPS without frustum culling or LOD.

It took 8 secconds to build but 1 seccond to generate. Loading and saving commands will cut out the build process. This has made me think about loading and saving commands. Loading and saving is next on the list so expect more on it shortly.

Plotinus
15
Years of Service
User Offline
Joined: 28th Mar 2009
Location:
Posted: 18th Oct 2009 11:28
This all sounds incredible. If you succeed in this it's going to be the best plugin one could get, at least for anything 3D.
kaedroho
16
Years of Service
User Offline
Joined: 21st Aug 2007
Location: Oxford,UK
Posted: 22nd Oct 2009 21:35 Edited at: 22nd Oct 2009 21:37
Thanks

I'm currently adding front to back rendering. This basically renders nearby sectors before far away sectors.

The DirectX rasteriser checks the Z buffer before rendering a point. If the Z buffer tells it that it can draw, it draws a point. The problem with this is that if a sector far away gets rendered before an object blocking it, it will render the object blocking it on top. Making it render 2 pixels to the same spot. But if you render the blocking sector first, DirectX automatically skips all the pixel rendering behind it. As if its just been culled.

KISTech
16
Years of Service
User Offline
Joined: 8th Feb 2008
Location: Aloha, Oregon
Posted: 27th Oct 2009 18:30
I came for my BlitzTerrain update fix, and there's no update.

How are things going?

kaedroho
16
Years of Service
User Offline
Joined: 21st Aug 2007
Location: Oxford,UK
Posted: 28th Oct 2009 00:01 Edited at: 28th Oct 2009 00:06
Sorry KISTech, Ill remember to update more often.

As its been a while, I have got quite alot done.

Frustum culling with Quadtrees is now working. This makes frustum culling a lot faster. LOD will be up and running soon.

Still working on front to back rendering. Might have this done soon too.

I have changed the blitzterrain colour heightmap format. I invented a new way which will allow normal heightmaps to be loaded with the same code.

Basically, this is the code which will load a colour heightmap.



So basically, the red channel does the main job. But the green and blue channels fine tune it. I've been working on a MOD for terrsculpt to import/export colour heightmaps.

I have also thought up a file format. My biggest worry about file formats is compatibility. As this is still a beta, the internal structures may change at any time. Which will be very annoying as I will have to convert my files from an old version.

I've looked into RAM use age too. it takes up 600 MB for a 2048x2048 terrain. (about 8.5 million polys). Which I think is too much. It renders every single one of these polys at 25 FPS. On my medium spec PC which is very fast. I might add a command to delete quadmaps. This will lower RAM useage but terrain editing will be very slow and the get ground height function will also be slow. Also, if the user alt tabbed out, the reload time will be similar to the load time. (which is quite a lot if not loading from a file).

KISTech
16
Years of Service
User Offline
Joined: 8th Feb 2008
Location: Aloha, Oregon
Posted: 28th Oct 2009 17:59
Quote: "Sorry KISTech, Ill remember to update more often."


It's all good.

kaedroho
16
Years of Service
User Offline
Joined: 21st Aug 2007
Location: Oxford,UK
Posted: 30th Oct 2009 15:37
I'm hoping to get LOD working today. If i get it working, then 1.06 will be released in november!

If not, then it depends when I get it working.

KISTech
16
Years of Service
User Offline
Joined: 8th Feb 2008
Location: Aloha, Oregon
Posted: 30th Oct 2009 18:45
WOOHOO!!!!

BMacZero
18
Years of Service
User Offline
Joined: 30th Dec 2005
Location: E:/ NA / USA
Posted: 31st Oct 2009 00:49
Seconded

kaedroho
16
Years of Service
User Offline
Joined: 21st Aug 2007
Location: Oxford,UK
Posted: 31st Oct 2009 02:21 Edited at: 31st Oct 2009 02:35
Didn't manage to get LOD working today .

But I did manage to half the RAM usage! I haven't checked it against 1.05 yet but I think its better. The speed is the exact same.

Heres how I did it.

Changed normals into 3 bytes instead of 3 floats, amazing accuracy is not needed, looks no different.
UVs are now calculated when the terrain planes are generated.
Mesh data is now deleted if optimisation is enabled.

kaedroho
16
Years of Service
User Offline
Joined: 21st Aug 2007
Location: Oxford,UK
Posted: 2nd Nov 2009 11:07
Added an optimisation in yesturday.

This optimisation is just reordering the vertices inside each mesh. Now all the vertices around the edge of the mesh are first in the vertex buffer and all the vertices in the middle are last. This allows me to lock only the edges when I use my LOD seam fixing code. I will also beable to lock individual edges too for more speed.

I did acctually work on a DBPro DLL which locks parts of the vertex buffer. Its a bit buggy though.

kaedroho
16
Years of Service
User Offline
Joined: 21st Aug 2007
Location: Oxford,UK
Posted: 6th Nov 2009 22:26
I'm still fixing bugs with this optimisation. I've been getting the old Square Triangles bug. Very annoying. I'm hoping to sort it out tonight and finish LOD over the weekend.

SpiderPig
14
Years of Service
User Offline
Joined: 11th Jul 2009
Location: Australia
Posted: 7th Nov 2009 23:55
Great stuff Kaedroho!
looking forward to the next release!

I've been thinking, (something I rarely do),
I'm planing to have a pretty big island, and i have a fence which runs through it for some distance.
At the start of the game if I position all the fences to match the ground height on a lod level of 1, when it changes to level 2, across hills the fences would either be half under the terrain or over it, right?

So would it be possible to make an extra map or something that decides which quads stay a certain lod level???

Like a black line in the middle of the terrain would stay lod level 1 not matter what....

or is this asking to much! lol

"I ask you,....what could possibly be in my eye that would explain this!?" - Jack O'Neill : Stargate SG-1
SpiderPig
14
Years of Service
User Offline
Joined: 11th Jul 2009
Location: Australia
Posted: 7th Nov 2009 23:57
oh and roads also would be good to stay level 1...I think.
otherwise looking through a scope to the far distance you would definatly be able to see a hill continue right through a road..

"I ask you,....what could possibly be in my eye that would explain this!?" - Jack O'Neill : Stargate SG-1
kaedroho
16
Years of Service
User Offline
Joined: 21st Aug 2007
Location: Oxford,UK
Posted: 8th Nov 2009 13:14
@ SpiderPig. Maybe later. I need to get the terrain going first.

@Everyone

I've just decided to rewrite the part of blitzterrain which generates QMaps for sectors. Its been getting buggy and slow recently. (takes 9 seconds to generate a 2048x2048 terrain). I hope to get this down to at the most, 7 seconds if not 5. This should take the whole day so no LOD this weekend unfortunately.

kaedroho
16
Years of Service
User Offline
Joined: 21st Aug 2007
Location: Oxford,UK
Posted: 8th Nov 2009 21:33
I've knocked a seccond off the loading time, Hopefully another few more secconds to be knocked off soon.

BMacZero
18
Years of Service
User Offline
Joined: 30th Dec 2005
Location: E:/ NA / USA
Posted: 8th Nov 2009 22:23
Sweet

SpiderPig
14
Years of Service
User Offline
Joined: 11th Jul 2009
Location: Australia
Posted: 9th Nov 2009 08:16
ok, just thought I'd put it out there
A function like that'd be really useful!

Nice job with the loading time decrease

"I ask you,....what could possibly be in my eye that would explain this!?" - Jack O'Neill : Stargate SG-1
kaedroho
16
Years of Service
User Offline
Joined: 21st Aug 2007
Location: Oxford,UK
Posted: 14th Nov 2009 19:08 Edited at: 14th Nov 2009 19:44
OH YES!!!!

Kaedroho->Mood=SUPERHAPPY!!!!!!!!

I HAVE NOW ADDED LOD BACK INTO BLITZTERRAIN!!!!

AND GUESS WHAT????


IT RUNS A 2048x2048 TERRAIN @ 220 FPS AND IT LOOKS ALMOST NO DIFFERENT TO THE ORIGINAL TERRAIN (which ran at 20 FPS).

WOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOO


edit.. sorry, the 220 FPS was it running in Debug mode..

THE ACTUAL TEST GIVES 260 FPS


[screenshot attached]


This means just 2 things left to do for release. LOD seam fix and Loading/Saving commands which Duffer has been waiting for about 8 months now.

I might have a start on loading/saving next. Its quite a big feature to add.

Attachments

Login to view attachments
BMacZero
18
Years of Service
User Offline
Joined: 30th Dec 2005
Location: E:/ NA / USA
Posted: 14th Nov 2009 19:22
kaedroho definitely is superhappy .

So am I!

kaedroho
16
Years of Service
User Offline
Joined: 21st Aug 2007
Location: Oxford,UK
Posted: 15th Nov 2009 00:46
Oh yes, Blitzterrain has now passed the 3000 lines mark!!

Plotinus
15
Years of Service
User Offline
Joined: 28th Mar 2009
Location:
Posted: 15th Nov 2009 10:08
Incredible! You are a coding god.

This is going to be a must-have plugin. Very much looking forward to seeing the finished thing.

(Are you still planning to do spherical terrains?)
Duffer
21
Years of Service
User Offline
Joined: 9th Feb 2003
Location: chair
Posted: 15th Nov 2009 10:56
@ Kaedroho,

Superb. Even at 1.06 this is going to be a top plugin for DBPro.

Spherical Terrains and more brush and paint commands would be wonderful - but the core stable plugin under that has go to be priority one. As ever, cant wait.

a long time dabbler with DBC and DBPro with no actual talent but lots of enthusiasm...
KISTech
16
Years of Service
User Offline
Joined: 8th Feb 2008
Location: Aloha, Oregon
Posted: 16th Nov 2009 17:12
Never ending seamless terrain.

That's what I'm after.

kaedroho
16
Years of Service
User Offline
Joined: 21st Aug 2007
Location: Oxford,UK
Posted: 16th Nov 2009 18:42
I'll make a "shift terrain" function after the first release.

This will shift the terrain the number of specified spaces and load in the new parts on the fly. You would have to add a few extra sectors which are in the fog in order to have a smooth effect. These would be culled anyway.

That's what I currently have in mind anyway... I WILL make the system keep the terrain at 0,0,0. No matter where it is located on the main heightmap. This just keeps floating points nice and accurate. It also helps you to know when you should delete or make objects.

KISTech
16
Years of Service
User Offline
Joined: 8th Feb 2008
Location: Aloha, Oregon
Posted: 17th Nov 2009 17:06
Can't wait. I've got big plans for that.

kaedroho
16
Years of Service
User Offline
Joined: 21st Aug 2007
Location: Oxford,UK
Posted: 30th Nov 2009 15:01
sorry about a massive lack of updates. I have been working pretty much non stop on getting blitzterrain very stable and fast. I managed to squeeze the RAM useage as far as it can possibly go.

So, there hasnt been much progress feature wise but I can now guarentee that the current features are as stable as can be!


I've now started work on making it produce DBPro objects for collision, etc. Using the following command:

BT MakePhyObject TerrainID, LODLevel, SectorID, ObjectID


After I will finnish off all the features which were in 1.05 and also add loading and saving.


And then... RELEASE!

BMacZero
18
Years of Service
User Offline
Joined: 30th Dec 2005
Location: E:/ NA / USA
Posted: 30th Nov 2009 15:38
Hey, hard work means progress. Keep it up!

N3wton
15
Years of Service
User Offline
Joined: 3rd Jun 2009
Location: Leeds, UK
Posted: 30th Nov 2009 20:41
Can
Not
Wait

Yours
N3wton

KISTech
16
Years of Service
User Offline
Joined: 8th Feb 2008
Location: Aloha, Oregon
Posted: 30th Nov 2009 20:51
Outstanding!!!!

Can't wait to see it.

kaedroho
16
Years of Service
User Offline
Joined: 21st Aug 2007
Location: Oxford,UK
Posted: 1st Dec 2009 13:19 Edited at: 1st Dec 2009 13:19
More updates!!

new commands going in tonight:

BT MakeSectorObject TerrainID,LODLevel,SectorID,ObjectID

For whatever reason you need to make an object for a sector, heres a command to do so.


BT GetSectorPosition(XYZ) TerrainID,LODLevel,SectorID

Gets sector position


BT GetSectorCount TerrainID,LODLevel

Gets the number of sectors on a LOD Level this will include the excluded sectors so it doesnt make your code very complicated.


BT GetSectorExcluded TerrainID,LODLevel,Sector

Returns true if the sector is excluded, false if not.


BT GetSector(Row/Collumn) TerrainID,LODLevel,Sector

Returns the row and collumn of a sector


BT GetSectorSize TerrainID,LODLevel

Returns the size of the sectors on the specified LOD level



there will be a few more if i think of any.

kaedroho
16
Years of Service
User Offline
Joined: 21st Aug 2007
Location: Oxford,UK
Posted: 3rd Dec 2009 10:17 Edited at: 3rd Dec 2009 10:28
Heres an example of how updating will be done. With multiple cameras and multiple terrains:



As BT has its own rendering engine, you can update Frustum culling for more than 1 camera.

If you want to update LOD for both cameras then do this:




Culling must be updated before LOD. This is because BT won't bother to update the LOD of sectors which are off screen.

kaedroho
16
Years of Service
User Offline
Joined: 21st Aug 2007
Location: Oxford,UK
Posted: 3rd Dec 2009 10:55 Edited at: 3rd Dec 2009 10:55
Just posted the new features of blitzterrain in first post. Here they are:

BlitzTerrain 1.06 NEW FEATURES!!

Blitzterrain has just been through a complete rewrite! 1.06 is the first version of BlitzTerrain to be released after the rewrite. Some of the new features are:


Brand New Rendering Engine

In the past, Blitzterrain used DBPros rendering engine. But this isnt very good for Terrains so I written a new DirectX Rendering engine from scratch. Its ALOT faster now, especially with high detail terrains.


QuadMapping

This is an algorithm I invented. It allows for quick terrain modification, get ground heights for terrains with quads of different sizes and rotations. It can completely regenerate a 256x256 terrain 70 times per seccond. Or edit every single vertex on a 256x256 terrain 100 times per seccond.


QSP (Quad Space Partitioning aka QuadTrees)

This is used for super fast frustum culling, LOD and front to back rendering. Front to back rendering renders all the detail closest to the player first, this is so DirectX doesnt render things on top of each other and improves Pixel Shader performance! This also causes lower detail sectors to be 4x as big, reducing draw calls and getting rid of most of the LOD "popping". Reducing draw calls is more speed enhancing than reducing the poly count.


Multi camera frustum culling

This is a feature that was added into Blitzterrain 1.05 but was very difficult to get working. It now works perfectly and is easy to add into your games.


60x Faster Loading speeds!

Yes, you have read that right, BlitzTerrain loads and generates a 256x256 terrain in 0.1 secconds. And can load and generate a 2048x2048 terrain in under 10 secconds. In the past it took 6 secconds to generate a 256x256 terrain and 10 minutes to generate a 2048x2048 terrain.


Easier to code with

All the parts of BlitzTerrain which were difficult to understand, are now alot easier! Such as the build loop has been made alot easier and doesnt even need to be used anymore if you don't want to use loading bars!


Dark GDK with OOP Support

Yes, I am being serrious about this. Blitzterrain now supports Dark GDK. Heres a code sample.




Loading and Saving

BT SaveTerrain TerrainID,File$
TerrainID=BT LoadTerrain(File$,ObjectID)

These 2 commands load and save terrains.

Save can be called anytime after BT BuildTerrain so you dont acctually have to generate the terrain after building it.
The Load command will load the terrain without generating it. You can then do a build loop to generate the terrain.

Plotinus
15
Years of Service
User Offline
Joined: 28th Mar 2009
Location:
Posted: 3rd Dec 2009 16:43
Great work, kaedroho. It sounds like this project is getting pretty close to completion. (That's what I'm hoping anyway!)
KISTech
16
Years of Service
User Offline
Joined: 8th Feb 2008
Location: Aloha, Oregon
Posted: 3rd Dec 2009 17:51 Edited at: 3rd Dec 2009 17:53
Outstanding stuff. You are a programming GOD!

Is 1.06 being released soon?

Login to post a reply

Server time is: 2024-07-03 05:58:22
Your offset time is: 2024-07-03 05:58:22