Sorry your browser is not supported!

You are using an outdated browser that does not support modern web technologies, in order to use this site please update to a new browser.

Browsers supported include Chrome, FireFox, Safari, Opera, Internet Explorer 10+ or Microsoft Edge.

AppGameKit Classic Chat / Mesh memblocks and LOD

Author
Message
Plotinus
15
Years of Service
User Offline
Joined: 28th Mar 2009
Location:
Posted: 21st Sep 2018 10:54
Link
I'm wondering what the best way is to create terrain LOD of a ROAM nature.

I haven't used mesh memblocks before but they look like the best way to manipulate vertices of existing objects. The question is: is it possible to insert *new* vertices into mesh memblocks, and then apply them to existing objects? The idea is to split up polygons in terrain as you get closer to it.

If you can't do it this way, is there a way it can be done in AGK...? I can't seem to find any discussion of this here, but perhaps I'm looking in the wrong places...
Back to top
Profile PM Website
Bengismo
6
Years of Service
User Offline
Joined: 20th Nov 2017
Location: Yorkshire, England
Posted: 21st Sep 2018 11:28 Edited at: 21st Sep 2018 11:30
Link
Quote: "is it possible to insert *new* vertices into mesh memblocks"


Yes, you can insert new triangles and manipulate the positions and normals.

However, this is not a quick operation to do (especially in interpreted basic)! So doing this every frame on even a small terrain would hit the frame rate very badly.

A better option is to have any large terrain split into chunks. Each chunk then can have 2-5 Levels of detail. The chunks closest to the camera are displayed at the highest LOD and those further away at a lower level.

As AppGameKit doesnt support LOD built in, you have to manually choose which LOD to display for each chunk based on a distance equation from the camera. (also, to avoid 'popping' of LOD levels..it helps to animate the terrain model between LOD levels...thats not easy to set up but is doable in AppGameKit by setting up the mesh with s aingle bone factor so that it very smoothly morphs to the next level of detail)

So It is possible to do this, its not particuarly hard but you are best pre-processsing your terrains to multiple LOD levels rather than trying to do it realtime in AppGameKit as you simply wont be able to do it (a real time technique like roam) unless its teir 2 and quite well optimised.
Back to top
Profile PM
Plotinus
15
Years of Service
User Offline
Joined: 28th Mar 2009
Location:
Posted: 21st Sep 2018 11:37
Link
I thought that might be the case. It's a shame as I would like to do this procedurally, rather than pre-processing the LOD levels, but I suppose we have to accept the limits of what we're working with!
Back to top
Profile PM Website
Santman
12
Years of Service
User Offline
Joined: 15th Sep 2011
Location: Inverness
Posted: 21st Sep 2018 11:46 Edited at: 21st Sep 2018 11:47
Link
As a side, I spent ages building a terrain creator and one of the things I added in was exactly what Bengisimo said, chunks that had varying levels of detail.

The results were, surprisingly, that it made little to no difference. I'm not 100% sure why, but let's look at the single biggest slow down in any 3d scene: redraw. So imagine looking out over your landscape and all the hills etc. Now imagine one big hill right at the front, blocking 80% of the back ones. As there's no occlusion culling, every single hill is drawn from the back forwards, then drawn over again and again until it reaches the front.

Now......if a mesh is way in the distance, it won't have a huge number of pixels to be drawn, so the real factor is how many vertices are processed....and I suspect that's where you may get a small gain. A bigger gain cones from using smaller textures....although mip mapping is built into AppGameKit, if you load a 2048*2058 texture for the landscape, then a 256*256 texture, the latter will render much, much faster as there is far fewer texel lookups and internal smoothing calculations needed. So dealing out distant chunks with smaller textures seems to work better - but actually load smaller textures, don't use the resizing option.

Just my two pence worth.
Back to top
Profile PM
Bengismo
6
Years of Service
User Offline
Joined: 20th Nov 2017
Location: Yorkshire, England
Posted: 21st Sep 2018 12:10 Edited at: 21st Sep 2018 12:13
Link
There are 2-3 major reasons why your 3D scene may not run faster and fill rate and pixel redraw are way more impacting then the shear number of vertices processed in most cases. Especially if your running at some very high resolution! (Ahem) The GPU is spending all its time working out pixel values rather than transforming vertices so LOD on terrains wouldnt help too much in that particular case.

Best thing to do is try force the drawing of the nearer objects first, (at least the nearest 3 chunks of terrain) that way the pixel is already written in the depth buffer close to the camera before the terrain further away attempts to draw. That way only the vertices are processed and most of the pixel values are never calculated are as the depth test fails and so they never get calculated on the terrain further away.

Quote: "I thought that might be the case. It's a shame as I would like to do this procedurally, rather than pre-processing the LOD levels, but I suppose we have to accept the limits of what we're working with!"


I would say:
Dont use the heightmaps built in by creating a mesh from an image....The meshes produced are not optimised in anyway and contain a massive number of excess triangles especially over the flat areas on your terrain which add no detail. Plus the 8 bit resolution is poor. Make a terrain and save it as an object. Loads of excess triangles can then be removed out making the drawing so much quicker! Model your terrain like you would a character...outside of AppGameKit and optimise it.

I used the method of calculating the terrain height value in the vertex shader from looking into in image....(this is possible to do in AppGameKit but its not how AGK's heightmaps work). That way you can select the mipmap level in the shader to get a s lower LOD version and then replace the mesh and so LOD level transitions are smooth. The negative effect of this is that the collision model or raycasting is more complicated then simply using AGK's version
Back to top
Profile PM
fubarpk
Retired Moderator
19
Years of Service
User Offline
Joined: 11th Jan 2005
Playing: AGK is my friend
Posted: 21st Sep 2018 12:57
Link
Modifying a mesh in real time with memblocks
fubar
Back to top
Profile PM Email Website
janbo
15
Years of Service
User Offline
Joined: 10th Nov 2008
Location: Germany
Posted: 21st Sep 2018 16:24 Edited at: 21st Sep 2018 16:26
Link
The order of drawing an object doesn't matter for opaque objects as long as depth testing is enabled, you wont gain speed unless you remove the object from the render queue in the first place.
But optimizing the mesh, using atlas textures and replace higher poly meshes with flat textured plains or 2D sprites in the 3D scene(billboards) will gain some performance as long as you can keep the overhead below the gain ofc.
I don't think you necessarily need to utilize memblocks for what you want.
Back to top
Profile PM Email Website
Bengismo
6
Years of Service
User Offline
Joined: 20th Nov 2017
Location: Yorkshire, England
Posted: 21st Sep 2018 16:41 Edited at: 21st Sep 2018 16:43
Link
Quote: "The order of drawing an object doesn't matter for opaque objects as long as depth testing is enabled, you wont gain speed unless you remove the object from the render queue in the first place."


Sorry but thats rubbish Janbo. A quick look on google for 3d overdraw will give plenty of articles on this. We arent using deffered rendering in AppGameKit so rear pixels are calculated.

If you have already drawn the near pixels and written to the depth buffer then the pixel colours on the far terrain dont even get calculated. Failing the depth test means the pixel colours of the far terrain dont even get calculated meaning a saving on the GPU. The depth test fails and so the pixel shader for those pixels isnt run at all meaning it doesnt bother looking up the texture. Draw order does affect speed and can reduce overdraw as santman says. It doesnt affect the final image but it does affect the speed. My testing of the FPS proves this too.

Quote: "I don't think you necessarily need to utilize memblocks for what you want."

This is absolutely correct...

I dont even touch memblocks for terrain editing (edit an image directly) and can keep >100fps unlike the example above which struggles while editing the landscape to keep over 20fps due to the memblock editing
Back to top
Profile PM
Santman
12
Years of Service
User Offline
Joined: 15th Sep 2011
Location: Inverness
Posted: 21st Sep 2018 17:55 Edited at: 21st Sep 2018 17:57
Link
Bengisimo, how would you force rendering close objects first, or do you mean similar to occlusion culling?

Janbo, I disagree here too....we're referring to one large mesh, so it is all drawn. Even in chunk via chunk versions, back chunks sare still drawn even when completely hidden. Ive got your shader lack and ypir own landscape clearly suffers from it too. Or have you got occlusion culling working?

That's why I abandoned my world builder, AppGameKit at it's core is just too slow for complex scenes. Without instancing working, imposters, and control over the draw queue I struggled to stay above 30 fps once I started filling the world.
Back to top
Profile PM
janbo
15
Years of Service
User Offline
Joined: 10th Nov 2008
Location: Germany
Posted: 21st Sep 2018 18:07 Edited at: 21st Sep 2018 18:28
Link
Quote: "google for 3d overdraw will give plenty of articles on this"

I guess the overdraw articles are related to transparent objects where the render order is important.
Quote: "in AppGameKit so rear pixels are calculated"

I didn't say they wont ? Actually I said/mean the opposite

Quote: "The depth test fails and so the pixel shader for those pixels isnt run at all"

This only applies if you have "early depth testing", this is true if AppGameKit would use OpenGL 4.2

@Santman: I focused on the looks and technique for the terrain "demo", I did nothing in terms of occlusion culling, not yet
also, are you disagreeing to me ?
Quote: "we're referring to one large mesh, so it is all drawn"

You are right, else you would need a geometry shader

Quote: "how would you force rendering close objects first"

Quick test forcing an object to render first

not sure if this is a dumb test
Back to top
Profile PM Email Website
Bengismo
6
Years of Service
User Offline
Joined: 20th Nov 2017
Location: Yorkshire, England
Posted: 21st Sep 2018 18:30 Edited at: 21st Sep 2018 18:32
Link
janbo wrote: "I guess the overdraw articles are related to transparent objects where the render order is important."

No it applies to opaque objects as you need to calculate rear pixel colours through transparent objects but dont when they are opaque


janbo wrote: "This only applies if you have "early depth testing", this is true if AppGameKit would use OpenGL 4.2"


Again, incorrect...you can manually control it in open gl 4.2 but it exists as an optimisation since the dawn of open gl and direct x.
https://www.khronos.org/opengl/wiki/Early_Fragment_Test

From open gl's own documentation above...optimization in about every single driver for open gl is that it doesnt need to calculate a pixel value if it is behind an opaque one
Optimization
Early fragment tests, as an optimization, exist to prevent unnecessary executions of the Fragment Shader. If a fragment will be discarded based on the Depth Test (due perhaps to being behind other geometry), it saves performance to avoid executing the fragment shader.

A specification only defines apparent behavior, so the implementation is only required to behave "as if" it happened afterwards.Therefore, an implementation is free to apply early fragment tests if the Fragment Shader being used does not do anything that would impact the results of those tests.


Most drivers dont bother calculating pixel colours unless they have to.

So....dont change the depth of the fragment and the driver can decide if your fragment is ever run or not.
Back to top
Profile PM
janbo
15
Years of Service
User Offline
Joined: 10th Nov 2008
Location: Germany
Posted: 21st Sep 2018 18:39 Edited at: 21st Sep 2018 19:04
Link
Quote: "you need to calculate rear pixel colours through transparent objects but dont when they are opaque"

I'm really sorry but I'm not sure where the argument is... you disable depth testing for transparent objects https://www.appgamekit.com/documentation/Reference/3D/SetObjectTransparency.htm
Quote: "it exists as an optimisation since the dawn of open gl"

How would I do that(early depth testing) in opengl 2.0 ?

Referring to:
Quote: "that way the pixel is already written in the depth buffer close to the camera before the terrain further away attempts to draw. That way only the vertices are processed and most of the pixel values are never calculated are as the depth test fails and so they never get calculated on the terrain further away"

Here for example it states: "The Depth Test is a per-sample processing operation performed after the Fragment Shader"
https://www.khronos.org/opengl/wiki/Depth_Test
They are drawn every time, you gain nothing as it calculates the vertex shader and fragment shader and after that you write to the depth buffer.
You utilize the depth so you don't write objects wich are far away over objects near the camera, but you don't gain much performance as the shaders are executed nevertheless.

Except you have Early Depth testing which you have to write something explicitly for it to work as described in the article you posted (except you use 4.2 ofc)

Do I miss something ?
I think we are discussing about two different things
Back to top
Profile PM Email Website
Bengismo
6
Years of Service
User Offline
Joined: 20th Nov 2017
Location: Yorkshire, England
Posted: 21st Sep 2018 19:09 Edited at: 21st Sep 2018 19:11
Link
Janbo, im really not looking for a fight over this and I know your a really smart guy but we are totally derailling this thread right now

So...please try this example


It creates 100 quads behind each other....it will render front to back or back to front....pressing "a" toggles the order


Rendering front to back (the optimal way" gives me >3700FPS, rendering back to front gives me just 700 fps

Feel free to try it.

If you were right then the numbers would be the same in either case...but they arent... the reasoning is simple.....early exits on the fragment shader. this has been true since open gl and direct x back in the 90's!



Anyway....this thread was about Terrain LOD techniques and possibilities in AGK....so maybe we should get back to topic? Have a good weekend all. Im off for a
Back to top
Profile PM
janbo
15
Years of Service
User Offline
Joined: 10th Nov 2008
Location: Germany
Posted: 21st Sep 2018 19:15 Edited at: 21st Sep 2018 19:22
Link
Quote: "we are totally derailling this thread right now"

Agreed

Well your test code gives me no change around 800fps for both(I can make a video if you like)
I have a GTX 970 what are you testing on ?
Back to top
Profile PM Email Website
Bengismo
6
Years of Service
User Offline
Joined: 20th Nov 2017
Location: Yorkshire, England
Posted: 21st Sep 2018 19:32 Edited at: 21st Sep 2018 19:34
Link
lol...your GTX970 is slower than my old test machines ancient GTX770 card then... ha

Looks like your card does not optimize and draws the same no matter what and slower for that matter...or - you have somehow disabled the optimisation



I will wait and see what others get if anyone wants to try it.

I just tried it on the laptop and its faster front to back on there too....

Might try my home pc but thats only got a 800 card in it so not sure what to expect tbh...
Back to top
Profile PM
janbo
15
Years of Service
User Offline
Joined: 10th Nov 2008
Location: Germany
Posted: 21st Sep 2018 19:41
Link
For sience I tested it on my thinkpad and I got 700 for front to back 145 for back to front.
But it has a faster cpu not sure if that is the culprit
It has a I7 8gen and my PC a I7 1gen...
Back to top
Profile PM Email Website
GarBenjamin
AGK Developer
7
Years of Service
User Offline
Joined: 30th Nov 2016
Location: USA
Posted: 21st Sep 2018 19:46 Edited at: 21st Sep 2018 20:02
Link
I know I am furthering the derail but just wanted to say I tested your code Bengismo on my laptop that also has an Nvidia GTX970M and of course an Intel HD integrated video card as well.

On the integrated graphics card drawing front to back was over 7 TIMES faster than drawing back to front so is truly a MASSIVE difference.

On the GTX970M both ran at exactly the same speed exactly as Janbo described. I even increased the number of objects to 15,000 and still there was no difference between front and back.
Maybe the Nvidia as an optimization queues up all of these incoming draw requests, orders them for max speed and then performs the rendering?

Or more likely since in my case on the Nvidia all tests are always running at 60 fps vsync is likely enabled on the control panel. I am waiting for the GeForce panel to open so I can change that. LOL

But anyway the integrated card definitely proves your point Bengismo. Front to back drawing is SIGNIFICANTLY faster.

EDIT: That was it... using 500 objects I get 700+ fps front to back and 600+ fps back to front. So there is a difference but nowhere as significant as the integrated graphics card. I am guessing the pipeline sees all of these drawing commands coming in so fast one after another it sorts them before drawing or something along those lines as some kind of internal optimization. Just my guess there of course. At any rate it shows this can make a HUGE difference on some devices / cards.
My Projects Most With Full Source...
[Attack of the Crows] [Christmas Command 2016]
[Simple Sprite Shaders Example] [Old School Style 3D FPS]
[Playable 2D Shmup Template] [2D Single Screen Platformer]
[FixSpriteToSprite System & Demo]
[Mutant Invaders 3D Shmup] [Unity - Atlantic Crisis]
[Unity - Treasures of Ali-Gar] [My YouTube Videos] [Twitter]
[AppGameKit BASIC Extended API]
Back to top
Profile PM
janbo
15
Years of Service
User Offline
Joined: 10th Nov 2008
Location: Germany
Posted: 21st Sep 2018 20:05 Edited at: 21st Sep 2018 20:05
Link
I did a reset on my NVIDIA settings and it made no difference.
Then I installed the newest driver and now I get 1500fps F to B and 980fps B to F.
So could that be a driver optimization ?
I'm really puzzled here as I never experienced it that way as you can imagine.
But good to know.
Back to top
Profile PM Email Website
GarBenjamin
AGK Developer
7
Years of Service
User Offline
Joined: 30th Nov 2016
Location: USA
Posted: 21st Sep 2018 20:27
Link
Could very well be it is implemented in the driver. A person would almost need to get a white paper on each particular driver / card to understand exactly what is happening. And sometimes driver updates actually do decrease performance on some things only to be fixed in the next update (like all software dev . I wonder of maybe the driver is even sending the draws over to the card until the Swap() is called? All I know is my integrated card seems to be processing them exactly as they are sent and the Nvidia seems to ve doing something smarter about it. Maybe when the swap is called the driver has queued all those calls and orders them before sending to the card. It seems like even the integrated card would be doing that if it makes this kind of difference.

Ah well... I always develop against the integrated card thinking whatever I can run at 60 fps there majority of people should be able to run at 60 (outside of mobile where apparently even 3D html5 builds are faster on my laptop than on some cell phones lol).
My Projects Most With Full Source...
[Attack of the Crows] [Christmas Command 2016]
[Simple Sprite Shaders Example] [Old School Style 3D FPS]
[Playable 2D Shmup Template] [2D Single Screen Platformer]
[FixSpriteToSprite System & Demo]
[Mutant Invaders 3D Shmup] [Unity - Atlantic Crisis]
[Unity - Treasures of Ali-Gar] [My YouTube Videos] [Twitter]
[AppGameKit BASIC Extended API]
Back to top
Profile PM
puzzler2018
User Banned
Posted: 21st Sep 2018 20:35
Link
I agree,

My Office 365 2016 had major display issues, until i reverted back to an older driver and then worked perfectly

Drivers are a blooming nightmare - its just trying to find the right one
Back to top
Profile
janbo
15
Years of Service
User Offline
Joined: 10th Nov 2008
Location: Germany
Posted: 22nd Sep 2018 10:11 Edited at: 22nd Sep 2018 20:57
Link
I'm sorry for hijacking your thread Plotinus
But I think your thread brought something really interesting up to light.

I figured I get 1500fps only at the beginning and after a few seconds it stays at 1000 fps even if I replace the draw order code with just sync().
Also, I only loose performance not gain performance if I use the draw order code(B to F) and the nearer I go to the planes the worse it gets(overdraw). With sync I stay at above 1000 again no matter where I look
I guess thats probably what Bengismo wanted to point out, Its not about the processing of shaders rather than sending the fragments to the buffer.

The Dist=Current-Old is only there because AppGameKit adds to the values every time it draws an object, I get the difference for each frame here.

To get back to you Plotinus.
I think Portal Culling is a good algorithm as its not too hard to implement in relation to other culling algorithms, its only really useful for indoor scenes but very fast.
And You can still replace your high poly objects with lower poly objects in the distance, even if the calculation is running on the CPU you should gain some performance, as long as you dont have only two objects in the scene
Back to top
Profile PM Email Website
Plotinus
15
Years of Service
User Offline
Joined: 28th Mar 2009
Location:
Posted: 22nd Sep 2018 16:02
Link
Quote: "I'm sorry for hijacking your thread Plotinus"


Hey, don't worry, it's all educational!
Back to top
Profile PM Website
Back to top

Login to post a reply

Server time is: 2024-04-16 10:08:25
Your offset time is: 2024-04-16 10:08:25