Sorry your browser is not supported!

You are using an outdated browser that does not support modern web technologies, in order to use this site please update to a new browser.

Browsers supported include Chrome, FireFox, Safari, Opera, Internet Explorer 10+ or Microsoft Edge.

AppGameKit Classic Chat / Landscape shader

Author
Message
Santman
7
Years of Service
User Offline
Joined: 15th Sep 2011
Location: Inverness
Posted: 11th Feb 2018 22:55
So I decided to give this a seperate post to keep things simple. Below is a video of the new version of the shader in action, with a couple of nice little features added (in a WIP state):

* there is a now a plane with a cloud texture on it repeating in hte sky - this image is used to generate shadows of the clouds that will move across the landscape smoothly
* my first bash at a VERY simple water simulation is now in, which allows simple vertex waves
* the cloud image is also used to cast a fake reflection on teh water surface - particularly happy with the fact the land shadow and water reflection seamlessly transition as the clouds move

All teh shadow and reflection work is done from the cloud image, so that can be anything. Next up I aim to generate a shader based cloud sky that will shadow and reflect accordingly. My newly learned vertext shader skills will then turn to making some shader animated grass and trees (hopefully) and my planet landcape generator is going to be ready in it;s simple format.



Once I get time to tidy it up I'll also share the basic landscape shader that lets you blend different textures together smoothly - at the moment very little of it can be controlled via AppGameKit as it's hard coded into the shader, but I'll change that.
puzzler2018
User Banned
Posted: 11th Feb 2018 23:00
take my hat off to you, well done - i think its time to get into memblocks and shaders
Mobiius
Valued Member
16
Years of Service
User Offline
Joined: 27th Feb 2003
Location: The Cold North
Posted: 12th Feb 2018 10:10
Not too shabby. It would look better if the cloud shadows were not so dark however.

I'd like to use this shader when you release it!
Santman
7
Years of Service
User Offline
Joined: 15th Sep 2011
Location: Inverness
Posted: 12th Feb 2018 11:41
Yeah, the cloud shadows are just fed from the alpha channel of the cloud image, so that's completely customizable- You could even colour it for a wide range of effects (including transparency/dissolve effects such as in the shader pack....I assume it's a similar technique you just scale the alpha threshold). Once it's in my game that will all be seed set, so I needed it changeable easily.

I'm going to put a cut down version up so people can play with it when I can.
george++
12
Years of Service
User Offline
Joined: 13th May 2007
Location: Hellas
Posted: 12th Feb 2018 14:17
I am looking forward to use this shader in my current project
Mobiius
Valued Member
16
Years of Service
User Offline
Joined: 27th Feb 2003
Location: The Cold North
Posted: 12th Feb 2018 15:17
Quote: "I am looking forward to use this shader in my current project"

As am I.
Scraggle
Moderator
15
Years of Service
User Offline
Joined: 10th Jul 2003
Location: Yorkshire
Posted: 12th Feb 2018 15:24
Sph!nx
10
Years of Service
User Offline
Joined: 3rd Dec 2008
Location: The Netherlands
Posted: 12th Feb 2018 17:22
Indeed. Very impressive!
Regards Sph!nx
blink0k
AGK Developer
Gold Codemaster
6
Years of Service
User Offline
Joined: 22nd Feb 2013
Location: the land of oz
Posted: 13th Feb 2018 22:10
Is there any code for that terrain height shader i could pinch?
Santman
7
Years of Service
User Offline
Joined: 15th Sep 2011
Location: Inverness
Posted: 13th Feb 2018 22:24
I am going to upload it, as soon as I get normal mapping working (which at this time I can't even get basic normal mapping on a 3d plane working right now!).
blink0k
AGK Developer
Gold Codemaster
6
Years of Service
User Offline
Joined: 22nd Feb 2013
Location: the land of oz
Posted: 13th Feb 2018 22:34
Ok awesome! Thanks heaps!!
Mobiius
Valued Member
16
Years of Service
User Offline
Joined: 27th Feb 2003
Location: The Cold North
Posted: 15th Feb 2018 09:41
Does it have terrain deformation?
Santman
7
Years of Service
User Offline
Joined: 15th Sep 2011
Location: Inverness
Posted: 15th Feb 2018 10:48
That's down to you. The main landscape isn't me, thats a height map, and that one is in sections as I plan to scale it out and need the speed. However if you kept it as a single mesh you could just create a memblock from it then adjust vertex heights and rebuild the object from the mesh...it would be quite easy. Yiu coukd even then use a memblock to make burn marks from an explosion etc, and blend it directly into the ground.

I FINALLY got a normal/specular effect working, and am playing about with a roughness texture on top....hopefully tonight I'll get a start on building that into the landscape and adding specular to the water, then I'll get something shared (probably across the weekend).
Santman
7
Years of Service
User Offline
Joined: 15th Sep 2011
Location: Inverness
Posted: 19th Feb 2018 09:50 Edited at: 19th Feb 2018 09:59
Spots taking time, but I think I've almost cracked the kidnapping seam. It's almost invisible now. Next is normal mapping.....I have 3 options in mind, but blending them is not so easy. Will get a proper play about in a couple of days as travelling right now, but what I'm thinking is:

1) calc each diffuse blend as I do now, with the normals for each
2) calc the normal split and do a single normal pass at the end of the diffuse blending
3) do a threshold test and make there be a more defined blend - which I think will make transitions much nicer and normal mapping ten times easier

Will update and add code as soon as I get one fully working...the base layer with full normal mapping looks AWESOME.
george++
12
Years of Service
User Offline
Joined: 13th May 2007
Location: Hellas
Posted: 19th Feb 2018 10:44 Edited at: 19th Feb 2018 10:44
Wish I could help
Here is a very good article regarding advanced blend
https://www.gamasutra.com/blogs/AndreyMishkinis/20130716/196339/Advanced_Terrain_Texture_Splatting.php
Santman
7
Years of Service
User Offline
Joined: 15th Sep 2011
Location: Inverness
Posted: 19th Feb 2018 12:49
Hi George. Thanks for that, I'll give it a read over. It's not so much how to blend it, more how BEST to blend it.

What I mean is, I had the different layers overlapping and smoothing together, which was fine.....but not natural. The base layer was cracked diet, but the cracks just added into the next layer and it looked odd up close. What I'm thinking now is a simple test to see which layer has "dominance".....So that dirt would start appearing on top of the cracked ground, initially patchy and getting more and more constant.

I'm using the landscape scales up by a factor of 5k, so the player will be up close and personal at that level. Testing and time will tell....maybe I'll mix and match or make it user controlled.
Alien Menace
AGK Developer
14
Years of Service
User Offline
Joined: 11th Jan 2005
Location: Earth (just visiting)
Posted: 20th Feb 2018 06:03
Great work! Do you use Fresnel for the water?
I love my Altair 8800 Replica.

http://altairclone.com/
Santman
7
Years of Service
User Offline
Joined: 15th Sep 2011
Location: Inverness
Posted: 20th Feb 2018 08:59
Not at the moment, the water is really basic and rubbish. It's literally just a mini height map that is used to set vertex heights in the shader by continually offsetting the uv coords.

I have modified it using a sine wave to flow back and forth that looks good on waving grass though.
Santman
7
Years of Service
User Offline
Joined: 15th Sep 2011
Location: Inverness
Posted: 21st Feb 2018 23:00
So this was far harder than it should have been but finally the mi-mapping seams are gone for good, as per the attached screen shot showing the default altas mapping with map mapping compared to the new version with no visible seams between atlas textures but mip mapping still working fine. Now to get on with normal mapping....

Attachments

Login to view attachments
Santman
7
Years of Service
User Offline
Joined: 15th Sep 2011
Location: Inverness
Posted: 22nd Feb 2018 01:52
Phew - that was a long session. However now I have full normal mapping built into the landscape shader. Note, this version doesn't have the seam fix applied to every layer yet, and the clods and water are still turned off as I need some sleep. Hopefully I'll get some tweaking done tomorrow and then get something uploaded this weekend.

In the meantime, what do you think? The textures need some work, but is it looking ok? I'll add a sun and do a simple day cycle tomorrow to show off the bump mapping properly maybe?

Attachments

Login to view attachments
Green7
13
Years of Service
User Offline
Joined: 23rd Nov 2005
Location: Switzerland
Posted: 22nd Feb 2018 07:50
Looks crispy!
Mobiius
Valued Member
16
Years of Service
User Offline
Joined: 27th Feb 2003
Location: The Cold North
Posted: 22nd Feb 2018 09:21
I like the look of that!
Santman
7
Years of Service
User Offline
Joined: 15th Sep 2011
Location: Inverness
Posted: 22nd Feb 2018 10:08
Thanks guys.

Still a bit of tidying up to do (need to fix the seams in the top layers - only the base layer removes them for now, add in the specular lighting, sort some textures- the different rock textures are to show the layers but some look out of place just now) and the basics are done. Cloud shadows and water both already back in and working fine, so now just to the the key variables back to uniforms That can be controlled through AppGameKit and version 1 is done.

Next up I want to look at an alpha controlled detail layer (loose debrus etc), and extending that landscape beyond one height map - but that's for my own game really. Then just to adapt the whole thing to make the inside of spaceships look better. I wonder hiw difficult reflections are?
janbo
10
Years of Service
User Offline
Joined: 10th Nov 2008
Location: Germany
Posted: 22nd Feb 2018 13:47
Did you add a border around the texture to make the atlas textures work with mipmapping ?
What kind of reflections you want to create ? Environment reflections or specular reflections ?
Santman
7
Years of Service
User Offline
Joined: 15th Sep 2011
Location: Inverness
Posted: 22nd Feb 2018 15:15
Yes and no.....that was cumbersome, I needed an automatic way to change out the textures. What I did was clamp the right and top pixels of the image then offset the texell locations of the start and end of the image read to essentially crop the mip mapping off and stitch the edges of the image back.

So code takes the texture, copies the first column of pixels to the end, the top row to the bottom, then offsets the lookup to stay within an invisible border. I can change this in real-time to find the settings that work for each size of tile. I'll video it later if you want.
RickV
TGC Development Director
19
Years of Service
User Offline
Joined: 27th Apr 2000
Location: United Kingdom
Posted: 22nd Feb 2018 15:31
Impressive demo Santman! Badges awarded to you for such a good effort (and probably too late in being awarded!)
Development Director
TGC Team
Rick Nasher
1
Years of Service
User Offline
Joined: 25th Jul 2017
Location: Amsterdam
Posted: 22nd Feb 2018 16:14
This is really becoming epic.
Santman
7
Years of Service
User Offline
Joined: 15th Sep 2011
Location: Inverness
Posted: 25th Feb 2018 20:46 Edited at: 25th Feb 2018 20:46
Hey all.

Ok, so attached is the landscape shader in a (hopefully) usable format. I need to make it so that the start and end positions of the images can be altere from within AppGameKit (think I might move it to a percentage based system), but for now you can just open the PS file and set each layer individually if you want to change it.

The attached code allows you to toggle bump mapping on and off, adjust tile size and land height to play with samples for yourself. There are two test files of landscapes (all from Google - use at your own risk) called "combined1.png" and "combined2.png" - feel free to jump between these. There is also a normals file for combined1 - but it actually looks quite impressive on the combined2 image as well - toggle it on and off to see how that works.

Let me know if you have any issues, but if you improve it (as I'm sure many will) all I ask is that you upload your improvements here too.

Once you've found settings you like just dump them in your own projects with the shaders and you are good to go.

Attachments

Login to view attachments
george++
12
Years of Service
User Offline
Joined: 13th May 2007
Location: Hellas
Posted: 26th Feb 2018 08:37 Edited at: 26th Feb 2018 08:38
Well done! It works nice and smooth on my PC.
When I'll have more time I'll try to experiment in order get a more detailed picture of the shader.
Thank you very much for sharing it.
janbo
10
Years of Service
User Offline
Joined: 10th Nov 2008
Location: Germany
Posted: 26th Feb 2018 10:31
If you use SetGenerateMipmaps(1) you still get seams around every subimage.
Santman
7
Years of Service
User Offline
Joined: 15th Sep 2011
Location: Inverness
Posted: 26th Feb 2018 11:07 Edited at: 26th Feb 2018 11:41
Ah, hold on, confusing my mip maps with my mag filters. You can use this no bother without mip maps (how much extra performance do these give these days?), but it should still be a case of applying the offset as a % of the mip map image width.....so the question Is, can I get that image width from gl?
Santman
7
Years of Service
User Offline
Joined: 15th Sep 2011
Location: Inverness
Posted: 27th Feb 2018 03:47 Edited at: 27th Feb 2018 03:50
Ok, so it's been an epic 7 hours of coding (and counting!) but this time I;ve cracked it. Attached is a video showing thelandscape shader that I uploaded, sitting at around 80fps. We turn on mip mapping and get a huge boost to around 350fps....but the dreaded seams are back.

So we switch to shader V2 (currently using a single texture across all layers but still texturing all 16 layers from the atlas) and we get 700+ fps....WITH NO SEAMS and mip mapping on, and NO PADDING out the textures (these are plain 512x512 images built into the atlas).

Where there's a will there's a way............

GarBenjamin
AGK Developer
2
Years of Service
User Offline
Joined: 30th Nov 2016
Location: USA
Posted: 27th Feb 2018 04:20
Looks like you've been doing some great work! I need to check this out on my laptop tomorrow.
TI/994a (BASIC) -> C64 (BASIC/PASCAL/ASM/Others) -> Amiga (AMOS/BLITZ/ASM/C/Gamesmith) -> DOS (C/C++/Allegro) -> Windows (C++/C#/Monkey X/GL Basic/Unity/Others)
Jack
14
Years of Service
User Offline
Joined: 4th Oct 2004
Location: [Germany]
Posted: 27th Feb 2018 05:19
You did a great work here!


[/url]
george++
12
Years of Service
User Offline
Joined: 13th May 2007
Location: Hellas
Posted: 27th Feb 2018 07:27 Edited at: 27th Feb 2018 07:27
Hi Santman,
Is technically possible to change the size of tile on one texture only?
Santman
7
Years of Service
User Offline
Joined: 15th Sep 2011
Location: Inverness
Posted: 27th Feb 2018 08:19
Thanks guys, it's getting there. Not gonna lie, this one was a slog. Lol. Next I'll add the specular highlights and look at animated grass - if nothing else I've learned a ton about image manipulation figuring this one out. Got something else I have to finish first though.

George, yeah it's perfectly possible - they could be any size and any position. In the original shader it just takes the uv texture position and multiplies it by 0.25 as there are your, so that's a quarter of the image. But as you would know the image width, you would just divide the start coords of any section by the width/height to get that offset. If they were uneven sizes then you'd need another to determine how far to read. For ease I'd just stick to even square tiles......You could make a texture as many across and down as you'd like really. I'll maybe look at making the shader a single routine that uses an array at some point.

Anyway, in the shader the key is the offset calcs.....in the original you'll see that it gets the start coords then adds a tiny value depending on the tile, then multiplies it by 0.998 - these are the magic numbers that remove the edges. But, these can be replaced with a calc too to work them put as a % for any other time sizes. That would take some work though, 16 is fine for me.
RickV
TGC Development Director
19
Years of Service
User Offline
Joined: 27th Apr 2000
Location: United Kingdom
Posted: 27th Feb 2018 10:26
Awesome work Santman! Glad you are working through the various 3D and shader issues and getting positive results.
Development Director
TGC Team
Santman
7
Years of Service
User Offline
Joined: 15th Sep 2011
Location: Inverness
Posted: 27th Feb 2018 11:31
Getting there Rik. I actually redid all the layers last night after posting (because who needs sleep really?) and even with normal mapping I now sit at between 350 and 1000 fps depending on how much of the landscape is visible. That's on GTX 1080 but confident that with smaller textures and and some reduction in landscape quality it'll scale back nicely to being usable on mobile. I think I can potentially also force a lower mip map selectionfor up close too, which might help older/less powerful mobiles.

Found an occlusion cull shader that works brilliantly....but think that would need some agk tweaking to allow it to work....but would allow the terrain to cull trees and bushes etc. Maybe Paul could cast a glance some time if I post a link?
Van B
Moderator
16
Years of Service
User Offline
Joined: 8th Oct 2002
Location: Sunnyvale
Posted: 27th Feb 2018 16:03
Looks great!

I've been working on an 8 texture terrain, ended up using 2 atlas textures, each with 4 tiles.

Anyway, I found a really nice little trick with AppGameKit fog that can boost atmosphere no end. Let me know if its something you're interested in and I'll add it to your demo.
The code is dark and full of errors
Rick Nasher
1
Years of Service
User Offline
Joined: 25th Jul 2017
Location: Amsterdam
Posted: 27th Feb 2018 21:43
Really nice stuff. Tad slow on my aging laptop, but my high end Android phone loves it.

So as long as this remains optional in AppGameKit for higher specced systems then it's a pretty nice addition.
Cheers
Resourceful
5
Years of Service
User Offline
Joined: 29th Jan 2014
Location: every ware
Posted: 27th Feb 2018 22:16 Edited at: 27th Feb 2018 22:30
@Santman

i added my own landscape map 2048 x 2048 using the program
"L3DT - Large 3D Terrain Generator"

it seems that your program did not like it
as it was at the same size as what is there and it had a lot of spiked hills
what am I missing to get to be more spread out ?

@Santman What program are you using to make your landscape ??
Santman
7
Years of Service
User Offline
Joined: 15th Sep 2011
Location: Inverness
Posted: 27th Feb 2018 22:20
VanB....go for it, that's why imposter it here.

Rick, thanks.....bear in mind the demo here is the old shader, I've not uploaded the new one yet.....the new one should REALLY fly if you phone did this one ok.

Resourceful.....is it running at all? If the shader is running and spikes are the problem, you possibly just need to set the smooth value of your terrain. Did you change the terrain settings or height map used? Screenshot?
janbo
10
Years of Service
User Offline
Joined: 10th Nov 2008
Location: Germany
Posted: 28th Feb 2018 14:37 Edited at: 28th Feb 2018 15:22
Santman wrote: "Found an occlusion cull shader that works brilliantly"

Van B wrote: "working on an 8 texture terrain, ended up using 2 atlas textures, each with 4 tiles"

You guys have my attention

@Santman: As you begin trying speed things up, you want to avoid branching and use the build in functions if you can.
Branching: if statements are slow you may get better results with the ternary operator instead (result = a < b ? c : d;) but I think that depends on the GPU if there is optimization for it.
Better you avoid them completely and find a way to calculate you final fragment color.
Build in functions: mix(),clamp(),min(),max() and so on... are optimized usually... you are already doing it extensively
You also want to calculate as much as you can in the vertex shader as there are obviously less vertices than fragments.
Note that the values get interpolated between vertices if you pass them to the fragment shader. You can only pass 32 components to the fragment shader.
You have much lines like: ((2000.0 - 1000.0) / 2.0) just create a constant for it and assign it a value const float maxmidlevel = 500;
Multiplying is faster than dividing *0.5 instead of /2.0 (nitpicking)
You are not using the colorVarying,tile,agk_MeshDiffuse,agk_ViewProj,agk_World,agk_WorldNormal: remove it.
You pass landheight to the fragment shader: you already have this information in posVarying.y
Textures 4 to 7 are reserved for shadow cascades of AGK's shadow mode 3.
Better create 2 shaders one for bumpmapping and one without, instead of constantly checking for bumpon.
Alternatively you could insert a preprocessor variable into the code (insert from AppGameKit code #define bumpon 1 and check in GLSL #ifdev bumpon)

Beside sampling the atlas texture many times, which you cant avoid, your biggest problem is branching.
Santman
7
Years of Service
User Offline
Joined: 15th Sep 2011
Location: Inverness
Posted: 28th Feb 2018 14:49
Thanks Janbo, much appreciated for the tips.....I'll look into further optimisation when I can (and learn more). The newer one is far better already - for example it only samples the atlas when it needs to, rather than sampling the diffuse 16 times for every single call. Would it be quicker to try this in the vertex shader? My aim is to learn how to pass all the data in via an array, so that the code is only one loop that repeats, but that's for ease of controlling it through AGK....not speed. The newer shader is seemingly quite fast anyway, so will see what my mobile gives at some point.

Currently have some (very basic) grass moving via a vertex shader, working on a tree version now. Have you had much joy with instancing objects? I.e. turning one grass object into many internally in the shader? I've read this is much, much quicker - and would mean we would render a single object in agk and the shader would render 49 more. Since numbers of objects seems a big bottle neck in agk I wonder if that's worth investigation?
janbo
10
Years of Service
User Offline
Joined: 10th Nov 2008
Location: Germany
Posted: 28th Feb 2018 15:21 Edited at: 28th Feb 2018 15:26
Don't sample the texture in the vertex shader, the vertex shader would sample the colors at the "corners" only and interpolate between them I guess, you don't want that.
Quote: "Have you had much joy with instancing objects"

Instanced Rendering like this ? ...not yet
I wonder what InstanceObject does exactly ?
PHeMoX
1
Years of Service
User Offline
Joined: 9th Jan 2018
Location:
Posted: 28th Feb 2018 15:38
Quote: "I wonder what InstanceObject does exactly ?"


I hope it renders the exact same mesh for a different object? Like a in-memory only object? Would be a huge performance saver for trees and such.
Santman
7
Years of Service
User Offline
Joined: 15th Sep 2011
Location: Inverness
Posted: 28th Feb 2018 18:01
The example I found did just that......the code sent one grass tuft to the shader, the shader instances it 9 times additionally based on image data from a height map. So for every single object sent, 10 were rendered.

Sure it's not a magic "make agk do unreal graphics" .... but my grass drops below 60fps as soon as I render more than about 5000 of them.

Sounds a lot, but I have a 1080 GTX.....and once the rest of the world etc is rendered, that number would drop hugely.

I'm trying to get it to drop rendering beyond a certain distance from the camera, see if that increases speed. I'm also doing a complex tree and a single plane with a static pic of that tree, then trying to get the shader to "choose" which to render based on distance. Kind of like a modified imposter system.
Santman
7
Years of Service
User Offline
Joined: 15th Sep 2011
Location: Inverness
Posted: 28th Feb 2018 20:09
SO I got a vertex shader that culls objects based on their distance to the camera, passes this to the fragment shader which then discards it.....but it's literally as slow as just drawing them. :-(

Also, using AppGameKit to instance them did nothing to the speed either, so I am going to see if the shader does it - there def seems to be a bottle neck in AppGameKit (T1 at least) drawing objects purely on how many there are. Maybe how it sorts them???
Preben
AGK Studio Developer
14
Years of Service
User Offline
Joined: 30th Jun 2004
Location:
Posted: 2nd Mar 2018 11:07
InstanceObject , only reuse the object and texture in memory , AppGameKit will still render each mesh individual , so there is no speed/drawcall increase only less memory use.

If you got GameGuru Loader you can look at the code how the grass is merged into single object in segments , you can use this for inspiration for your own functions. I think this is the only way to get the increased speed in AGK.

first screenshot:
Got 25249 grass objects , that are merged into 418 segments ( objects ) , 320 is currently inside camera distance , but as it use segments AppGameKit will cull segments that are behind you , so actually there is not many grass object to render.

So you need to look at memblocks , hope you can use the info
Subscribe and checkout great AppGameKit video's here: Videos click here
Latest GameGuru Loader news: News click here
Get GameGuru Loader PBR version here: Steam click here
best regards Preben Eriksen,

Attachments

Login to view attachments
janbo
10
Years of Service
User Offline
Joined: 10th Nov 2008
Location: Germany
Posted: 2nd Mar 2018 14:43 Edited at: 2nd Mar 2018 14:44
Yeah, I was afraid this is the best solution. I hoped to have them as seperate "objects" in some kind so I can manipulate them later on in runtime as you cant easily extract the mesh from the memblock again.
For example let the grass grow as it comes in view range so you don't notice the the grass appeared in front of you. You would need to work with a whole chunk of grass patches then.
But for now its the best solution I guess... I played with vegetation myself for my Terrain demo.

Login to post a reply

Server time is: 2019-05-22 18:16:42
Your offset time is: 2019-05-22 18:16:42