Sorry your browser is not supported!

You are using an outdated browser that does not support modern web technologies, in order to use this site please update to a new browser.

Browsers supported include Chrome, FireFox, Safari, Opera, Internet Explorer 10+ or Microsoft Edge.

DarkBASIC Professional Discussion / How many shader programs?

Author
Message
Fallout
23
Years of Service
User Offline
Joined: 1st Sep 2002
Location: Basingstoke, England
Posted: 29th Nov 2012 17:05 Edited at: 29th Nov 2012 17:05
Link
Has anyone done any performance testing or have any insight into how many shader programs affect performance? I know the basic rule: as few as possible, but apart from that I haven't done any investigations into how many you can have.

I'm currently thinking about a lighting solution that will have many lights on screen. My shader can handle 7, but I may have upto 30 or 40 visible. My idea is to apply different shader programs to different areas of my scene. Each area will have no more than 7 lights, so this gets around my shader limit while still rendering many more to the screen. They're all using the same shader code, but they're using different instances of it with it's variables set to different light positions/brightnesses etc. As far as I know, you cannot change variables on a per object basis when they're using the same shader. Please correct me if I'm wrong.

So does anyone have any experience with stress testing/using loads of shaders and know what bottle necks I may encounter before I jump into a massive code overhaul?

Back to top
Profile PM
mr Handy
18
Years of Service
User Offline
Joined: 7th Sep 2007
Location: out of TGC
Posted: 29th Nov 2012 17:25
Link
Alpha blending is perfomance killer.
tex2D kills fps if used too many times in one pass or samples are taken too far away from original UV.

also fillrate comes into play - smaller objects are way faster than big one full screen.

tests, tests, tests... the best way to know your game limits.
Back to top
Profile PM
Andrew_Neale
16
Years of Service
User Offline
Joined: 3rd Nov 2009
Location: The Normandy SR-2
Posted: 29th Nov 2012 18:29
Link
Quote: "As far as I know, you cannot change variables on a per object basis when they're using the same shader."


I would imagine there must be some way to store variables "in" the object itself to achieve this. Either via a lookup texture or in the vertices, potentially in an extra set of UV data? Whether this would end up more or less efficient than the extra shader though I couldn't say as I'm afraid I've not really done any performance testing with something like this.


Previously TEH_CODERER.
Back to top
Profile PM Email Website
Fallout
23
Years of Service
User Offline
Joined: 1st Sep 2002
Location: Basingstoke, England
Posted: 29th Nov 2012 18:52
Link
That would indeed be possible Andrew, but the reason why I'm limited to 7 lights is because the instruction count is maxed out. I'd require additional instructions to read texture data and convert it, which I don't have.

I've been reading about this on the net and "many lightweight shaders" seems to be an accepted practice, so I may give that a go. I'm probably looking at 30 shader programs tops, and as few as 10 at use at any one time, so perhaps the overhead will be negligible.

Back to top
Profile PM
Andrew_Neale
16
Years of Service
User Offline
Joined: 3rd Nov 2009
Location: The Normandy SR-2
Posted: 29th Nov 2012 19:43
Link
Ah, sorry, I didn't realise that was the case. I can't imagine there should be too much of an issue with the multiple shader approach as it would still be the same number of vertices and pixels being rendered. I guess the only real downside is the you would lose the ability to optimise render batches where objects use the same textures because they would now be using different shaders.


Previously TEH_CODERER.
Back to top
Profile PM Email Website
Fallout
23
Years of Service
User Offline
Joined: 1st Sep 2002
Location: Basingstoke, England
Posted: 29th Nov 2012 20:23
Link
This is true, and I've put a lot of work into making sure objects are using texture atlases and all reading from the same textures. Seems a real shame you can't have 1 shader program and many different sets of data that different objects can read from. If you can, you defo can't do it with DBPro.

I suppose even if the shader program is swapped a lot the textures won't be swapped unless they change, so I'm probably still get the benefit of using the same textures. I'll report back with my findings once I've had a play.

Still interested if anyone has experimented with this kind of problem/solution before though!

Back to top
Profile PM
MrValentine
AGK Backer
15
Years of Service
User Offline
Joined: 5th Dec 2010
Playing: FFVII
Posted: 30th Nov 2012 00:35
Link
Can you not just duplicate the texture atlas? it may increase memory or such, but should allow multiple Shaders...

I am new to this so I may have something out of the box to say here, or learn something myself...

I actually was playing around with DarkShader today, and I may play about with various Shaders soon... I may get some test ideas from here and report back the results, along with screenies

Back to top
Profile PM
Darkhog
13
Years of Service
User Offline
Joined: 25th Nov 2012
Location: Mushroom Kingdom
Posted: 30th Nov 2012 04:14
Link
I think it all comes down to how powerful your GPU and CPU is. With more GPU/CPU power, you should be able to use as much shaders as you please, although then it would be slideshow on older machines.
Back to top
Profile PM
Andrew_Neale
16
Years of Service
User Offline
Joined: 3rd Nov 2009
Location: The Normandy SR-2
Posted: 30th Nov 2012 09:41
Link
Quote: "This is true, and I've put a lot of work into making sure objects are using texture atlases and all reading from the same textures."


That is what I've been doing lately as well. Optimising the heck out of a power-of-2 texture map to as concisely contain a whole bunch of potentially quite different shaped sub-textures has made it so much more time consuming but I do think it is worth it for performance, especially on older machines. That and the fact that the atlas can look kind of awesome at the end, almost like a scrap book. XD

Quote: "I suppose even if the shader program is swapped a lot the textures won't be swapped unless they change, so I'm probably still get the benefit of using the same textures. I'll report back with my findings once I've had a play."


That is true, so perhaps not as much of a problem as I was thinking. I really should know the answer having written the start of a C++ game engine using DirectX 11 (with 9 & 10 fallbacks) directly. There is clearly a reason people use pre-existing frameworks, but it was good fun/stress.


Previously TEH_CODERER.
Back to top
Profile PM Email Website
TheComet
18
Years of Service
User Offline
Joined: 18th Oct 2007
Location: I`m under ur bridge eating ur goatz.
Posted: 30th Nov 2012 09:45
Link
From the little experience I have with Ogre3D, I can tell you that every object has to have a "shader" specifying it's graphical properties. There is actually no such thing as an object without it.

DBP also has a default shader for objects that aren't using effects.

So my assumption is that there is no limit to the quantity of shaders you can have, but it all comes down to how many passes are performed. There are cheap operations that only require one pass, and there are expensive ones that require multiple passes. The total speed can probably be estimated with this:

Total speed = (GPU power)/(total passes)

TheComet

Back to top
Profile PM Email Website
Green Gandalf
VIP Member
21
Years of Service
User Offline
Joined: 3rd Jan 2005
Playing: Malevolence:Sword of Ahkranox, Skyrim, Civ6.
Posted: 30th Nov 2012 15:45
Link
Quote: "DBP also has a default shader for objects that aren't using effects."


Does it? The default DBPro behaviour uses the DX9 fixed function pipeline states not shaders as far as I know.



Back to top
Profile PM Email
Dar13
18
Years of Service
User Offline
Joined: 12th May 2008
Location: Microsoft VisualStudio 2010 Professional
Posted: 30th Nov 2012 16:10
Link
Quote: " Does it? The default DBPro behaviour uses the DX9 fixed function pipeline states not shaders as far as I know."

The fixed function pipeline in DX9 is simply a hardcoded default shader.

Quote: "Seems a real shame you can't have 1 shader program and many different sets of data that different objects can read from. If you can, you defo can't do it with DBPro."

The only way I could think of doing it would be to pass in a struct of object-specific data along with the shader-specific data(light positions, texture information, worldview matrices, etc).

Kinda like this:


I'm not sure how you'd manage to pull that off in DBP,maybe instead of using a struct just use individual variables, but in Ogre/C++ it'd be fairly simple to do, I think.

Of course, I could be totally off in how I interpret interaction between shaders and the program.

Follow WTLD's progress on Twitter
Git repository for WasTeLanD
Back to top
Profile PM Email
Fallout
23
Years of Service
User Offline
Joined: 1st Sep 2002
Location: Basingstoke, England
Posted: 4th Dec 2012 20:53
Link
Just to report back, I have a system which is dynamically increasing shaders and sharing them between objects. Though the test only went from 1 shader to 4, there was no noticeable performance drop (though app was running at 80fps, so more room for unnoticeable slow down).

Looks like using multiple shaders so that different objects can receive different tweakables is a valid approach, performance wise.

Back to top
Profile PM
david w
20
Years of Service
User Offline
Joined: 18th Dec 2005
Location: U.S.A. Michigan
Posted: 4th Dec 2012 22:43
Link
You should not experience any performance problems using multiple shaders.

I'm sure that you already know but you can break a shader down into parts, and have multiple Vertex Declarations. You could have a generic vertex shader and then pass in the structure of your vertex data for what effect your object and shader need.

IDK what your minimum target is but I assume shader 2.0 but you can do much more with shader 3.0.

You dont have to have the pixel and vertex shaders in one file, but I'm sure your already aware of that. Just have a bunch of shaders that do their part and take input from a generic vertex shader.

If you need a sample of what im talking about I'll glady put one together.
Back to top
Profile PM Website
Fallout
23
Years of Service
User Offline
Joined: 1st Sep 2002
Location: Basingstoke, England
Posted: 4th Dec 2012 23:36
Link
Hi David. The issue I was really having was not with the shader code itself needing to follow another path, but with the 'tweakables' passed into the shader. I wanted each object to use exactly the same shader code but have different values passed in (in this case for light positions).

DB only allows you to set tweakables on a per shader basis. So you set a variable in shader 1 for example, and every object using shader 1 will have this variable value. I was hoping all objects could use shader 1 but have their own version of the variables. If this is possible in HLSL I'm not sure but it doesn't seem possible in DB without having a different shader for each object.

Either way, having a different shader for each object doesn't seem to incur any realtime performance hit, as you've said. The only issue is the compilation time which is fairly noticeable. i.e. when "load effect" is called many times it can take several seconds to compile. A clone effect command would be quite handy!

Back to top
Profile PM
david w
20
Years of Service
User Offline
Joined: 18th Dec 2005
Location: U.S.A. Michigan
Posted: 4th Dec 2012 23:44
Link
Yes I can see how the load time would noticeable, and in my opinion should not be that way.

I think I understand your problem of course this is a DBP limitation because of how it "syncs" and then everything is basically rendered by dbp at once so your variable change will change with all objects.

Is there anyway to make dbp draw each object seperatly then draw the next object but with a different set of shader data?

for example

render object with its variable data
then
render next object with its variable data
maybe some sort of hide object/show object based upon what is being rendered?

I think I have an idea -- I will post back if it works...........
Back to top
Profile PM Website
Fallout
23
Years of Service
User Offline
Joined: 1st Sep 2002
Location: Basingstoke, England
Posted: 4th Dec 2012 23:51
Link
That'd work, though if the render was performed with a dbp sync, I have a suspicion it'd be slow, as I think DBP performs all the rotation maths and other per-frame processes when sync is called. Plus I think the z-buffer is cleared so you couldn't render objects on top of each other.

Would be interested to see what you come up with though!

Back to top
Profile PM
Green Gandalf
VIP Member
21
Years of Service
User Offline
Joined: 3rd Jan 2005
Playing: Malevolence:Sword of Ahkranox, Skyrim, Civ6.
Posted: 5th Dec 2012 00:52
Link
Quote: "when "load effect" is called many times it can take several seconds to compile"


Do you mean compile time or run time?

A good way of speeding up shader load time (i.e. at run time) is to set the DoNotGenerateExtraData flag. It seems that DBPro calculates tangents and binormals by default even when the shader doesn't need them. That can make a BIG difference if you're trying to load hundreds of shader copies. Doesn't help if you need the extra data though.
Back to top
Profile PM Email
david w
20
Years of Service
User Offline
Joined: 18th Dec 2005
Location: U.S.A. Michigan
Posted: 5th Dec 2012 01:40
Link
My ideas have failed I think the only way using dbp is just to simply have multiple shaders. The problem is that you can't control what is being drawn and when.

If you could simply say, this->drawme, then change data you would be in business but you just can't.

I was trying to add the data as an extra input in the vertex data information but considering how limited dbp vertexdata commands are you can't simply get the pointer and then iterate through it that way you can only change position/normal/uv data using FVF codes and even then its totally limited.

I have one last idea based on batching, I'll see if I can get it to work.............................
Back to top
Profile PM Website
david w
20
Years of Service
User Offline
Joined: 18th Dec 2005
Location: U.S.A. Michigan
Posted: 5th Dec 2012 02:07
Link
IDK I can't do it. I'm gonna say can't be done or at least not easily. All my ideas come crashing down because of the way DBP sync's.

Sorry couldn't be any help..good luck.
Back to top
Profile PM Website
Fallout
23
Years of Service
User Offline
Joined: 1st Sep 2002
Location: Basingstoke, England
Posted: 5th Dec 2012 09:19 Edited at: 5th Dec 2012 09:20
Link
Thanks for taking a look anyway David. Yep, DBP is limiting as soon as you start trying to drill down, but I suppose that's why it's so easy to use when you keep things simple. I'm often wanting more control over rendering, especially things like draw order.

@Green Gandalf - Good shout. I've seen that command knocking about but never seen the use for it. This could well be that situation. And yes, I mean at run time. I'm fairly sure there is a GPU compilation step of the raw DBS file when you "load effect" and that seems to be causing a decent bit of lag.

Back to top
Profile PM
Back to top

Login to post a reply

Server time is: 2026-07-07 15:09:56
Your offset time is: 2026-07-07 15:09:56