Sorry your browser is not supported!

You are using an outdated browser that does not support modern web technologies, in order to use this site please update to a new browser.

Browsers supported include Chrome, FireFox, Safari, Opera, Internet Explorer 10+ or Microsoft Edge.

Program Announcements / jGfx - Rendering and Effects Plugin for DBPro

Author
Message
revenant chaos
Valued Member
10
Years of Service
User Offline
Joined: 21st Mar 2007
Location: Robbinsdale, MN
Posted: 8th Aug 2017 17:50 Edited at: 10th Nov 2017 01:24
Hi Everyone, I would like to present my newest plugin: jGfx. It combines the commandsets of my previous ID3DXEffect and MRT plugins, plus adds commandsets for dynamic cubemap textures, geometry instancing and postfilter cameras. There are no help files yet, but the download comes with a keywords file and a few example projects.

Shader_ Commandset Features:
*) Get/Set effect constants using pointers or by value
*) Supports all 32-bit numerical HLSL datatypes and arrays
*) Uses parameter handles to decrease string-based constant lookups

MRT_ Commandset Features:
*) Provides concurrent rendering to up to 4 target images with a single sync

CubeTex_ Commandset Features:
*) Provides creation of dynamic cubemaps using any supported image format
*) No hard-coded limit to the number of dynamic cubemaps in existance
*) Supports cubemap resolutions which are greater than the screen height
*) Allows rendering individual cubemap faces
*) Able to Save rendered cubemaps to file

GeoInst_ Commandset Features:
*) Provides hardware accelerated rendering of mesh instances by interleaving vertexdata with an array of instance data
*) Supports user-defined element definition for storing custom data on a per-instance basis

EfxCam_ Commandset Features:
*) Performs off-screen quad renders without being slowed by dbpro object counts
*) Mimics the native camera commands for ease of use



Update 11/2/2017:
================================================
*) Added commands for getting/setting shader vector arrays using arrays of vector pointers
*) Added a command for setting shader techniques by name
*) Added commands for sorting the GeoInst render queue based on user-designated priority values
*) Added MRT capabilities to CubeTex and EfxCam command sets



New Commands (10):
--------------------------------
Shader_SetVectorPointerArray pEfx, ptr_VecPtrArr, VecStride, Count
Shader_GetVectorPointerArray pEfx, ptr_VecPtrArr, VecStride, Count
Shader_SetTechniqueName pEfx, NameStr$

GeoInst_SetRenderPriority pGeoInst, Priority
Priority = GeoInst_GetRenderPriority(pGeoInst)
Index = GeoInst_GetRenderIndex(pGeoInst)
GeoInst_SortRenderQueue

CubeTex_SyncMRT pCubeTex, pCubeTexB, pCubeTexC, pCubeTexD, CamID
CubeTex_SyncFacesMRT pCubeTex, pCubeTexB, pCubeTexC, pCubeTexD, CamID, dwFaceMask

EfxCam_SetMRTImage FxCamID, TargetIndex, ImageID [, D3DFMT]



Command List (154 Total):
+ Code Snippet

All comments, questions, and suggestions are welcome.

Attachments

Login to view attachments
EVOLVED
14
Years of Service
User Offline
Joined: 9th Feb 2003
Location: unknown
Posted: 9th Aug 2017 14:06
This is a fantastic plugin revenant, finally we can have the ability to have a fully deferred rendering pipeline.

The current AL sys has been converted to use your plugin here AdvancedLightingMRT.zip
revenant chaos
Valued Member
10
Years of Service
User Offline
Joined: 21st Mar 2007
Location: Robbinsdale, MN
Posted: 10th Aug 2017 05:14
Thanks EVOLVED, I'm glad to see this is being put to good use.
revenant chaos
Valued Member
10
Years of Service
User Offline
Joined: 21st Mar 2007
Location: Robbinsdale, MN
Posted: 10th Aug 2017 11:32
Updated the first post with a new version.
Kuper
9
Years of Service
User Offline
Joined: 25th Feb 2008
Playing: Planescape:Torment
Posted: 10th Aug 2017 14:04
Good job!Plugin is realy great!
I want to use your object instance system instead of DBPro's - its works fine both on my PC and laptop
but will it work on others? You have "CheckDeviceSupport" commands so is there a chance that someone will not
get things work properly?
revenant chaos
Valued Member
10
Years of Service
User Offline
Joined: 21st Mar 2007
Location: Robbinsdale, MN
Posted: 10th Aug 2017 14:20
GeoInst_CheckDeviceSupport() returns true if the device supports concurrent data streams, and probably wasn't necessary. Even old Dx8 gpus supported up to 8 so it is very unlikely any hardware in use today couldn't support it.
revenant chaos
Valued Member
10
Years of Service
User Offline
Joined: 21st Mar 2007
Location: Robbinsdale, MN
Posted: 13th Aug 2017 23:21
Updated the first post with a new version.
Chris Tate
DBPro Master
9
Years of Service
User Offline
Joined: 29th Aug 2008
Location: London, England
Posted: 22nd Aug 2017 12:28
I look forward to trying this out
Kuper
9
Years of Service
User Offline
Joined: 25th Feb 2008
Playing: Planescape:Torment
Posted: 22nd Aug 2017 19:41
I try to use GeoInst system with objects which have multiple UV texture stages and custom diffuse color and this looks to be impossible.
GeoInst use TEXCOORD1-TEXCOORD4 to generate matrix and TEXCOORD5 for special individual data. COLOR0 still not in use but do not work.
So I guess only one solution is to move object UV data to stages TEXCOORD6,TEXCOORD7 manually?
Also it requires to switch on up to 8 UV stages by "Convert Object FVF" function - this will inflict more memory
usage.When you load complex level geometry it can be crucial.
Maybe it will be better to move GeoInst functions to stages TEXCOORD3 - TEXCOORD7 which offen are not in use at all?

revenant chaos
Valued Member
10
Years of Service
User Offline
Joined: 21st Mar 2007
Location: Robbinsdale, MN
Posted: 23rd Aug 2017 07:42 Edited at: 23rd Aug 2017 11:45
Quote: "GeoInst use TEXCOORD1-TEXCOORD4 to generate matrix and TEXCOORD5 for special individual data. COLOR0 still not in use but do not work."

The TEXCOORD indices used for passing instance data (world matrix and/or custom data) are not rigid and will be shifted to accommodate a mesh's existing UV stages. If the object uses TEXCOORD0 and 1, then the world matrix will be passed using TEXCOORDs 2-5. The plugin uses DBPro's tangent and binormal generation which is known to cause issues with vertex colors, does your shader make use of normal mapping?

Quote: "So I guess only one solution is to move object UV data to stages TEXCOORD6,TEXCOORD7 manually?"
That won't work, the world matrix (and custom data) are stored within a separate vertexbuffer which is interleaved with the mesh's vertexbuffer at render time. If you add TEXCOORDs 6 and 7 to the source object's vertexdata it will consume all 8 TEXCOORDs and there will be no remaining elements for passing instance data.

I've attached an example which demonstrates geometry instancing for objects with 2 texture stages and vertex diffuse.

Attachments

Login to view attachments
revenant chaos
Valued Member
10
Years of Service
User Offline
Joined: 21st Mar 2007
Location: Robbinsdale, MN
Posted: 23rd Aug 2017 09:56 Edited at: 23rd Aug 2017 11:44
[Orig Post Deleted]
It turns out the "bug" I had been experiencing with FVF formats was just me being silly...
0x100 = 256
0x200 = 512
0x300 = 768
0x100||0x200 = 768 = 0x300
Kuper
9
Years of Service
User Offline
Joined: 25th Feb 2008
Playing: Planescape:Torment
Posted: 23rd Aug 2017 18:35
Quote: "I've attached an example which demonstrates geometry instancing for objects with 2 texture stages and vertex diffuse."

Thanks a lot! For me works fine even with SetCustomData function.
Conflict between DIFFUSE and binormal can be avoided with computing binormal data to some of UV chanels which can be done manually
with DBPro.

Quote: "It turns out the "bug" I had been experiencing with FVF formats was just me being silly..."

Yep it is kind of sum. All my life I did this the wrong way

PS. This works also:

`dont generate TEXCOORD3 at all
Convert Object FVF Object,0x002||0x010||0x100
Lock Vertexdata for limb Object,0
VtxCount = Get VertexData Vertex Count()-1
for i=0 to VtxCount
u#=get vertexdata u(i,0)
v#=get vertexdata v(i,0)
set vertexdata uv i,0,u#*0.125,v# //TEXCOORD0
set vertexdata uv i,1,u#*0.25,v# //TEXCOORD1
set vertexdata uv i,2,u#*0.5,v# //TEXCOORD2
set vertexdata uv i,3,u#*2,v# //TEXCOORD3
next i
Unlock Vertexdata

So, DBPro generate FVF automatically when you start to edit uv data which is not
exist yet.

revenant chaos
Valued Member
10
Years of Service
User Offline
Joined: 21st Mar 2007
Location: Robbinsdale, MN
Posted: 23rd Aug 2017 19:24
Quote: "PS. This works also:"
Quote: "So, DBPro generate FVF automatically when you start to edit uv data which is not exist yet. "

Actually it seems those calls to set TEXCOORD1-3 would simply do nothing. If you apply a shader which requires those additional stages it will not work unless the FVF was setup properly.
Kuper
9
Years of Service
User Offline
Joined: 25th Feb 2008
Playing: Planescape:Torment
Posted: 23rd Aug 2017 20:28
Also find this small bug:
"disable object zwrite" command doesn't work =(
It useful for backdrops and all such things which most be always be drawn behind all others.
( I mean that disable object zwrite is set to object which is NOT instanced )
revenant chaos
Valued Member
10
Years of Service
User Offline
Joined: 21st Mar 2007
Location: Robbinsdale, MN
Posted: 23rd Aug 2017 22:23
Hmm.. I hadn't noticed that issue, thanks for bringing it to my attention. Could you check if the problem exists when using blitzterrain as well? At the moment I don't have a clue what could be causing this problem, but in the meantime a possible work-around is to achieve the same effect using zbias:
+ Code Snippet
Kuper
9
Years of Service
User Offline
Joined: 25th Feb 2008
Playing: Planescape:Torment
Posted: 24th Aug 2017 10:41 Edited at: 24th Aug 2017 10:46
Your snippet works excellent, as well!
So, bug become unnoticeable IMHO

Yes, disable object zwrite doesn't work with BlitzTerrain.
revenant chaos
Valued Member
10
Years of Service
User Offline
Joined: 21st Mar 2007
Location: Robbinsdale, MN
Posted: 24th Aug 2017 11:47
Quote: "Your snippet works excellent, as well! So, bug become unnoticeable IMHO"

Awesome, I'm glad it works.

Quote: "Yes, disable object zwrite doesn't work with BlitzTerrain."

I am both relieved and troubled to hear the bug also exists with BlitzTerrain. Relieved because I think it means I am not doing something incorrectly in my render code, but troubled because that leads me to think the problem occurs when external renderers are setup with dbpro... what other undiscovered problems may crop up? On the other hand BT has been around for quite some time and has been used by many, so hopefully this small issue is the exception. Thank you for testing Kuper
revenant chaos
Valued Member
10
Years of Service
User Offline
Joined: 21st Mar 2007
Location: Robbinsdale, MN
Posted: 27th Aug 2017 04:31
Updated the first post with a new version.
Kuper
9
Years of Service
User Offline
Joined: 25th Feb 2008
Playing: Planescape:Torment
Posted: 29th Aug 2017 19:19
Is it available to create two CustomElements float4 with GeoInst?
I've tried something like this:

GeoInst_DeclareCustomElements pGInst,1
GeoInst_AddCustomElement pGInst, 4
GeoInst_DeclareCustomElements pGInst,2
GeoInst_AddCustomElement pGInst, 4
GeoInst_FinishCustomElements pGInst

but it doesn't work
revenant chaos
Valued Member
10
Years of Service
User Offline
Joined: 21st Mar 2007
Location: Robbinsdale, MN
Posted: 29th Aug 2017 21:03
Try this:
+ Code Snippet
Kuper
9
Years of Service
User Offline
Joined: 25th Feb 2008
Playing: Planescape:Torment
Posted: 31st Aug 2017 18:17 Edited at: 31st Aug 2017 20:18
Thanks again, man!
PS
Looks like instanced object doesn't work correctly with alpha blending.
On example image left- instnaced , right standart DBPro plain
Blue is the color of camera backdrop - so alpha chanel on instanced objects blends with it not the background
object, however blending between instanced object themselves look correct.

Attachments

Login to view attachments
revenant chaos
Valued Member
10
Years of Service
User Offline
Joined: 21st Mar 2007
Location: Robbinsdale, MN
Posted: 1st Sep 2017 02:07
Could you post a small example for me to experiment with?
Kuper
9
Years of Service
User Offline
Joined: 25th Feb 2008
Playing: Planescape:Torment
Posted: 1st Sep 2017 12:11

Attachments

Login to view attachments
revenant chaos
Valued Member
10
Years of Service
User Offline
Joined: 21st Mar 2007
Location: Robbinsdale, MN
Posted: 2nd Sep 2017 04:14 Edited at: 2nd Sep 2017 04:18
I had a look around the DBPro source and it seems news isn't good. Correctly handling certain render states (such as alpha blending) requires usage of dbpro's object manager, but the object manager wasn't written to be utilized by third-party renderers. External render functions are called once per sync, but it seems proper support would require DBPro to call external render functions once for each phase of it's rendering process. Unless I am mistaken (and I hope someone can correct me), it looks like third-party renderers are limited to alphatest/clip based transparency. Ghosting also seems to be effected.
Kuper
9
Years of Service
User Offline
Joined: 25th Feb 2008
Playing: Planescape:Torment
Posted: 2nd Sep 2017 18:40
So the only solution is that all objects must be created witn GeoInst to have proper blending..
Kuper
9
Years of Service
User Offline
Joined: 25th Feb 2008
Playing: Planescape:Torment
Posted: 19th Oct 2017 21:09 Edited at: 20th Oct 2017 01:31
Find that alpha blending do not work perfect also with GeoInstances.
Something wrong with z-sorting, I guess

Hope there is someway to solve this
Maybe way to set z-sorting manually?
revenant chaos
Valued Member
10
Years of Service
User Offline
Joined: 21st Mar 2007
Location: Robbinsdale, MN
Posted: 20th Oct 2017 21:37 Edited at: 20th Oct 2017 21:38
Hi Kuper,
Geometry instances are rendered as one gigantic object using a single draw call. Unfortunately this doesn't play well with alpha blending as polygons which are rendered first cause future polygons to fail their z-depth test.

+ Code Snippet

Instances are drawn sequentially by their index so you could manually sort them, but updating geoinstances involves manipulating a vertex buffer which can be costly to perform each loop and would still not solve all potential problems. Have you considered using shader-based batch rendering?
Kuper
9
Years of Service
User Offline
Joined: 25th Feb 2008
Playing: Planescape:Torment
Posted: 20th Oct 2017 22:57
@revenant chaos
Quote: "Have you considered using shader-based batch rendering?"

I just play with little demo which I upload time before.So I use common shader ( which you provide for geo instances )
Quote: "so you could manually sort them"
- how can I do this? I try to finish my GUI system.I use your plugin for instances
because it is pretty fast - but I need to sort all stuff ( windows, buttons etc. ) at least once
revenant chaos
Valued Member
10
Years of Service
User Offline
Joined: 21st Mar 2007
Location: Robbinsdale, MN
Posted: 22nd Oct 2017 02:41
You could use a UDT array to store each instance's properties along with a depth variable, then use MatrixUtils' Sort Array command to sort by depth in descending order. Once everything is sorted write each element's data into the instances (geoinst index = array element index+1).
Kuper
9
Years of Service
User Offline
Joined: 25th Feb 2008
Playing: Planescape:Torment
Posted: 22nd Oct 2017 21:19 Edited at: 22nd Oct 2017 21:22
Thanks I'll try this
but found that z-sorting is not working between different GeoInstance object groups ( I mean GeoInst itself , not each of it instance )
GeoInst A and B here are made in sequence
revenant chaos
Valued Member
10
Years of Service
User Offline
Joined: 21st Mar 2007
Location: Robbinsdale, MN
Posted: 23rd Oct 2017 00:07 Edited at: 23rd Oct 2017 00:08
I'm not really sure what to do about that one. I could add functions to query and modify the order which geoinstances are rendered, but I'm not sure how user friendly it would be, and it would only work in cases where all of GeoInstA's geometry can be guaranteed to be infront of all of GeoInstB's geometry or vice versa. Mabey this article could help you, the two pass rendering technique it describes might be what you are looking for.
Kuper
9
Years of Service
User Offline
Joined: 25th Feb 2008
Playing: Planescape:Torment
Posted: 23rd Oct 2017 01:35
Thanks for answer!
Quote: "I could add functions to query and modify the order which geoinstances are rendered, but I'm not sure how user friendly it would be"

each instance ZDepth = instance to render camera distance
then GeoInst_SetZDepth pGInst, InstanceId ,ZDepth
something like that, I guess
though z-sorting can slow down your super fast instance rendering but if it is not updating every sync it can be avoid
revenant chaos
Valued Member
10
Years of Service
User Offline
Joined: 21st Mar 2007
Location: Robbinsdale, MN
Posted: 23rd Oct 2017 03:45 Edited at: 23rd Oct 2017 03:57
Quote: "each instance ZDepth = instance to render camera distance
then GeoInst_SetZDepth pGInst, InstanceId ,ZDepth
something like that, I guess"
GeoInst groups are rendered as individual objects, so that would not solve any problems when it comes to sorting polygons across different GeoInst groups. When I said I could add functions to query and modify the order which geoinstances are rendered, I meant the GeoInst groups themselves (not the individual instances within each group). Adding the ability to specify per-instance depth would substantially complicate the library, yet would not provide anything that cannot already be achieved by ordering indices by depth.

In your screen shots ObjectA and ObjectB are being rendered using two separate draw calls; I could allow user control over the order in which those draw calls occur. Properly rendering all instances back-to-front would require each individual instance to be rendered using a separate draw call, but that is exactly what geometry instancing was designed to avoid.

It seems your best option might be to use a single geoinst group combined with a texture atlas. You could pass UV offset and scale values as custom elements, then use them to control which parts of the atlas are mapped to each instance.
Chris Tate
DBPro Master
9
Years of Service
User Offline
Joined: 29th Aug 2008
Location: London, England
Posted: 30th Oct 2017 19:24
Can't wait to try this plugin out, I could do with a post rendering camera which works with anti-aliasing; hopefully this plug-in will contain the solution.
Kuper
9
Years of Service
User Offline
Joined: 25th Feb 2008
Playing: Planescape:Torment
Posted: 30th Oct 2017 22:09
@Chris Tate
I think only way to achieve true anti-aliasing is to use STYX plugin
revenant chaos
Valued Member
10
Years of Service
User Offline
Joined: 21st Mar 2007
Location: Robbinsdale, MN
Posted: 31st Oct 2017 04:15
Hi Chris,
I'm afraid Kuper is correct. The EfxCam commandset doesn't add anything which isn't already possible using the native camera commands, it merely provides a faster way to render post-filter screen quads. DBPro's native cameras slow down as the object count increases, regardless of how many objects actually render to those cameras. The advantage of using EfxCams is that they are rendered without iterating through dbpro's object list, in exchange for being limited to rendering a single object per camera. I'm not sure how/if anti-aliasing could be implemented, but EfxCams should be useful for speeding up existing shader based solutions.
Chris Tate
DBPro Master
9
Years of Service
User Offline
Joined: 29th Aug 2008
Location: London, England
Posted: 31st Oct 2017 13:47 Edited at: 31st Oct 2017 13:48
Quote: "@Chris Tate
I think only way to achieve true anti-aliasing is to use STYX plugin "


In my game I have been using multi-sampling parameter in display mode. It's good enough for my needs, the only problem is it appears to disregard multisampling when you render to an image; which is a pain. For this reason I have not paid much attention to post rendering because for my project I would rather the Anti-aliasing than post rendering because of the basic cartoon look I am going for.

I have a feeling it does not work on all GPUs because I am sure it didn't work on my old system. I might consider using Styx as an alternative option for certain machines.

Here is the code from my project; where the Antialias variable was set to the median of the amount available on the GPU. My edges are nice and smooth, by no means any pun intended.

+ Code Snippet
revenant chaos
Valued Member
10
Years of Service
User Offline
Joined: 21st Mar 2007
Location: Robbinsdale, MN
Posted: 2nd Nov 2017 19:07 Edited at: 2nd Nov 2017 19:09
Uploaded a new version, see first post for details.

Login to post a reply

Server time is: 2017-11-25 07:36:51
Your offset time is: 2017-11-25 07:36:51