Sorry your browser is not supported!

You are using an outdated browser that does not support modern web technologies, in order to use this site please update to a new browser.

Browsers supported include Chrome, FireFox, Safari, Opera, Internet Explorer 10+ or Microsoft Edge.

3D World Studio / DBP 3dw Importer Underway

Author
Message
FINN MAN
20
Years of Service
User Offline
Joined: 2nd May 2004
Location:
Posted: 11th Feb 2008 22:41
Pixel Perfect would you mind sharing the culling algorithm.

jason p sage
17
Years of Service
User Offline
Joined: 10th Jun 2007
Location: Ellington, CT USA
Posted: 12th Feb 2008 00:44
@Pixel Perfect - I AM SO Behind you on this project - though I know I'm just routing for you from the bleachers! Can't Wait!

@FINN MAN - DbPro or DarkGDK? what's your Flavor Bro? Lost In Thought (aka LIT) is the first person I know of to port Frustrum Culling to DBPro. with help from him and his code - (Yeah I bugged him with questions once in a while) - I managed to get decent results in DarkGDK with it. There is a thread in the DARKGDK forum called Frustrum Culling or something - I'm sure if you search for it etc. for the Original - from LIT, Search the DBPro thread for Frustrum Culling or Frustrum Occlusion Culling...

I forget the specifics but I think Frustrum Occlusion culling - means - "Hide if not in Frustrum"

and pure: "Occlusion Culling" is hide stuff that can't be seen because something is blocking it from your view anyway. I don't think anyone has done Pure "Occlusion Culling" in dbpro or DarkGDK yet, though I just read about a "Program Annoucement" (freedll) that does "Portal Culling" ... You design the "tree" (groups of something should be seen in a room... and it manages it.

Ok nuff Babble.... Great Work Pixel Perfect! (You know - you NAME makes it so your work has to be awesome - anything less... well... wouldn't be Pixel Perfect!)

Pixel Perfect
17
Years of Service
User Offline
Joined: 21st Feb 2007
Location: UK
Posted: 12th Feb 2008 20:08
lol thanks Jason, I'm really glad your following this project with so much interest, but don't put me on a pedestal I'll probably just fall off Besides all the credit is due, yet again, to the originator of this thread - Lost In Thought without whom none of this would of been possible, and Tireswing for his superb front end loader classes!

@FINN MAN
Quote: "Pixel Perfect would you mind sharing the culling algorithm"


By all means, if it proves to be successful then I'll post the code for all to use.

I have coded a test function which adds bounding spheres to all of my objects, now I just need to figure out how I initialise the camera and frustum routine (its been written for openGL) and I should be able to call the sphere culling routine and determine if it works well. It's certainly a much simpler algorithm that the one LIT has been using so should be more efficient but I never count my chickens till they hatch

By the way, I discovered, much to my surprise, that the complex level does render faster with everything loaded as individual objects. I found a bug in my test code which was actually loading the level items as object and single limb pairs when set to create just discrete objects. When I fixed this the average frame rate went up by about 3 and in some places by up to 15 fps. I really wasn't expecting that! Might re-run all the tests again tonight.

Anyhow, summing up, if the culling is effective then we should be looking at some pretty decent frame rates even for high object density levels.

I still wish someone from TGC would 'spill the beans' on how they are getting such performance gains from their internal DBO loader!

Might also help if they fixed the shader bug with PS 1 techniques as they did in DBPro 6.6b. This would up the frame rates a bit more. I notice my bug log is still not even acknowledged!

By the way, I'm not implementing any functionality to do with the 3DWS Groups and Entities; I'll leave that for individuals to code as the use of these is likely to be fairly specific to individual game engines and level design. All the information is loaded from the file and available within the classes so you have easy access to it!

No matter how good your code is, someone will improve on it
Pixel Perfect
17
Years of Service
User Offline
Joined: 21st Feb 2007
Location: UK
Posted: 14th Feb 2008 21:22 Edited at: 14th Feb 2008 21:25
DGDK Loader For 3DWS Update

OK, well things are just looking better and better ... the new frustum culling algorithm I've been experimenting with is up and running and working a treat!

This is a screenshot from the complex level with the culling de-activated:



and the same again with the culling active:



Some areas of the map have risen to over 120 fps with others still down around 20 fps where virtually most of the brush objects are in view, but this is to be expected with frustum culling.

The culling routine has not yet been optimized and is currently only culling objects; it could easily be extended to limbs too.

However, as I have currently set the loader to create predominately objects with no additional limbs it's pretty effective here. Only the terrain currently has additional limbs.

The culling routine is effectively doing what the dbObjectInScreen function was designed to do but fails to do in anything but the simplest of levels.

I will now get down to tidying the loader code up and applying any optimizations I can with a view to releasing the code hopefully in a couple of weeks time.

[EDIT] I will release the frustum culling code at the same time as I need to tidy that up a bit too!

No matter how good your code is, someone will improve on it
Pixel Perfect
17
Years of Service
User Offline
Joined: 21st Feb 2007
Location: UK
Posted: 16th Feb 2008 22:28 Edited at: 16th Feb 2008 22:30
DGDK Loader For 3DWS Update

Found a small bug in the loader today whilst testing which is now fixed. Forgot to initialise a dynamic multidimensional array and then referenced an element which had never had a value assigned. Pre initialising them all to zero fixed it.

Have extended the testing today to check that it handles multiple terrains created within 3D World Studio. Not sure if any one ever uses these but if you do then good news ... it seems to handle it fine! Example screenshot below:



This pretty much concludes my testing and now that the frustum culling is working too I'm moving on to the final tidying up of the code, optimising where I see opportunities and checking for memory leaks.

Shouldn't be too long now.

No matter how good your code is, someone will improve on it
Pixel Perfect
17
Years of Service
User Offline
Joined: 21st Feb 2007
Location: UK
Posted: 17th Feb 2008 15:20
DGDK Loader For 3DWS Update

Well looks like I spoke too soon. Loaded up all the meshes for the complex level today and there are a few that seem to have an angular rotation problem so there is still a small bug in there that needs fixing.

Screenshot below of level with the meshes:



No matter how good your code is, someone will improve on it
jason p sage
17
Years of Service
User Offline
Joined: 10th Jun 2007
Location: Ellington, CT USA
Posted: 17th Feb 2008 15:32
Where is the flaw? the 55gal barrel seems MAYBE to be in the air... I can't see anything "wrong" per say... enlighten me to your whoas...

Pixel Perfect
17
Years of Service
User Offline
Joined: 21st Feb 2007
Location: UK
Posted: 17th Feb 2008 16:24 Edited at: 17th Feb 2008 16:25
Hi Jason ... no problem with that screenshot per say. It just seems to happen with a few mesh objects ... look at the two shots below. The box is fine; as is the bench on the far right but the wooden palettes are totally screwed rotation wise!!!

In 3DWS all is well:



In DGDK all is not ... lol:



Guess its back to checking the code again

No matter how good your code is, someone will improve on it
jason p sage
17
Years of Service
User Offline
Joined: 10th Jun 2007
Location: Ellington, CT USA
Posted: 17th Feb 2008 17:06
Well - the top of the crate seems inverted, but the bottom ALMOST looks like its not world oriented but oriented some how based on the leaning portion of the pallet skid. Hmm. Overall - I'd say this is coming along nicely.

Revisting code - if its any help - misry loves company - I'm rewriting my whole DarkGDK library!!!! I've decided after talking with David Westfall that - for speed reasons - I should for the most part rid my core stuff of Dynamic Linked lists - and instead I've implemented a lean dynamic array class not to unlike the STDLIB vector class - so I'm doing alot of cut-n-past'ing - also part of this is to modularize in such a way I can script the DarkGDK + Physx + AI so my game can have a level creation/gameplay "engine" similiar to other engines with the Genre I'm shooting for.

Pixel Perfect
17
Years of Service
User Offline
Joined: 21st Feb 2007
Location: UK
Posted: 17th Feb 2008 20:03
Yeah, there are some leaning tires that look like they are simply inverted too, but then the odd occasional meshes, like that palette, that just appear at weird angles! 95% seem to be correctly located! The bug will be there somewhere ... just need to find it. Gets a bit frustrating though when it's so close.

Good luck with the code conversion, I have been using stdlib vectors in a lot of my code and find them really efficient on the whole. I'm also looking to achieve as much separation between the game engine and the level design / game play as I can. Seems to be the way to go although involves a bit more careful design at the front-end.

By the way Jason, have you tried the dbSetGlobalObjectCreationMode(1) command. It certainly adds frame rates but there seems to be an associated problem with it when you exit the game. Seems to take forever to exit! Not sure what’s going on there .... probably just need to say 'Dark Basic' lol ... nuf said

No matter how good your code is, someone will improve on it
jason p sage
17
Years of Service
User Offline
Joined: 10th Jun 2007
Location: Ellington, CT USA
Posted: 17th Feb 2008 20:44
Thanx for the wish of luck.

dbSetGlobalObjectCreationMode(1) ? No I didn't know it existed. Hmm.

StdLib Vectors - Good Idea - I'm dumb - I wrote my own LOL

I'm fighting some STUPID code myself. I have a union - its fine - used it before - not very complex - but its complaining about a missing ";" - but its there - which means SOMEWHERE I probably have a extra "}" - AHHHH - means I have to fine tooth comb it all - ahhhh LOL

jason p sage
17
Years of Service
User Offline
Joined: 10th Jun 2007
Location: Ellington, CT USA
Posted: 17th Feb 2008 20:48
What does it do? and what's with the "Upgrade" and no docs?

http://darkgamesdk.thegamecreators.com/?f=upgrade

Email me if ya want - I lost your email address.

Pixel Perfect
17
Years of Service
User Offline
Joined: 21st Feb 2007
Location: UK
Posted: 20th Feb 2008 00:40
DGDK Loader for 3DWS Update

OK, well the mesh positioning problem is now resolved. Screenshot below showing the correct positioning. Turns out I had to add 180 degrees to all the angles and then they all came in right!



I have also put some switches in to allow you to either load in the standard mode where objects are created with limbs or create purely objects with no additional limbs. The second has the advantage of additional speed as I can now do the texture blending on the brushes without the shader which has increased the fps. Also its allowed me to brighten the terrain by applying modulate 2X making it far closer to the brightness in 3DWS.

Testing is definitely now complete and I'm in the final stage of tidying up the code and optimizing where possible.

Interestingly, the number of objects is now reaching the point where the simple frustum culling is no longer adding much to the frame rate due to the amount of time its taking to calculate the exclusions. However, the increased frame rate brought about by the recent changes means this is less important.

No matter how good your code is, someone will improve on it
jason p sage
17
Years of Service
User Offline
Joined: 10th Jun 2007
Location: Ellington, CT USA
Posted: 20th Feb 2008 00:57
Culling - Are you processing a single list of objects/limbs or did you implement a Quadtree?

The Quadtree will help DRASTICALLY depending on camera view. Looking from one end of the level to another tends to be a lot of work regardless - but otherwise - you'd notice huge speed ups. The trick thing that would ice the cake would be occlusion culling - I don't know how to do that - and think you need some sort of vertex list processing - to see if your vision is obstructed or not - .. kinda like portals but a bit more advanced... I guess.. I really havent done one - but portals are defined by "area" you're in on a hierarchy list of sorts where occlusion is simply - YOU BLOCKED MY VIEW? Cool exclude all the obstructed stuff. Which makes a lot of sense again - I don't know how to go about it - never gave it much thought.


I stand by the quadtree thing though - but you should use arrays - vector(c++) and not linked lists - as you want it to process as fast as possible.

Pixel Perfect
17
Years of Service
User Offline
Joined: 21st Feb 2007
Location: UK
Posted: 20th Feb 2008 01:16
No, its simply processing a list at the moment. There is loads of room for improvements such as using quadtrees as you suggest, I have not tried any form of optimization yet just confirmed that it works, mainly because its very much secondary to getting the loader finished! Once thats out of the way I can concentrate on improving the culling code.

By the way, here is a screen shot of the improved (brightened) terrain:



I was planning on using groups in 3DWS as a method of applying simple portal type occlusion, should work fairly well with carefull level design and instantly cut out large amounts of Polys!

No matter how good your code is, someone will improve on it
Pixel Perfect
17
Years of Service
User Offline
Joined: 21st Feb 2007
Location: UK
Posted: 20th Feb 2008 21:32
DGDK Loader for 3DWS Update

A couple more screenshots I took whilst optimizing the code today:





No matter how good your code is, someone will improve on it
jason p sage
17
Years of Service
User Offline
Joined: 10th Jun 2007
Location: Ellington, CT USA
Posted: 20th Feb 2008 21:38
I love that level or model or whatever you want to call it - looks GREAT.

I hope you realease the code and have no objections to my slicing and dicing it to fit my little C++ lib....

I'm drueling for this. I hate when I make a level and it doesn't look the same in game! You're modeling looks awesome.

Pixel Perfect
17
Years of Service
User Offline
Joined: 21st Feb 2007
Location: UK
Posted: 20th Feb 2008 22:01 Edited at: 20th Feb 2008 22:04
Thanks Jason, but this is not my modelling. This was a commercially put together model so that Leadwerks (Josh) could show off the capabilities of 3D World Studio ... and very capable it is too! I have just had to convert all the models and textures so that I can display it in all its glory in DGDK. I have not written loaders for the Leadwerks smf or stf files currently as nobody outside of 3DWS is using them.

Quote: "I hope you realease the code and have no objections to my slicing and dicing it to fit my little C++ lib"


Yep, that's the whole intention of this project. The code needs to be integrated into individuals game engines. I will simply release the code and let people run with it from there. I have no intension of supporting it beyond that as I have diverted the best part of two months on this project and need to get back to my own engine.

Just hold on a short while longer whilst I ensure the code is tidy and there are no glaring memory leaks lol

With regard to everything displaying exactly as in 3DWS, there is still a small technical problem in that we currently have no way of determining how Josh is applying his vertex lightmapping to the X file models. I know LIT has talked to Josh about this and a few other shortcomings but I don't believe anything has been forthcoming. So as a result, currently the meshes as not light mapped!

One solution might be to use the smf files. Tireswing has written a loader for this, so all the data is there, but I haven't attempted to do anything with that yet! It's not ideal as it would mean having to convert all models to the smf format first but if that's the only way to get the lightmapping onto the meshes then we might need to do that. I believe that LIT has also written a .mesh loader and I think there is a convertor available in the Leadwerks engine SDK for converting other model formats to .mesh so this might also be a solution. I will look into it.

No matter how good your code is, someone will improve on it
jason p sage
17
Years of Service
User Offline
Joined: 10th Jun 2007
Location: Ellington, CT USA
Posted: 20th Feb 2008 22:42
Wow - I agree about the 3DWS being an AWESOME editor - but I find it SOOOOO FRUSTRATING that we can't use the output from it with a simple "Save As" whatever.

Also - because I'm mostly using TGC stuff - I can't be sure if its Direct.X's fault it comes in messed - (Glory lost), TGC, 3DWS.

We all know the file format you and LIT managed to reverse engineer mostly is not documented all that well - and the lightmapping...well... THAT'S one MAIN reason I bought it! (Everything lokks so flat without it!)

I'm thankful you did this work - and I hope the 3DWS creator takes note - because - I for one - LOVE the application - but the output eludes me.

So far - I'm not too too worried - cuz I'm not in media creation mode yet - but your loader is coming at a good time for me as I'm writing my (2nd version) of Frustrum/limb culling now - with a few new angles not unlike portalling and the interesting part - is - that in order to get REALLY TIGHT and LEAN code working for Frustrum Culling and LOD - they need to be VERY Tightly Woven from the Object itself right into the culling/lod system... (Example - Will the Object by Static or move around potentially? This changes how its processed both with FRustrum and lod!) I should call it one system because it almost is.

I decided to set up a hierarchy like a portal system and like a quadtree - but its going to kinda be both.... in theory - It's designed so you can set up "Regions" and in side them - smaller Regions - and in side them - smaller etc... when you decide on a "SIZE" for this - you then register objects into the CullSystem - which - if its static - the object finds its way to its own region "Node" based on Collision Center - and attaches to the LIST for that region or node.

In theory this system should allow me to do a list culler like you have - OR do somefancy legwork to get the regions in there - I'm TRYING to make it versatile - and all of this needs to 100% work with DarkPhysics and DarkAi's way of doing things or its all for Naught.

Ahhh.. It will be good to see you back working on your own engine! That's where the .... um... more code to write is! hahha

Pixel Perfect
17
Years of Service
User Offline
Joined: 21st Feb 2007
Location: UK
Posted: 21st Feb 2008 00:05
I like the sound of the combined frustum/lod functionality Jason, good luck with that!

By the way, seemingly DBPro, and presumably DGDK, does back face culling. There is a command dbSetObjectCull(objID,Flag) which seems to control this. Do you know if it’s on by default or do we have to explicitly set it for each object?

Also, have you tried you existing frustum culling routine in levels with thousands of object/limb combinations. I'm just wondering if you have a feel for where it starts to cease to increase fps due to the sheer time spent computing the exclusions?

Also, I might have done LIT an injustice in my previous statement regarding vertex color lighting on the meshes. It does look like it is working to some degree, its just that the colors are a little subdued, I'm going to see if I can improve this. Might be able to do something like I did with the terrains which would involve converting all the limbs to objects and using blending commands to gain the increased flexibility. It’s the strangest thing, but so far increasing the object count at the expense of limbs has only improved the fps! Not what I'd been led to believe at all! However, as I currently clone the mesh objects I might lose a lot of efficiency in converting all the limbs to individual objects, but it’s the only way to get greater control over the texture blending.

No matter how good your code is, someone will improve on it
jason p sage
17
Years of Service
User Offline
Joined: 10th Jun 2007
Location: Ellington, CT USA
Posted: 21st Feb 2008 00:50 Edited at: 21st Feb 2008 00:50
Quote: "
I like the sound of the combined frustum/lod functionality Jason, good luck with that!"


Me too - it's still in the works - and I'm constantly trying to think of ways to prevent "redundant" anything in this culling system. Its one of those things that get so complex - the best way to do it it is to make the simplist pieces work - and the advanced stuff (like regions) designed in such a way that the whole thing works with or without the advanced stuff.. build with legos where possible!



Quote: "
By the way, seemingly DBPro, and presumably DGDK, does back face culling. There is a command dbSetObjectCull(objID,Flag) which seems to control this. Do you know if it’s on by default or do we have to explicitly set it for each object?
"


Pretty sure default is on. Case-n-point - make a dbMakeObjectPlain - flip it around... invisible? I think it will be!

Quote: "
Also, have you tried you existing frustum culling routine in levels with thousands of object/limb combinations.
"
Not there yet - getting STUPID linking errors - not with external stuff... getting junk like this:
But the JGC_OBJECT is DEFINATELY decalred - and the include file for it is EVERYWHERE! grrrrrr... How can you not have circular references? I know - ONE HUGE source file! slows down the editor - and compiling time though - decision decisions. everything is so tightly woven - the compiler LIES to me! grrr


I'm convinced of a few things (forgive obvious stuff):

1: Machine Speed determines how fast/big a list of things is processed. The key is - make the lists smaller!

2: checking the FRUSTRUM stuff is EXPENSIVE - So check as little as possible! (QuadTree/Portal'ish Type Logic - is my means to an end here - Via Region. I have invisible "BOXES" - that have lots of smaller ones inside. If that Invisible BOX is not in the Frustrum then I skip to another REGION - this eliminates TONS of unneccesssary checks.) I also try to limit how many levels DOWN I use because past experience showed me it takes a lot of trial and error to get the right combination if regions etc. You don't want 5 levels of checks before you hit your first REAL object.

3: Logic Dictates - Region Based - Quadtree/octtree - Portal based stuff Frustrum Code only "Touches" stuff that "Qualifies" to be flipped on. What about flipped off? I think this needs to be somehow made as fast as possible - because this is the REAL WORK (easy - but - has the most things to touch each loop - always more offscreen then on! Wel... guess that depends but.... I want to work it out so that this is done in such a way - that if a "Region" was "all Off" from a previous check - and that Whole Region is STILL not in Frustrum - then move on - But invalidate when it becomes in VIEW - to basically t4ry to have as lean and smart a CULLING SYSTEM on both sides of the COIN - "In the frustrum" and "out" Goal ofcourse - to do as little actual processing as possible.

I'm also thinking - it might not be the best way - but Static objects always fall into the rules I just said I was trying to follow above - While Dynamic objects I'm treating as a straight up list. I thought Regions for these also - (and I may revisit this idea) but you would have to make it so everytime an object moved - you would need to check what region it should be in - this seems to be as much work as just checking the frustrum for it and calling it a day. Especially when you add DarkPhysics into the picture + LOD. Ok - Objects move without your control - (Forget Region thing here... just Cull like I said but - with LOD - and DarkPhysics you ALSO need to check what moved - then if its in LOD 2, 3, or out of frustrum - you display the correct OBJECT - with orientation etc based on the excluded/hidden object that the physics engine is moving around. 2 birds one stone - might as well do LOD Physics aligning, and Culling in one system. Maybe this demonstrates some reason for this kind of approach - but its tricky.

A hard rule for when to much is to much? Dunno - Again - my last working Quadtree system - was a balance of 2-4 levels of quading - and then a limit to the objects/limbs mattered. I say tweak because the settings I imagined would work best for my design sucked - but variations to it screamed - either way - beat the pants of not doing it at all.

Quote: "
Also, I might have done LIT an injustice in my previous statement regarding vertex color lighting on the meshes.
"
He's a pretty resilient fellow.

Quote: "It does look like it is working to some degree, its just that the colors are a little subdued, I'm going to see if I can improve this.
"
Weird - but good news. you saying shadow mapping might be working....sorta? Or did I misunderstand?

Quote: "
Might be able to do something like I did with the terrains which would involve converting all the limbs to objects and using blending commands to gain the increased flexibility.
"


There's an improvement we could use - Limbs with just as much control as objects - but if you do an object command - (I definately understand/appreciate/utilize if) have that override the last limb commands... like SET LIMB LIGHT zero, would do what it sounds like but afterwards something like SET OBJECT LIGHT 100 would override - kinda a global.

Quote: "
It’s the strangest thing, but so far increasing the object count at the expense of limbs has only improved the fps! Not what I'd been led to believe at all!
"

This is a good thing! I look at like this:
1: Create/Delete Objects as LITTLE AS POSSIBLE
2: Too MANY objects (ids in use) Can cause a slow down
3: Create Limbs unless its really complex unless rule 2 biting you.


Quote: "
However, as I currently clone the mesh objects I might lose a lot of efficiency in converting all the limbs to individual objects, but it’s the only way to get greater control over the texture blending
"


Well sir - this is a VERY interesting point. I have an idea. You Might want to KEEP the code as is with the Limbs AND add support the other way - and allow the user to control which method to invoke during the load. This kinda makes a win win in the long run. If someone finds one way works best for their level - fine - or what if TGC fixes the Object Count thing? Then tons of objects is great! What if they give us that extra limb control? then you just add the same code you applied to the individual objects to the limb "method" and you are back to the user picking what they like most.

Running out's a time - need to get back to my code! LOL

Pixel Perfect
17
Years of Service
User Offline
Joined: 21st Feb 2007
Location: UK
Posted: 21st Feb 2008 01:38 Edited at: 21st Feb 2008 01:42
Quote: "This is a good thing! I look at like this:
1: Create/Delete Objects as LITTLE AS POSSIBLE
2: Too MANY objects (ids in use) Can cause a slow down
3: Create Limbs unless its really complex unless rule 2 biting you.
"


I think you might of mis-understood what I was saying. All my experimentation with complex levels has shown that the greater the number of objects and the samller the number of limbs the greater the frame rate! Everything as objects with no limbs = highest frame rate.

Quote: "I have an idea. You Might want to KEEP the code as is with the Limbs AND add support the other way - and allow the user to control which method to invoke during the load."


All ready done Jason ... it's switchable. Plus I have a version where you can define how many limbs you want per object making it completely configurable. I might add that in as a third switchable option!

Quote: "you saying shadow mapping might be working....sorta? Or did I misunderstand?
"


No that's right, it is working. 100% for terrains and brushes, it's the lightmapping on the meshes thats a slight problem it just doesn't look as pronounced as it is in the 3DWS editor itself.

Got to go to bed its getting late here. Thanks for your input Jason. Catch you soon!

No matter how good your code is, someone will improve on it
Pixel Perfect
17
Years of Service
User Offline
Joined: 21st Feb 2007
Location: UK
Posted: 21st Feb 2008 01:49
Just before I go, this is the setup code in the main.cpp file controlling the loader just to give you a taste of what it looks like:



No matter how good your code is, someone will improve on it
jason p sage
17
Years of Service
User Offline
Joined: 10th Jun 2007
Location: Ellington, CT USA
Posted: 21st Feb 2008 01:52
Awesome about the configurable options Pixel Perfect!

Quote: "I think you might of mis-understood what I was saying. All my experimentation with complex levels has shown that the greater the number of objects and the samller the number of limbs the greater the frame rate! Everything as objects with no limbs = highest frame rate.
"


No I understood 100%. I was quietly eluding to - I'll use limbs... say... My Tanks have moveable limbs... I might make some objects with limbs - but ...rereading my post....I guess I did sound a little off there....

I found the same thing you did - not only is objects FASTER - the culling (EXCLUDE OBJECT) versus HIDE limb has a much bigger impact - except instances! ....DARN - Hmmm... Got to bring those into the mix... I GOT IT!.... nevermind - I have a way for that too! (My Object class has a "IsInstance" flag and an "IsClone" flag - if you make one object from another - the appropriate flag is set - this will determine HIDE versus EXCLUDE for Objects! (Culling Trees ain't getting off that easy!)

jason p sage
17
Years of Service
User Offline
Joined: 10th Jun 2007
Location: Ellington, CT USA
Posted: 21st Feb 2008 01:53
Pixel Perfect
17
Years of Service
User Offline
Joined: 21st Feb 2007
Location: UK
Posted: 21st Feb 2008 14:24 Edited at: 21st Feb 2008 14:26
Thanks Jason.

Quote: "No I understood 100%"

Sorry about that ... it just read like you were arguing the opposite. Glad you have come to the same conclusion because it kind of goes against what most people in the forums tell us!

I'm always interested to hear about other peoples approaches to their game engines. I need to re-write all my game engine classes now that I have a much better idea of what I'm doing and where I'm going, but will probably wait until I have had a better look at Dark Physics.

Anyhow, back onto the 3DWS Loader. I'm going to revisit the mesh code tonight to see if I can successfully implement the meshes as individual objects and use the extra blending functionality afforded there to do the same for the meshes as I did for the terrains. i.e. use modulate x2 blending to increase the lightness of the textures whilst retaining the vertex color lightmapping.

If I can pull that off then at last we will have a loader that gets as close as it's possible to get with DGDK in rendering the 3DWS levels to look like they do in the original editor.

I'll keep you posted!

No matter how good your code is, someone will improve on it
jason p sage
17
Years of Service
User Offline
Joined: 10th Jun 2007
Location: Ellington, CT USA
Posted: 21st Feb 2008 14:43
Sounds Awesome. Hey - I've been touting this - its a problem - but some would argue its not - but keep it in mind for Frustrum - (Biting me hard - the workaround is NASTY without modifiying every model you own - load - scale - save! Effects DarkPhysics also.

If you load a model - and just use it - everything is cool. If you load a model and scale it - all the functions you use to interogate the Model's Size report back the original model size - not the scaled version. Scaling is Visual Only!

3 Possible Workarounds so far

1: Load in a modeller - Scale - save. (I don't have 3dmax - nor will pay that much $$ - and I don't know a modeller that will load all my DarkMatter Models - allow me to scale - then save - without messing them up - animations for example.)

2: Make your Stats function Wrappers around the DarkGDK ones - and apply the Scale X,Y,Z percentages your self to the result (Though this won't help DarkPhysics - but this would work for culling "Setup" routines.

3: Load The Model - Make Mesh from Object, Make Object From Mesh - (Use this new object for Physics and Frustrum Set Up Routines but don't display it - Exclude it (not sure if exclude stops physics from using it yet) - Then EVERY LOOP - Set Your Original (scaled) model position, orientation to the Invisible Model. (Twice as many Objects YUCK! - Some Say Delete Original - but then texturing and Aniamtions get hosed)

Ahhh... FRustrating - but you need to be made aware. David W. Tried to warn me I'd run into this kind of stuff.

Dewi Morgan
16
Years of Service
User Offline
Joined: 16th Jan 2008
Location: London, UK
Posted: 21st Feb 2008 23:24
I wouldn't put off writing the octree properly. A single level of "boxes" just is not enough.

With only single-level boxes, you'll effectively be doing "iterate over every object", only for every single box. Then for boxes that span your frustrum boundary, you'll be doing "iterate over every item within the box". And then for all items that span the frustrum boundary, "iterate over every limb".

Why? It's just plain inefficient to iterate over everything. Doing three levels of it makes the inefficiency slightly less, but if you're doing three levels why not add a few more and do it right?

No need to maintain a list of "what's in" and "what's out" - no need for a list at all. Just recurse the octree!

(I say octree, but a quadtree would work if you think the library will never be used on map levels with a significant vertical component).

Yet another game programmer
Pixel Perfect
17
Years of Service
User Offline
Joined: 21st Feb 2007
Location: UK
Posted: 22nd Feb 2008 18:14 Edited at: 22nd Feb 2008 18:16
Octrees/Quadtrees are something I need to look in to for my frustum code, I hear what you're saying Dewi.

Quote: "If you load a model - and just use it - everything is cool. If you load a model and scale it - all the functions you use to interrogate the Model's Size report back the original model size - not the scaled version. Scaling is Visual Only!
"


I have heard others mention this Jason, I think in connection with tying physics in. However, I'm a little perplexed, as my simple frustum culling routine is using dbObjectSizeX etc when generating the bounding spheres initially and there are plenty of scaled objects, yet it seems to be working perfectly!

Will knock up some code to investigate this when I have time.

No matter how good your code is, someone will improve on it
jason p sage
17
Years of Service
User Offline
Joined: 10th Jun 2007
Location: Ellington, CT USA
Posted: 23rd Feb 2008 00:58
Get this - In DBPro - there is Get Object Size (ObjID, ActualSizeFlag) ... didn't know about it - doesn't exist in darkGDK - BUT then monotonic found the defines for them - (didn't make it to DarkGDK.h file - ok Got that) works Great... But there is another issue I posted about after getting through this in the DarkPhysics + AI forum - I won't get to heavy here - You and LIT's thread more or less

Pixel Perfect
17
Years of Service
User Offline
Joined: 21st Feb 2007
Location: UK
Posted: 23rd Feb 2008 09:55 Edited at: 23rd Feb 2008 12:34
Thanks Jason, I do appreciate people not going too far off topic. I'll take a look at your post.

Suffice to say, until I see some real indication from TGC that they are taking any of the myriad of problems with their products seriously, and the problems those cause for their customers, then I am going to regard their languages as nothing more than prototyping languages before developing the game in something else.

Getting back to the loader. I have had some success in converting the code to use blending and now have much lighter textures on the meshes, just need to do the tricky bit of getting the vertex colors blended in for the light mapping. I'm determined to crack this last part and release a useful tool for the community, which as you have commented on before, is one of the best communities out there!

No matter how good your code is, someone will improve on it
jason p sage
17
Years of Service
User Offline
Joined: 10th Jun 2007
Location: Ellington, CT USA
Posted: 23rd Feb 2008 15:26
I Did fix my issue However - The StarWars Game was released- its Awesome - So I'm not convinced yet its only a prototyping tool - though friends around the forums have been saying such things to me - I'm gonna stick it out for awhile before I draw that conclusion. I like more than I dislike.

I CAN'T Wait until you have this one done bro! I might not get to it right away but I'm quite happy that when its time for me to start getting ready for some NICE MEDIA via 3DWS - I can rest assured there is finally a way to get WYSIWYG in the game!

I REALLY REALLY HOPE the author of 3DWS takes a serious look at what you've done - down to the actual DB Calls - as I Bet most of his clients are us TGC addicts - and frankly - if he makes a new version that isn't readily WYSIWYG - I won't bother buying it...Period - I Gave Ted a SERIOUS Go coding - using etc... and it kept falling short of the mark - I Just Want WYSIWYG....

You my friend - KNOW the importance of this - its clearly obvious with the work you've done here!

Pixel Perfect
17
Years of Service
User Offline
Joined: 21st Feb 2007
Location: UK
Posted: 23rd Feb 2008 23:53
Thanks again for your kind comments Jason, although I wouldn't blame Josh too much. I don't think he's an expert in DBPro and as such probably did the only sensible thing he could at the time, which was to work with the published DBO specification and provide an export to that. That produces a reasonable representation but, as you say, What You See in 3DWS isn't quite What You Get in DBPro or DGDK!

Personally, if I had written 3DWS I think I would have paid a good DB programmer to do just what LIT and I have done and included the importer as part of the package, and likewise for a few of the other popular engines. I believe that would have boosted sales considerably through word of mouth, which in these communities stands for a lot! No one wants to invest a large amount of time in a design package only to be disappointed with the final rendition in their engine of choice.

However, I love 3DWS and I have to congratulate Josh on a job well done, hence the reason I'm investing my blood sweat and tears in this project ( hmm ... well slight exaggeration lol )

btw ... I downloaded tobi453's Starwars game tonight and am looking forward to running that. It's not that I don't like Dark Basic or that I don't think people could produce games with it ... its just such hard work without decent documentation and with bugs riddled throughout the code. The recent release of Dark Physics for DGDK with half the commands un-documented and no search facility just makes me think they are going backwards not forwards and frankly I was annoyed, I paid good money for that. If I honestly believed TGC cared it would be a different matter but I no longer believe they do. Their continued silence despite all the criticism speaks volumes.

Anyhow, back to the project .. rant over lol

Success ... Success ... Success I'm very happy this evening because after thinking earlier on today that it might not be possible I have got mesh light mapping and texturing working with modulate x2. There is a slight proviso which I will explain after some screenshots showing off the latest renderings:

Building internal clearly showing the light mapping effects on the mesh models ... the boxes show it quite nicely along with the sunlight on the railing at the end. Now its finally looking as it does in 3DWS



An outside shot showing the lightened terrain (no more DBO export darkness) and the now well lit meshes (power poles)



OK, well that over, there is still a slight problem that I'm grappling with. The 'Industrial Complex' level which I have been using to demonstrate the loader only uses meshes with single limbs, so it’s been easy to deal with the change in the way the texture is applied and blended with the vertex color. However, much to my surprise I discovered that when these single limb x file models are imported using dbLoadObject, DGDK reports three limbs (0,1 & 2) and the dbMakeObjectFromLimb (un-documented) returns valid objects with full vertices for both limb 1 and 2 !!! Exact duplicates of the overall model.

This poses me real problems as in many levels I will have to deal with real multi-limbed models and I cannot tell which are real and which are not. I am going to have to either rely on the limb info contained in the .3dw file and ignore the limb counts in DGDK or find another way of deciding what’s a valid limb that needs converting to an object in DGDK. Anyhow, between LIT and I we have overcome all the other hurdles so I'm sure this one will get solved too.

No matter how good your code is, someone will improve on it
jason p sage
17
Years of Service
User Offline
Joined: 10th Jun 2007
Location: Ellington, CT USA
Posted: 24th Feb 2008 02:23
WOW WOW and WOW - What a Post!

And Josh has made the best EDITOR I've seen for Game Level Design so Far. (Not having tried the GAMESPACE + LEVEL MAKER ADDON Combo)

His Systems IS AWESOME - that's not my issue - you are tackling my Issue Thanks!

Pixel Perfect
17
Years of Service
User Offline
Joined: 21st Feb 2007
Location: UK
Posted: 24th Feb 2008 10:10
Thanks again Jason and my apologies to any one else who might be waiting in the wings for this. I hope you appreciate my efforts to ensure the loader really does what it's designed to do and not release it prematurely. It really won't be too much longer as you can see from the screenshots.

No matter how good your code is, someone will improve on it
Pixel Perfect
17
Years of Service
User Offline
Joined: 21st Feb 2007
Location: UK
Posted: 26th Feb 2008 20:59 Edited at: 26th Feb 2008 21:33
DGDK Loader for 3DWS Update

OK, well using the Mesh, Brush and Terrain Creation Type switches I've built in I am now able to apply modulate x2 texture blending to everything including the meshes which are basically converted from multi-limbed objects to single limbed objects at the input stage.

This has resulted in two things:

1.
The meshes and terrains are now looking bright and full of rich color which when combined with the light mapping is as close as we are ever likely to get to duplicating how it looks in the 3DWS editor (very hard to tell the difference in most cases).

2.
Since first working with the DBO exports from 3DWS, I was aware of a transparency related problem where transparent objects demonstrated a fringe effect where you could see through the next layer to what was actually behind that in the z order. This was more apparent with foliage type objects than with others and could really visually impare the scene at times. This has completely disappeared when converting meshes into single limbed objects which was an unexpected bonus.

I have now reached the stage where I can see no need for any improvement in the loaders functionality so will proceed to tidy up the code and check for memory leaks with the aim of releasing the code as soon as possible.

Below are a couple of my latest screen shots showing an internal shot from the 3DWS 'Industrial Complex' level, followed by one of my own demonstrating modulate x2 blending applied to what were multi-limbed models (the trees):





No matter how good your code is, someone will improve on it
Cellbloc Studios
20
Years of Service
User Offline
Joined: 15th Mar 2004
Location: Atlanta, GA
Posted: 26th Feb 2008 23:11
@Pixel Perfect:

This is, quite, beautiful. Good job!

Your mod has been erased by a signature
FINN MAN
20
Years of Service
User Offline
Joined: 2nd May 2004
Location:
Posted: 27th Feb 2008 03:14
Is there an easy way to get the name associated with each visGroup. I know how to get the integer that the importer uses, but how do I get the name that the integer represents.

Cellbloc Studios
20
Years of Service
User Offline
Joined: 15th Mar 2004
Location: Atlanta, GA
Posted: 27th Feb 2008 04:42
I was thinking that.

It would be nice to have a txt export also of the names, so that a person can read it into their engine and using Sparkys Collision set the collision per object or to set the Dark AI.

Your mod has been erased by a signature
jason p sage
17
Years of Service
User Offline
Joined: 10th Jun 2007
Location: Ellington, CT USA
FINN MAN
20
Years of Service
User Offline
Joined: 2nd May 2004
Location:
Posted: 27th Feb 2008 19:00
What I want to do is have a lua script to be passed the object ID and the VisGroup number from my game engine. Then my lua script would determine, based on the VisGroup number, what DarkPhysics and DarkAI properties to set to the object. But I first need to find out how to get the real names of the VisGroup and not just the integer. Any one know how?

Cellbloc Studios
20
Years of Service
User Offline
Joined: 15th Mar 2004
Location: Atlanta, GA
Posted: 27th Feb 2008 21:13
@FINN MAN:

I think basically we are talking the same thing. I am looking/hoping the exporter can export some information as of the objects (maybe VisGroup number).

I never thought of using lua script, that maybe a good choice.

Your mod has been erased by a signature
Pixel Perfect
17
Years of Service
User Offline
Joined: 21st Feb 2007
Location: UK
Posted: 27th Feb 2008 21:55
Thanks for the comments guys and in answer to your questions ...

Currently visGroups are not fully supported so I'm guessing you want me to add that in. I store the visgroupIDs in the Brush, Mesh and Entity objects but I'm not currently loading the visgroup Object data which you would need to get the visGroup names.

Entities however are fully supported and are stored as follows:

The C3DW_File class has the following public functions:

unsigned int getEntityCount();
This returns the number of entities

C3DW_Entity *getEntity(signed int nIndex);
This returns an Entity object matching the given index value.

So basically iterating from 1 to EntityCount would return all the Entities as objects.

Each C3DW_Entity object has the following public functions:

signed int getGroupIndex();
vector3df getPosition();
signed int getVisGroupIndex();

signed int getKeyCount();
string getName(signed int nIndex);
string getValue(signed int nIndex);

So again, iterating through the keys would get you access to each key name and value/values

If I provide a similar (so be it simpler) interface for the visGroups then likewise you would be able to access all of the data.

I don't really intend taking it any further than that as individuals can construct whatever higher level code they feel is appropriate for their own needs and engine.

Let me know if that sounds ok!

No matter how good your code is, someone will improve on it
Cellbloc Studios
20
Years of Service
User Offline
Joined: 15th Mar 2004
Location: Atlanta, GA
Posted: 28th Feb 2008 03:43
@Pixel Perfect:

I'm ok with that.

What is a "Visgroup" anyways?

Your mod has been erased by a signature
FINN MAN
20
Years of Service
User Offline
Joined: 2nd May 2004
Location:
Posted: 28th Feb 2008 04:59
Pixel Perfect, would this make it able to get the true name associated with VisGroup and not just the integer name used inside the importer? Also I am using dbp.

Pixel Perfect
17
Years of Service
User Offline
Joined: 21st Feb 2007
Location: UK
Posted: 28th Feb 2008 20:18
@FINN MAN
Quote: "would this make it able to get the true name associated with VisGroup and not just the integer name used inside the importer?"


Yes it would

Quote: "Also I am using dbp"


Yes, I'd assumed as much. So this is not going to help you unless you decide to jump to C++ and DarkGDK. However, I will look at LIT's code and see if I can point you at how to extract the actual visGroup Name from the DBPro loader. I'm sure he loads the visGroup object data so this shouldn't be difficult.

@Cellbloc Studios
Quote: "What is a "Visgroup" anyways?"


OK, well a visGroup is a way of grouping Brushes or Meshes (models) in 3DWS so that they can be made visible or invisible on the screen by clicking a checkbox alongside the list of visGroups.
3DWS defaults in some basic visGroups of its own but you can add your own named visGroups too. The properties dialog for any brush or mesh allows you to tie the object to any of the available visGroups.

For example you might create a Vegetation visGroup and assign all your trees and plant models to this. When you are looking at adding or modifying other items in your level design de-selecting the Vegetation visGroup checkbox will remove all of these items from the screen allowing you a less cluttered view of your design.

Of course, visGroups can also be used for other purposes as FINN MAN is planning on, so long as you can access them from your importer. One possible use is a simple portal occlusion type system.

Personally I think the user configurable entity and key system offers much more flexibility over the visGroups for game engine use. Entities (these are non visible objects) can be added representing anything from Player Start Position to locations for effects such as particle effects or simply trigger points, all of which can be embedded into the level design and used by your game engine as you feel fit. Likewise, user defined keys and associated values (one or many) can be assigned to any brushes or models and could be used for example to indicate whether to include the item in a collision system or not. The keys belonging to inbuilt entities such as lights for example are used to pass specific information about that entity to your game engine, such as the light Intensity, Color, InnerConeAngle etc. It's a powerful system that once realised can be used to drive your game engine from the level design itself.

Josh really has written a blinder of a level designer! There is so much flexibility.

Hope this helps.

No matter how good your code is, someone will improve on it
jason p sage
17
Years of Service
User Offline
Joined: 10th Jun 2007
Location: Ellington, CT USA
Posted: 28th Feb 2008 20:30
Let me reiterate so he knows where I stand - Josh did write a Killer Editor (Cartography Shop is awesome to IMHO) - I just hated the TGC marriage - when you buy it HERE at TGC then its not WYSIWYG. Like if I bought it elsewhere - I would expect such complications - but HERE - I was a little piturbed - but YOU FIXED that Pixel Perfect - sounds like we'll see that any day now???!?!? LOL ... No Rush! (HURRY UP!) Lol ...just kidding take your time - I'm not personally even close to ready for it yet!

3DWS is the BEST level/world editor I've seen so far - I know I'm not the all knowing modelling software guru - but I like it - Love the snapin grid stuff - the texture each face ... burn in lights etc.. it rocks. I used Cartography Shop BTW to make the Cockpits for my Heli's and will do so again!

Pixel Perfect
17
Years of Service
User Offline
Joined: 21st Feb 2007
Location: UK
Posted: 29th Feb 2008 00:11
@FINN MAN

Here is a function to return the visGroup Name from LIT's 3DWS loader for DBPro:



Just include that at the bottom of the loader file and call the function from either the Read_Mesh or Read_Brush functions once the visGroupInd value has been loaded passing that value. The following shows the call from the Read_Mesh function:



LIT has already partly done this for you by setting the realvisgroupInd value which actually points directly to the Name_Table$ array position so you could simplify the above function by passing that value. I'll leave that up to you.

No matter how good your code is, someone will improve on it
FINN MAN
20
Years of Service
User Offline
Joined: 2nd May 2004
Location:
Posted: 29th Feb 2008 00:47
Pixel Perfect thanks for that code, I will try it out after I get home from class to night.

Pixel Perfect
17
Years of Service
User Offline
Joined: 21st Feb 2007
Location: UK
Posted: 1st Mar 2008 22:16 Edited at: 1st Mar 2008 23:01
Hmmm .. whilst testing today I discovered that the loader is crashing whilst processing the brush objects if you run the executable outside of the C++2008 Editor.

Looks like a real nasty little bug as it makes no difference if it's in release or debug mode, it runs perfectly in the editor but always crashes at the same point when you run the executable in Windows.

I have managed to track it down to a specific section of the code where it appears that the program flow logic has been disrupted resulting in it trying to read the third element in an array which has zero. The only way I can debug it is by outputing values to a text file which is making progress slow! Really could have done without this!!!

[EDIT]
lol ... kicking myself now! Classic C++ trap to fall into. A class member variable which was involved in the program flow had not been explicitly initialised. In the editor it was seemingly getting set to -1163005939, however when the exe was run in windows it was getting a 0 value which was unfortunately a valid value for switching the program flow.

All working nicely now

No matter how good your code is, someone will improve on it

Login to post a reply

Server time is: 2024-06-26 14:12:33
Your offset time is: 2024-06-26 14:12:33