Sorry your browser is not supported!

You are using an outdated browser that does not support modern web technologies, in order to use this site please update to a new browser.

Browsers supported include Chrome, FireFox, Safari, Opera, Internet Explorer 10+ or Microsoft Edge.

DarkBASIC Professional Discussion / Tutorial to write to DBO format using DBPro and pointers / structures etc [complex]

Author
Message
Bored of the Rings
17
Years of Service
User Offline
Joined: 25th Feb 2005
Location: Middle Earth
Posted: 30th Nov 2022 10:03 Edited at: 14th Dec 2022 22:19
This is a placeholder for my tutorial(s) on writing to DBO format. This will be updated and tutorial part links will be posted here. Watch this space. If you have a fear of pointers and pointers to pointers, please feel free to turn away, or persevere and try the code and understand it. I will break downs of what's going on when and where I can.

Tutorial I - Overview : https://forum.thegamecreators.com/thread/228802#msg2672697
This includes the initial skeleton code for saveDBO.
Pro Programmer / Data Scientist, languages: SAS, C++, SQL, PL-SQL, DBPro, Purebasic, JavaScript, others
Bored of the Rings
17
Years of Service
User Offline
Joined: 25th Feb 2005
Location: Middle Earth
Posted: 6th Dec 2022 21:44 Edited at: 6th Dec 2022 21:44
I continue to write the code for the writing to DBO format. Please continue to watch this space and bear with me. DBPro has a lot of restrictions and workarounds are doable but tricky so please bear with me. This is a lot different to the original code I wrote but can make this more flexible than the orginal C code and will be able to write out to many formats using this language , although a lot easier in C / Purebasic. AppGameKit version is possible but in Tier 2.
First tutorial coming soon. Watch this space.
Pro Programmer / Data Scientist, languages: SAS, C++, SQL, PL-SQL, DBPro, Purebasic, JavaScript, others
Bored of the Rings
17
Years of Service
User Offline
Joined: 25th Feb 2005
Location: Middle Earth
Posted: 10th Dec 2022 09:16 Edited at: 17th Jan 2023 14:07
A few functions so far to get your teeth stuck into e.g. WriteCODE, WriteCR, WriteComma etc. Code to be tidied up a little, there are a lot of functions so please be patient: Tutorial write up to come:

[code removed temporarily]
Pro Programmer / Data Scientist, languages: SAS, C++, SQL, PL-SQL, DBPro, Purebasic, JavaScript, others
Bored of the Rings
17
Years of Service
User Offline
Joined: 25th Feb 2005
Location: Middle Earth
Posted: 10th Dec 2022 11:55 Edited at: 10th Dec 2022 12:47
a little error in my code :
inc pdwSize,dwLen
should be changed to:
inc *pdwSize,dwLen
And all the other ones in the functions.

I will.update properly when I get more time.
Pro Programmer / Data Scientist, languages: SAS, C++, SQL, PL-SQL, DBPro, Purebasic, JavaScript, others
Bored of the Rings
17
Years of Service
User Offline
Joined: 25th Feb 2005
Location: Middle Earth
Posted: 12th Dec 2022 20:33 Edited at: 17th Jan 2023 14:07
added 2 more functions : WriteVector and WriteMatrix.

[code removed temporarily]

As you can see from all of these functions there are 2 possible "modes". You can write out as text or byte data. This is determined by the global variable g_bWriteAsText. If this is set to TRUE, then it will output to text format, otherwise FALSE will output in byte form.

NOTE: We can ignore the TEXT output mode as it's not really going to be used unless yoi want to dump out in visual form for debugging. Note also, pNumberStr has a limit of 256 chars. Code to handle strings has been handled differently in DBPro to that of the C code.

Tutorial to come soon, still writing the internal write functions and adding my own.
Pro Programmer / Data Scientist, languages: SAS, C++, SQL, PL-SQL, DBPro, Purebasic, JavaScript, others
Bored of the Rings
17
Years of Service
User Offline
Joined: 25th Feb 2005
Location: Middle Earth
Posted: 12th Dec 2022 22:01 Edited at: 17th Jan 2023 14:09
Tutorial I:

Welcome to Tutorrial I, how do we save our model data to DBO?

DBO is really just embedded codes e.g. 1, 101, 102, 103 etc written as DWORDs. Along with the size of the data and then the actual data OR a pointer.

There are 3 main functions:

SaveDBO ( LPSTR pFilename, sObject* pObject )
which calls functions DBOConvertObjectToBlock ( pObject, (DWORD*)&pDBOBlock, &dwBlockSize ) and DBOSaveBlockFile ( pFilename, (DWORD)pDBOBlock, dwBlockSize ))

From the C code extract below, the function SaveDBO simply does the following:
1. Is parsed the filename to be saved
2. Is parsed the pointer to where the object data is stored, in our case it will be the array we used in the LoadDBO i.e. dim ppobject(0) as sObject and ppobjectptr=get arrayptr(ppobject()). There will be various sub pointers to the data which we need to access.
3. Converts all the object data held in our array ppObject(0) to DBO Block data i.e. embedded codes, size and data/pointer. The function calls ScanObject which writes 3 main object blocks including header info of 24 bytes at the beginning 1. Root Frame 2. Animation 3. Custom Data
4. Saves the DBO Block to parsed in filename. This uses the Win32 function CREATEFILEA from kernel32.dll. It parses the pointer to the object block and the size of the block
5. safe deletes the object memory by deleting the memory and also setting the pointer variable to NULL or 0 . e.g. delete ( p ); ( p ) = NULL in DBPro this will be free ptr : ptr = NULL or 0 (constant var NULL is setup as 0)
DARKSDK_DLL int SaveDBO ( LPSTR pFilename, sObject* pObject )
{
// DBOBlock ptr
DWORD dwBlockSize = 0;
DWORD* pDBOBlock = NULL;

// convert pObject to DBOBlock
if ( !DBOConvertObjectToBlock ( pObject, (DWORD*)&pDBOBlock, &dwBlockSize ) )
return -2;

// save DBOBlock to file
if ( !DBOSaveBlockFile ( pFilename, (DWORD)pDBOBlock, dwBlockSize ) )
return -3;

// free block when done
SAFE_DELETE(pDBOBlock);

// okay
return 1;
}

Initial Skeleton Code for saveDBO:

[code removed temporarily]




MORE to COME - please bear with me.
Pro Programmer / Data Scientist, languages: SAS, C++, SQL, PL-SQL, DBPro, Purebasic, JavaScript, others
Bored of the Rings
17
Years of Service
User Offline
Joined: 25th Feb 2005
Location: Middle Earth
Posted: 14th Dec 2022 10:37
code corrections / additional functions coming soon.
Pro Programmer / Data Scientist, languages: SAS, C++, SQL, PL-SQL, DBPro, Purebasic, JavaScript, others
Bored of the Rings
17
Years of Service
User Offline
Joined: 25th Feb 2005
Location: Middle Earth
Posted: 14th Dec 2022 13:31 Edited at: 17th Jan 2023 14:09
code corrections and added functions , this is not the full functioning code yet just the main DBO Write functions plus my own additions e.g. MemStrLen which finds / returns the length of a string held in memory:

Functions are subject to updates as I see fit

[code removed temporarily]
Pro Programmer / Data Scientist, languages: SAS, C++, SQL, PL-SQL, DBPro, Purebasic, JavaScript, others
Bored of the Rings
17
Years of Service
User Offline
Joined: 25th Feb 2005
Location: Middle Earth
Posted: 14th Dec 2022 21:52 Edited at: 17th Jan 2023 14:10
All write functions complete, subject to changes for fine tuning:

SAFE_DELETE(ptr) function added.

[code removed temporarily]

Can now start to put together the other main functions saveDBO, DBOConvertObjectToBlock, DBOSaveBlockFile->CreateFile, ScanObject
Pro Programmer / Data Scientist, languages: SAS, C++, SQL, PL-SQL, DBPro, Purebasic, JavaScript, others
Mage
15
Years of Service
User Offline
Joined: 3rd Feb 2007
Location: Canada
Posted: 7th Jan 2023 05:14 Edited at: 7th Jan 2023 05:18
It's been a few weeks, and I am not sure if you are finished with this thread or not.
I see you mention at the end setting up additional functions.
So this might come off as a bit unfair, apologies.

Looking at the code so far:
This is quite detailed and extensive. I'm just going to point out some issues and maybe help things along.

1. I find it hard to follow initially, the code is largely uncommented. So it's not clear what is being done and not clear WHY it's being done.

This is important because if you were to use this you would need to know how to feed vital data into this code (like vertices) and how to make sure it's a valid DBO, like all vertices needing to be part of a polygon or ensuring the vertex count is set correct.


2. I don't see Animation data being written. I could be incorrect but normally that happens at the end of the DBO in it's own section. (I am under the complete belief you know this)

3. There doesn't seem to be any Frames being written. The DBO is organized into structures called Frames, and there are just a huge list of code blocks. There's also this confusing mess of recursion in a DBO. This code does not seem to do any of that. However there could be an exception here or there.

Also: This code makes very heavy use of your own conventions. A lot of clever use of memory peek/pokes and custom data structures. This would clearly condense complicated code, probably runs lightning fast, but it also makes things harder to read. For what it's worth.

I realize that in the DBO file a lot of the time you have to write in a code block's size at the beginning of the code block, and this makes things difficult because usually you only know the size after the block is written. So you sort of have to store the code block in memory as you put it together, then write the size to file, then the actual code bock to file. Then this is taken to insane levels because of recursion. You write your first frame, and every other frame and most the rest of the file can be in that frames code block. Also the same all over again for each frame in the recursion.

It might amuse you to hear about another approach:
What I did to address this issue was to virtualize the writing of the file. Custom wrapper functions for all output just puts all of the bytes into a byte array. So I write the code block's size as 0, it returns the current byte array index. Then I process and write the code block. Then I check where the current byte array index is after that. This lets me calculate the code block size. I can then use that initial byte array index value and just overwrite that 0 code block size I previously wrote. So I an basically just go back and correct the code block sizes. At the end I dump the byte array to file.

Furthermore:
On my own attempt at writing a conversion, I got as far as converting an MS3D model to DBO and back. However no animation was supported and all of the bone angles were zeroed. I also didn't figure out how vertex weights worked in DBO. The File Format Specification is very vague in places. One of the main issues was the university level math needed for converting Euler to Quaternion angles in the specific proper way. MS3D and DBO use different angle systems. The bones all have initial rotations that are not always zero for every model. Then in the animations well there's tons of angles. This massively wasted my will power to continue working on this. I eventually broke the calculations, and did it. I just ran out of gas and set the project aside.

What I did was convert an MS3D model to X format, load it in DBP and then save it as a DBO. Then I was trying to use the documentation to convert the MS3D file to DBO and compare against the DBO I already made. I would check see I was off by a few bytes, check the format specification about a hundred times, then figure out some vague section needs to be 16bytes or there's a block code, or whatever.

Mage's Modular Code - Get awesome UI controls and powerful Bitmap Fonts!
Screen Device Recovery - Stop your apps from crashing when minimized/tabbed/screen locked.
CPU Friendly Frame Limiter - Make your app generate less computer heat + noise, and use less battery life!
Bored of the Rings
17
Years of Service
User Offline
Joined: 25th Feb 2005
Location: Middle Earth
Posted: 8th Jan 2023 09:21
@Mage- thank you for your constructive comments, I will reply in full once I get more time.
Pro Programmer / Data Scientist, languages: SAS, C++, SQL, PL-SQL, DBPro, Purebasic, JavaScript, others
Mage
15
Years of Service
User Offline
Joined: 3rd Feb 2007
Location: Canada
Posted: 9th Jan 2023 06:26
Sure I don't meant to punch holes, but I'm just thinking some interaction or feedback here might be helpful.

Mage's Modular Code - Get awesome UI controls and powerful Bitmap Fonts!
Screen Device Recovery - Stop your apps from crashing when minimized/tabbed/screen locked.
CPU Friendly Frame Limiter - Make your app generate less computer heat + noise, and use less battery life!
Bored of the Rings
17
Years of Service
User Offline
Joined: 25th Feb 2005
Location: Middle Earth
Posted: 9th Jan 2023 10:44
@Mage, no glad for the feedback, I know there a lot of things missings like what the code is doing and comments etc, I will add as soon as I can. I wil ltry to respond as soon as possible Hopefully tons of updates soon and explanation of what's going on. I'll try to explain the TGC recursion code, it's quite complex.
Pro Programmer / Data Scientist, languages: SAS, C++, SQL, PL-SQL, DBPro, Purebasic, JavaScript, others
Bored of the Rings
17
Years of Service
User Offline
Joined: 25th Feb 2005
Location: Middle Earth
Posted: 11th Jan 2023 22:08 Edited at: 12th Jan 2023 07:27
Quote: "
1. I find it hard to follow initially, the code is largely uncommented. So it's not clear what is being done and not clear WHY it's being done.
"


Code comments will be added later, detailing WHY it's being done.

Quote: "
2. I don't see Animation data being written. I could be incorrect but normally that happens at the end of the DBO in it's own section. (I am under the complete belief you know this)
"


There is still a lot of code to be added, so yes, I know this.

Quote: "
3. There doesn't seem to be any Frames being written. The DBO is organized into structures called Frames, and there are just a huge list of code blocks. There's also this confusing mess of recursion in a DBO. This code does not seem to do any of that. However there could be an exception here or there.
"


Yes, the TGC recursion code is confusing. There is still a lot to add and I will explain the recursion process. I have, since uploading the initial code, made a lot more updates that progress further into the DBO structure. Each frame can have either a child or sibling frame , the pointer to these frames are stored as pointers. This helps when rebulding the object "skeleton" hierarchy. The code to do this as you can imagine uses recursion.

Tons more to do, and spare time is pretty much none existent at the moment due to home life and work. Please bear with me on providing further updates to the DBO Load and Save process. I may upload my DBOFastReader code , which reads in DBO and saves out to .X / .OBJ formats (.X fully textured / animated). The code shows how to recreate skeleton bone hierarchal structure. I haven't released it as a tutorial as it reads DBO but doesn't write to DBO, so the idea was to use the TGC C code and see if it could be converted to DBPro equivalent code.

To give you a bit of insight on how the DBOFastReader constructs the Frame (Skeleton) / limb hierarchy, here is an extract of the frame post process code:

Pro Programmer / Data Scientist, languages: SAS, C++, SQL, PL-SQL, DBPro, Purebasic, JavaScript, others

Login to post a reply

Server time is: 2023-02-02 14:02:28
Your offset time is: 2023-02-02 14:02:28