Sorry your browser is not supported!

You are using an outdated browser that does not support modern web technologies, in order to use this site please update to a new browser.

Browsers supported include Chrome, FireFox, Safari, Opera, Internet Explorer 10+ or Microsoft Edge.

DarkBASIC Professional Discussion / Tutorial to read in DBO format using DBPro and exporting to a format

Author
Message
Bored of the Rings
17
Years of Service
User Offline
Joined: 25th Feb 2005
Location: Middle Earth
Posted: 19th Nov 2022 16:11 Edited at: 25th Nov 2022 07:48
I'll be doing a tutorial here on how it's possible to read in DBO and convert to a format. Will be concentrating on obj, x and some other formats.
In additon, I will show how to write out to DBO.

This is a complex task, so you will need to.be able to use pointers and structures and even arrays and pointer to pointers as well as use the Win Api.
Furthers posts will be placed here.

All tutorial Part links below:

Part I - Setting up some initial variables and loading the DBO file into memory (pointer).
https://forum.thegamecreators.com/thread/228776#msg2672403

Part II - Adding the sObject structures and related structures, DX and other constants and reading in DWORDs/Embedded DBO codes + other functions required to find specific structure item pointers.:
https://forum.thegamecreators.com/thread/228776#msg2672426

Part III - Initial Construction of an Object and Object Frame
https://forum.thegamecreators.com/thread/228776#msg2672464
Pro Programmer / Data Scientist, languages: SAS, C++, SQL, PL-SQL, DBPro, Purebasic, JavaScript, others
LBFN
15
Years of Service
User Offline
Joined: 7th Apr 2007
Location: USA
Posted: 19th Nov 2022 21:08
Sounds like an interesting tutorial. I wonder, is it possible to convert .dbo to .ms3d format and still retain the animation / joint info?



So many games to code.....so little time.
Bored of the Rings
17
Years of Service
User Offline
Joined: 25th Feb 2005
Location: Middle Earth
Posted: 19th Nov 2022 21:25
yes definitely
Pro Programmer / Data Scientist, languages: SAS, C++, SQL, PL-SQL, DBPro, Purebasic, JavaScript, others
Bored of the Rings
17
Years of Service
User Offline
Joined: 25th Feb 2005
Location: Middle Earth
Posted: 20th Nov 2022 08:53 Edited at: 21st Nov 2022 10:00
I originally wrote a program DBOFastReader which reads in DBO and exports static/animated objects out to DirectX and OBJ formats using DBPro commands and IanM Matrix1Utils.
This program / tutorial is more nearer to a VS C version of DBOLoader but with DBPro workarounds due to restrictions/limitations DBPro has.

DBO Format documentation can be found in your DBPro install folder under "DBO". In there you should be able to see a file DBOFormat.doc. This spec document, gives you a basic overview of how the structure works and how it can be loaded in. Under the same folder, you have the original DBO VS source code.

For the tutorials you will need Matrix1Utils written by IanM which allows us to use ALLOC and various Array Pointer commands, plus loads more.
NOTE: I have not used #include as when compiling with DBPRO and hitting errors does not show true line number , so once the program is complete / tested and works like a charm, then we can start adding include files to organise or our code better.

Part I - Setting up some initial variables and loading the DBO file into memory (pointer).

Initial Code:

As I get a bit more time on my hands, more and more breakdown info on what the code is doing will be added below the code snippet.



How does DBO work?
DBO works by embedding special codes that are represented in DWORD (LONG) format i.e. 4 bytes e.g. 1,0,0,0 = 1, 101,0,0,0 = 101, anything over 255 will carry over into the next byte.

How does the code work?
This 1st part of the code works by setting up some initial global variables, i.e. pointers to the DBO data and embedded codes and size. This is because DBPro does not understand "&" symbol used in VS C.
In DBPro we can allocate memory using memblocks, banks and ALLOC ( the last 2 you need IanM Matrix1 utils plugin).

Constant variables are initialised including a complete list of embedded DBO code 1 - 406 and some WIN32 API constants needed for the CreateFileA function. This gives us more control on the way a file is read in and is very fast. Originally , I used MAKE BANK FROM FILE and then obtained pointer from there.

To use the "CreateFileA" and "ReadFile" functions, we have to load in the "kernel32.dll" dll. Then obtain a handle to the CreateFileA function. Pointer variables are parsed into the ReadFile function.
dppBlock is used to allocate memory with size of the file obtained from *dpdwSize (i.e. return the size value contained in pointer dpdwSize). Print dpdwSize and Print *dpdwSize are 2 different things. The 1st returns the memory address and the 2nd returns the actual value stored in that address.

ppBlock is used as a "floating" address pointer to obtain the next DBO address, but is not set up as a pointer. It's purely a copy of the base pointer pDBOBlock which is static meaning the value will never change.

Anything that was setup as allocated memory is deallocated using FREE command at the end of the program (CleanUp function). I've added a message statement (for those that have Robert Knights BLUEGUI plugin) just to ensure that the cleanup works correctly without prematurely "bombing" out the program. For the ppBlock variable, it simply gets "NULLED" or set to zero.

In the future tutorial sections, there will be references to LPSTR, LPVOID and other LPxxx etc type vars , these are simply windows structures that really can be translated as DWORDs as they are all pointers to a particular windows data type.
Don't forget that all strings in DBPro are essentially DWORDs containing the address that points to the actual text of a string.

So if you have a fear of pointers and pointer to pointer, turn away now
Pro Programmer / Data Scientist, languages: SAS, C++, SQL, PL-SQL, DBPro, Purebasic, JavaScript, others
LBFN
15
Years of Service
User Offline
Joined: 7th Apr 2007
Location: USA
Posted: 21st Nov 2022 13:14
Had couple of errors. DBP didn't like the use of NULL as a constant and had trouble with the EXTRACT FILENAME$ and EXTRACT FILEEXT$ commands in the FileShortName function. I changed NULL to NOLL and remmed out the other two and it ran fine.



So many games to code.....so little time.

Attachments

Login to view attachments
Bored of the Rings
17
Years of Service
User Offline
Joined: 25th Feb 2005
Location: Middle Earth
Posted: 21st Nov 2022 13:32
@LBFN - thanks for taking the time to try the code. You need IanM Matrix1 plugin to use the EXTRACT FILENAME$ and EXTRACT FILEEXT$ commands, or I have my own version functions for those if you need them. It's weird that #constant NULL 0 doesn't work for you. Not sure why. Of course, change as you need to , as long as it works for you.
Pro Programmer / Data Scientist, languages: SAS, C++, SQL, PL-SQL, DBPro, Purebasic, JavaScript, others
LBFN
15
Years of Service
User Offline
Joined: 7th Apr 2007
Location: USA
Posted: 21st Nov 2022 15:00
Quote: "You need IanM Matrix1 plugin to use the EXTRACT FILENAME$ and EXTRACT FILEEXT$ commands"

I have the Matrix1 plugin. I have it in the plugins-user folder. I see 33 different .dll's with it. I tried moving it to the plugins-licensed folder, but it still doesn't like the extract filename / fileext commands.



So many games to code.....so little time.
Bored of the Rings
17
Years of Service
User Offline
Joined: 25th Feb 2005
Location: Middle Earth
Posted: 21st Nov 2022 15:12 Edited at: 21st Nov 2022 15:16
@LBFN - the Matrix1util plugin dlls should all go in plugins-user. Also, make sure you have c++ restribs for VS2010 and that dotnet 3.5 is installed under windows features

Pro Programmer / Data Scientist, languages: SAS, C++, SQL, PL-SQL, DBPro, Purebasic, JavaScript, others

Attachments

Login to view attachments
Bored of the Rings
17
Years of Service
User Offline
Joined: 25th Feb 2005
Location: Middle Earth
Posted: 22nd Nov 2022 09:44 Edited at: 25th Nov 2022 18:23
Part II - Adding the sObject structures and related structures, DX and other constants and reading in DWORDs/Embedded DBO codes + other functions required to find specific structure item pointers. This is complex.



What's Been Added?

1. Initialisation of Windows related Constants
2. DirectX related Structures
3. Added DBO related structs (only sObject structure for now)
4. Added Functions: ConstructObject, ReadString, ReadDWORD, ReadCODE, Stricmp (compare string), memcpy, GetFieldType$, GetStructItemCount, GetStructPtrs(ptr, FindStructItem

What do the functions do?

ConstructObject
parses in the ppObject pointer and current ppBlock pointer (incrementing or floating pointer to the next address or code if you like). Reads in header info (the first 24 bytes) and then sets up an array of sObject type that will hold our object data. without that you can't save out a dbo structure.

ReadDWORD
As it suggests , reads in a DWORD value at ppBlock address and stores it in ppDest which points back to dwVersion etc which are set up as ALLOC'd memory pointers of size 4 i.e. a DWORD to hold an address. It then increments the ppBlock pointer by 4 bytes

ReadCODE
As it suggests , reads in an embedded DBO code and the Code Size. It then increments the ppBlock pointer by 4 bytes for both dwCode and dwCodeSize. For all possible embedded codes see the DBO related #contants 1-406

Stricmp
Compares 2 strings and returns 0 if they match. A sort of quick version of the C function stricmp.

memcpy
Copies a number of bytes (dwLength) into a pointer (which has previously been ALLOC'd).

GetFieldType$
Returns an array item structure as text e.g. DWORD = D, String = S etc. Called within my GetStructPtrs function. See code comments for full list

GetStructItemCount
Returns the number of items (or fields) within an array

GetStructPtrs
Reads all array items (fields) within an array UDT by parsing in an array pointer which is linked to a temporary array of same structure type and details about the item are stored in a pointer list e.g. ptrlist() as ptrlist_t

FindStructItem
Returns an address of an array item by parsing in the address pointer variable and number of the item. I've added the item number in the structures using []. This is because we have no way of using ppObject->pFrame type methods.
Pro Programmer / Data Scientist, languages: SAS, C++, SQL, PL-SQL, DBPro, Purebasic, JavaScript, others
Bored of the Rings
17
Years of Service
User Offline
Joined: 25th Feb 2005
Location: Middle Earth
Posted: 22nd Nov 2022 09:50
all tutorial Part links will be posted to 1st post to keep it all together in one place for convenience and so you don't get "lost down a rabbit hole"
Pro Programmer / Data Scientist, languages: SAS, C++, SQL, PL-SQL, DBPro, Purebasic, JavaScript, others
Mage
15
Years of Service
User Offline
Joined: 3rd Feb 2007
Location: Canada
Posted: 24th Nov 2022 23:21
Great work so far, eagerly awaiting any info on writing to DBO.

Mage's Modular Code - Get awesome UI controls and powerful Bitmap Fonts!
Screen Device Recovery - Stop your apps from crashing when minimized/tabbed/screen locked.
CPU Friendly Frame Limiter - Make your app generate less computer heat + noise, and use less battery life!
Bored of the Rings
17
Years of Service
User Offline
Joined: 25th Feb 2005
Location: Middle Earth
Posted: 25th Nov 2022 05:17 Edited at: 25th Nov 2022 07:49
@Mage - many thanks. - I will come onto saving to DBO , the main functions will be:

SaveDBO ->
->DBOConvertObjectToBlock ->
-> ScanObject
->DBOSaveBlockFile

The good thing is we have the size of the DBO already and so we would create a block of memory of size pdwBlockSize , I would be using ALLOC although you can use what you like in your own programs whether it be memblocks or banks.

We need a few WriteXX functions e.g. WriteCR, WriteSpeechMark, WriteComma. These are pretty simple.

Top of my memory, we would write the following:

1. Write header info - 24 bytes
2. Write the root frame and size of the root block
3. Write each embedded code along with size and then the data i.e. dwCode, dwCodeSize, pData - pData could be Frame Name or Translation Matrix etc
4. Write the Animationset Root and size of the block ,and then the data ( if Animation exists)
5. Write any custom data dwCode = 406 and size, then the data in bytes (if custom data exists)

So there are 3 main blocks, Root, Animation and Custom Data.

I will get onto all this as soon as I can.
Pro Programmer / Data Scientist, languages: SAS, C++, SQL, PL-SQL, DBPro, Purebasic, JavaScript, others
Bored of the Rings
17
Years of Service
User Offline
Joined: 25th Feb 2005
Location: Middle Earth
Posted: 25th Nov 2022 05:40 Edited at: 27th Nov 2022 19:07
Part III - Initial Construction of an Object and Object Frame

Updated Code (fixes)


Unfortunately, I lost the text I placed here. Will rewrite when I get more time.
Pro Programmer / Data Scientist, languages: SAS, C++, SQL, PL-SQL, DBPro, Purebasic, JavaScript, others
Bored of the Rings
17
Years of Service
User Offline
Joined: 25th Feb 2005
Location: Middle Earth
Posted: 27th Nov 2022 07:42
Tutorial 3 needs a little update. For now i have just commented out the call to ReadString within the ConstructFrame function. Tons to come, for those that want to write to DBO i will start to make a separate tutorial so it doesn't gets mixed up with this one, will see how it all works out. It's just basically getting the info stored in Object structures and converting to DBO , poking lots of bytes etc.
Pro Programmer / Data Scientist, languages: SAS, C++, SQL, PL-SQL, DBPro, Purebasic, JavaScript, others
Bored of the Rings
17
Years of Service
User Offline
Joined: 25th Feb 2005
Location: Middle Earth
Posted: 30th Nov 2022 06:10 Edited at: 30th Nov 2022 06:10
More updates to come for tutorial 3 as we need to be able to expand the frame pointer list with each "array insert ...." and return/access each structure item value i.e. make the array dynamic. So the object is to e.g. find a DBO frame, create a frame object, copy the frame contents/data to the frame array structure. This will allow us to access the content when exporting to a required format or save back out in DBO format if we wanted to.
Pro Programmer / Data Scientist, languages: SAS, C++, SQL, PL-SQL, DBPro, Purebasic, JavaScript, others

Login to post a reply

Server time is: 2022-12-01 03:32:19
Your offset time is: 2022-12-01 03:32:19