Sorry your browser is not supported!

You are using an outdated browser that does not support modern web technologies, in order to use this site please update to a new browser.

Browsers supported include Chrome, FireFox, Safari, Opera, Internet Explorer 10+ or Microsoft Edge.

AppGameKit Classic Chat / Howto? Structured Types from Files to Memory?

Author
Message
Xaby
FPSC Reloaded TGC Backer
17
Years of Service
User Offline
Joined: 17th Apr 2007
Location: Berlin
Posted: 23rd Mar 2017 15:41
Hello AppGameKit users,

I want to decode and later maybe encode a complex file format. I have C++ code and what I understand is, that it is possible to Read from a File into a Structured Datatype in one command.

e.g.: fread(name, 1, 0x60, fp);

fp is the File. Name is the first element of the before declared "type" and it will read 96 bytes. ! But I don't know, how I could do that trick in AGK2 Basic.

e.g.

Type MyType
B as Byte // oh, we have no Byte, so as Integer, but Integer is to long
Text as String // first problem, can't say, that my String contains only e.g. 28 Characters
...
EndType

TyVariable as MyType // for better understanding in later text

If I would read MyType from a position in a File, all Variables would be "filled". At this time, I have to use GetMemblockShort( memID, offset ) and some other commands and set the results into my TyVariable. So I am not be able to define my Type once, but I also have to set all my Variables from MyType in TyVariable by "Hand".

I read here https://www.thegamecreators.com/post/appgamekit-development-road-map that Load & Save Arrays: Extra commands to quickly save out and load in array data. will be in a future update.
Maybe that will solve my request.

So maybe you AppGameKit users have a better work arround or something to make my live easier

Thanks for help. Kind regards. Xaby (formaly known as f4ktor)






janbo
15
Years of Service
User Offline
Joined: 10th Nov 2008
Location: Germany
Posted: 23rd Mar 2017 16:58 Edited at: 23rd Mar 2017 17:01
Hey

First you could overcome your byte integer problem like so:

It also have the advantage of reading 4 bytes at the same time

Second I don't fully understand what you are trying
maybe you are looking for SetFilePos( iFileID, pos ) ?

Using AGKv2 Tier1
Dybing
12
Years of Service
User Offline
Joined: 12th Sep 2011
Location: Bergen, Norway
Posted: 23rd Mar 2017 18:59 Edited at: 23rd Mar 2017 19:19
The easy way is to use 'stringly' typing for export-import and read to and from a .csv (comma separated values) file. It do not have to be commas used as separators - any character will do just as long as you know it'll not also crop up as part of a variables value.

So if I were to for instance exchange data between a program written in say Go (since that is what I'm most familiar with) and AppGameKit I'd do this:

1: Have a save-file routine, converting all values of the original datastructure into strings - this particular routine adds a user to an existing file with three fields, originally one string and two bools, but through conversion it's only strings.



2: To read from AppGameKit you just reverse the process - keeping in mind that there are no bools in AppGameKit so that'll have to get to be integers:



(included a little print-to-screen routine just to validate it all went honky-dory

All you need is a routine to change stringly-byte to integers and good to go using much the same process.

edit: oh and the test file "accGet.csv" in the media folder looks like this:

Phaelax
DBPro Master
21
Years of Service
User Offline
Joined: 16th Apr 2003
Location: Metropia
Posted: 24th Mar 2017 00:30
Load your file into a memblock, makes things easier.

"I like offending people, because I think people who get offended should be offended." - Linus Torvalds
Carharttguy
7
Years of Service
User Offline
Joined: 8th Jul 2016
Location: Belgium
Posted: 28th Mar 2017 09:07
I don't my English is good enough to fully understand your question, but a small answer (maybe)

Type MyType
B as Byte // oh, we have no Byte, so as Integer, but Integer is to long
Text as String // first problem, can't say, that my String contains only e.g. 28 Characters
...
EndType

Use String

Just store your file in a MemoryBlock and use the GetMemblockByte() and GetMemblockString() methods.
There is no problem with using a Integer instead of a byte. A Byte is a value that holds a value ranging 0-255, Integer can hold that. Only downside is the memory usage. Integer is 4 bytes, a Byte is .. 1 byte.
So if you're reading huge files, this might become a problem.

About the string: You say you don't know how long the string will be, but how does the C++ code handles this? There must be a delimiter in some way to know where to stop reading. Or maybe a Byte before the string that determines the length of the string that follows?
No computerlanguage could possibly know "the end of a string". As that concept doens't exist in binary data (or every other file format)

Maybe you could specify what file format you're trying to read?
Xaby
FPSC Reloaded TGC Backer
17
Years of Service
User Offline
Joined: 17th Apr 2007
Location: Berlin
Posted: 28th Mar 2017 10:59
@Carharttguy

How can I limit STRINGs in AGK2 Tier 1 Basic? I know how long the String in the File would be.

The Structure is
28 Bytes interpreted as Chars for a 28-long-String
after that, some Bytes and Unsingned Shorts (in Pascal it's a Word).

I am using MemBlocks allready. I put the whole file-stream into a MemBlock, after that, I look on some positions in the MemBlock for my Data and set my Variables in the Structure (Type).
But!, that is a complicated way.

Because I have first to define the Structure and after that, I have to assign every part of the MemBlock to the Structure. I can't limit the AGK-Integers to Bytes and I can't limit AGK-Strings to a specific lenght. So I can't use a one line-read from a File or MemBlock to my Structure.

I think, there is no shortcut, I have to do it the long way. In C is the limiter written like that "char name[28];" The String is "name" and it is more like an array of Chars. In Pascal it was possible to Limit a String[XY] also.



Carharttguy
7
Years of Service
User Offline
Joined: 8th Jul 2016
Location: Belgium
Posted: 28th Mar 2017 11:07
I must have been unclear.

You can indeed not limit a string. In most languages you can't.
C uses a array of char. It could be possible in Pascal.

It must be my bad English, but I still don't understand what advantage it would be to limit an Int to a Byte or a String length.

Quote: "So I can't use a one line-read from a File or MemBlock to my Structure."

Why not?

Could you show a snippet of the file to read, and describe why it would make coding easier with a Byte type and a String with a fixed length?
Xaby
FPSC Reloaded TGC Backer
17
Years of Service
User Offline
Joined: 17th Apr 2007
Location: Berlin
Posted: 28th Mar 2017 14:13 Edited at: 28th Mar 2017 14:24
Structure in C

/* The S3M file header */
char name[28]; // song name
unsigned char eofchar, typ, dummy[2];
unsigned short ordnum, insnum, patnum, flags, cwtv, ffi;
char scrm[4];
unsigned char Vxx, Axx, Txx, mastervolume, uc,dp,dummy2[8];
unsigned short special;
unsigned char channelsettings[32];
-------------------------------------------------------------
fread(name, 1, 0x60, fp); // Read S3M file header

With "name" being limited to 28 Bytes, the other bytes (96-28=68) will be also read into memory. And because of the defined Structure, all following bytes in the File will stored into the memory, but also at the right variables.

There is no AGK2-Tier-1-Command, which could do MyTypeFromMemBlock() or something.
So in the File there are the first 28 Characters for a String. And after that, there are some other Variables. But without having "Byte" or a limited String, I have to use

integer GetMemblockByte( memID, offset )
integer GetMemblockByteSigned( memID, offset )
integer GetMemblockShort( memID, offset ) ... and integer GetMemblockUnsignedShort( memID, offset ) (Had to write it myself)
string GetMemblockString( memID, offset, length )



I can't say: MyType = GetMemblockByType(memID, offset, Type)

Type MyType
i as Integer
b as Byte
... a as Byte[10] // array of 10 Bytes for later M.a[i]=...
EndType

M as MyType

M = GetMemBlockByType(0,0,MyType) // this is one line

But I have to do: M.i = GetMemBlock... M.b=GetMemBlock ... and so on, for every Variable in MyType. And with Arrays of something, I would have to do Loops.

I you wish, you can look here. I am trying to "convert" that code. There is also a PDF linked at this video description


How Strings work in Pascal http://wiki.freepascal.org/String
The next problem is, that I don't know, if the Command reads Characters or Bytes from the MemBlock in AGK2-Tier-1-Basic. A Character can be 1 or 2 Bytes. Because of ANSI / ASCII, UTF8, Unicode. So what would happend, if I had a bunch of Bytes which are UTF8? I don't have to save the lenght of a String in my File. I could do so, but I don't have to and a lot of the old DOS/Amiga file Formats are not so dynamic.
Carharttguy
7
Years of Service
User Offline
Joined: 8th Jul 2016
Location: Belgium
Posted: 28th Mar 2017 14:45
Aha

Thanks for the explanation, now I understand. It was clearly my lack of C skills that kept me from understanding. I was unaware that you could fread() directly to a struct.
So, now to be a bummer. No that is indeed not possible.
It would be nice if we could do some kind of introspection on types to make this possible. But for now, impossible.
The only way to do this is as you told:
M.i = GetMemblock..
M.b = GetMemblock..

In defense of AppGameKit, there are very few languages that have this paradigm.
Most languages will do this in the constructor of a class or something. And then you also have to do the reading manually.

Sorry I can't help you more. And thank you for learning me the fread() to struct in C.
Mobiius
Valued Member
21
Years of Service
User Offline
Joined: 27th Feb 2003
Location: The Cold North
Posted: 28th Mar 2017 14:58
Oooo, amiga mod/screamtracker files.
Oldschool!
Signature removed by mod because it's larger than 600x120... please resize and try again.
Xaby
FPSC Reloaded TGC Backer
17
Years of Service
User Offline
Joined: 17th Apr 2007
Location: Berlin
Posted: 28th Mar 2017 15:03
@Mobiius I can't make any promise yet, but I want to have MOD, S3M and XM nativly without DLL in AGK2-Tier1-Basic. Maybe for one or two badges in return
Phaelax
DBPro Master
21
Years of Service
User Offline
Joined: 16th Apr 2003
Location: Metropia
Posted: 29th Mar 2017 02:16
Quote: "The next problem is, that I don't know, if the Command reads Characters or Bytes from the MemBlock in AGK2-Tier-1-Basic. A Character can be 1 or 2 Byte"


It reads bytes. My Base64 library I just rewrote in AGK2 relies entirely on doing all string manipulation with memblock bytes and not string characters (for speed purposes). You write a string to a memblock, it will write each character as a byte value. It will also null terminate the string when writing it to the memblock. I would suggest storing the size of the string when you save it so you know how many bytes to read to retrieve the string. If not, you could technically search byte by byte for a byte 0, then you'll know where the end of the string is.

Quote: "B as Byte // oh, we have no Byte, so as Integer, but Integer is to long"

Why is integer too long?

Quote: "it is possible to Read from a File into a Structured Datatype in one command."

No, but you can read the file straight into a memblock using createMemblockFromFile(). Then loop through the data to input it into your UDT.


I get that you're trying to read data from a file into a custom data structure, what I can't understand is what the trouble is. Or what purpose you'd have for limiting the size of a string. Simply only read as much of the string as you want to store. Is it the S3M structure you're trying to read or just an example?

"I like offending people, because I think people who get offended should be offended." - Linus Torvalds
Phaelax
DBPro Master
21
Years of Service
User Offline
Joined: 16th Apr 2003
Location: Metropia
Posted: 29th Mar 2017 02:56
Based on the S3M format I found here, http://www.shikadi.net/moddingwiki/S3M_Format.

S3M file header read into a UDT (struct).

"I like offending people, because I think people who get offended should be offended." - Linus Torvalds
Xaby
FPSC Reloaded TGC Backer
17
Years of Service
User Offline
Joined: 17th Apr 2007
Location: Berlin
Posted: 3rd Apr 2017 11:57
@Phaelax and the rest of you, thanks for your posts,

maybe we could make a community project like the "Game Templates" for AGK. Where we Update or show some File-Types-Loading.

JPEG, PNG, OGG, OBJ, X, ... and some others are native integrated.
Some formats are not fully supported like Spine 3.5/3.6 Format. Or music-modules. "Level-Files", etc.

This could be a great example library, maybe with some tutorials also.

Thanks for help.

Login to post a reply

Server time is: 2024-04-19 16:30:02
Your offset time is: 2024-04-19 16:30:02