Sorry your browser is not supported!

You are using an outdated browser that does not support modern web technologies, in order to use this site please update to a new browser.

Browsers supported include Chrome, FireFox, Safari, Opera, Internet Explorer 10+ or Microsoft Edge.

AppGameKit Classic Chat / Skyroads level file format

Author
Message
blink0k
Moderator
11
Years of Service
User Offline
Joined: 22nd Feb 2013
Location: the land of oz
Posted: 10th Mar 2019 20:40
I am trying to decipher the layout of the skyroads level file (roads.lzs)
I have of course fallen over at the very first hurdle.

The wiki says
"The file starts with a header as follows:"
UINT16LE offset Offset of this level (in bytes since start of file)

When i look in the file the first vale is 0x7C00

When i assume the first level should begin at offset 0 am i wrong?

It's freeware so i'm assuming it's ok to post the roas.lzs file. If not let me know and i'll take it down

Here's a link to the game anyway

Any help is greatly appreciated
Thanks

Attachments

Login to view attachments
puzzler2018
User Banned
Posted: 10th Mar 2019 20:45
Im happy to see that your creating something for the GameJam

Hope this helps further

http://www.shikadi.net/moddingwiki/SkyRoads
Tier 1 Developer
blink0k
Moderator
11
Years of Service
User Offline
Joined: 22nd Feb 2013
Location: the land of oz
Posted: 10th Mar 2019 21:43 Edited at: 10th Mar 2019 21:55
Unfortunately that doesn't help me because i'm trying to parse the the level information so i can recreate the levels in my game.
So i need to understand the layout of the roads.lzs file

Understanding it a bit more;
"This structure is repeated until the offset of the first level is reached. The length field indicates how much space is required to store the level data once it has been decompressed. "

So it's a table of level offsets, until you reach this offset of the first level
puzzler2018
User Banned
Posted: 10th Mar 2019 21:46 Edited at: 10th Mar 2019 21:47
It really is all there

http://www.shikadi.net/moddingwiki/SkyRoads_level_format

and the road compression and decompression

http://www.shikadi.net/moddingwiki/SkyRoads_compression

Ouch!!! good luck trying to figure that format out

Tier 1 Developer
Bengismo
7
Years of Service
User Offline
Joined: 20th Nov 2017
Location: Yorkshire, England
Posted: 10th Mar 2019 21:47 Edited at: 10th Mar 2019 21:49
The 4 byte headers start at offset 0 so the level data cant be there. the first level will follow the headers

That first value says that the first level information starts 0x7C into the file (124 bytes into the file)

Since each header is 4 bytes that means there has to be 124/4 levels stored in the file = 31.....That would imply 31 levels are stored. The wiki says that there are 30 levels though so i might be wrong. unless its 30 levels plus the demo??


So basically there are 31 headers followed by level 0, level1, level 2 etc....
Bengismo
7
Years of Service
User Offline
Joined: 20th Nov 2017
Location: Yorkshire, England
Posted: 10th Mar 2019 22:03
Those post edit times tell a story ...
blink0k
Moderator
11
Years of Service
User Offline
Joined: 22nd Feb 2013
Location: the land of oz
Posted: 10th Mar 2019 22:06
Thanks bengismo. That is very useful.

One question; It appears the integers are byte reversed. How can i swap the bytes in an integer
puzzler2018
User Banned
Posted: 10th Mar 2019 22:06
Wow - we have just discovered the code for a compression / decompression

https://en.wikipedia.org/wiki/Lempel%E2%80%93Ziv%E2%80%93Stac
Tier 1 Developer
puzzler2018
User Banned
Posted: 10th Mar 2019 22:07
Thanks too. Hope to see you interpret that roads file for the GameJam
Tier 1 Developer
Bengismo
7
Years of Service
User Offline
Joined: 20th Nov 2017
Location: Yorkshire, England
Posted: 10th Mar 2019 22:11
The file format states that the data is little endian.....which means the lower byte is stored first.

So to load a UINTLE16 correctly you should:

Value = Readbyte(file) + Readbyte(file)*256

or

Value = Readbyte(file) + Readbyte(file)<<8
blink0k
Moderator
11
Years of Service
User Offline
Joined: 22nd Feb 2013
Location: the land of oz
Posted: 10th Mar 2019 22:12
Fantastic! Thank you bengismo
puzzler2018
User Banned
Posted: 10th Mar 2019 22:13
Like I say - good luck and hope to see SkyRoads by using the Roads file
Tier 1 Developer
blink0k
Moderator
11
Years of Service
User Offline
Joined: 22nd Feb 2013
Location: the land of oz
Posted: 11th Mar 2019 00:51 Edited at: 11th Mar 2019 22:03
Palette converted to 24 bit seems to look good (Just eyeballing it from the youtube video so not 100% sure)



Always thanks for your help

Attachments

Login to view attachments
blink0k
Moderator
11
Years of Service
User Offline
Joined: 22nd Feb 2013
Location: the land of oz
Posted: 11th Mar 2019 22:13 Edited at: 11th Mar 2019 22:15
Having some issues with decompression;
The wiki code reads (I find the algorithm easier to understand)




My code


Skyroads .lzs file format

My code gets a length of 2257 when it should be 2240.
If someone could have a look at it and maybe point me in the right direction i would appreciate it.
Thanks in advance

I have attached a project containing the roads.lzs file

Attachments

Login to view attachments
puzzler2018
User Banned
Posted: 11th Mar 2019 22:33
Thanks

Im sure its a bear with us for a bit whilst we decompose the pseudo comparing to your code - Thanks for the roads file too - thats helps
Tier 1 Developer
puzzler2018
User Banned
Posted: 11th Mar 2019 22:38
Where is your else


else copy 8 bits from input to index, add 1 to index
Tier 1 Developer
blink0k
Moderator
11
Years of Service
User Offline
Joined: 22nd Feb 2013
Location: the land of oz
Posted: 11th Mar 2019 22:44 Edited at: 11th Mar 2019 22:45
while len(output) < length
v = ReadBits(file, 1)

select v
case 0
dist = ReadBits(file, width2) + 2
count = ReadBits(file, width1) + 2
output = output + mid(output, len(output) - dist, count)
endcase
case 1
v = ReadBits(file, 1)
select v
case 0
dist = ReadBits(file, width3) + 2 + mult
count = ReadBits(file, width1) + 2
output = output + mid(output, len(output) - dist, count)
endcase
case 1
output = output + chr(ReadBits(file, 8))
endcase
endselect
endcase
endselect
endwhile


Maybe if you downloaded the project and tried it. Then you could fiddle with it and maybe find a solution
puzzler2018
User Banned
Posted: 11th Mar 2019 22:48
Its not within case 1 - its within the else of the first IF- maybe case default
Tier 1 Developer
puzzler2018
User Banned
Posted: 11th Mar 2019 22:55
That Psedo is mind boggling

Its calling "if getbits(1) ==0 twice in one if / else statement - is that strange - its never going to enter the 2nd elseif cause always going to hit that first IF cause its the same statement..





My mind boggles
Tier 1 Developer
blink0k
Moderator
11
Years of Service
User Offline
Joined: 22nd Feb 2013
Location: the land of oz
Posted: 11th Mar 2019 22:58
Quote: "Its not within case 1 - its within the else of the first IF- maybe case default"

The else of the if is when the bit value is = 1 so it's dealt with in the case 1 code
Quote: "Its calling "if getbits(1) ==0 twice in one if / else statement"

GetBits() gets the NEXT bit
puzzler2018
User Banned
Posted: 11th Mar 2019 23:00 Edited at: 11th Mar 2019 23:01
Maybe write is as its written in Psedo instead of Case select swicthes - then maybe better - have a go with that first - it will only be about 10 mins out of your life
Tier 1 Developer
puzzler2018
User Banned
Posted: 11th Mar 2019 23:06
There is too many IFs and elseif statements which cannot be interepreted easily by case switches
Tier 1 Developer
blink0k
Moderator
11
Years of Service
User Offline
Joined: 22nd Feb 2013
Location: the land of oz
Posted: 11th Mar 2019 23:12
It's quite straight forward.
puzzler2018
User Banned
Posted: 11th Mar 2019 23:16
Im sure it is - ill review it again tomorrow unless someone else finds the fault
Tier 1 Developer
blink0k
Moderator
11
Years of Service
User Offline
Joined: 22nd Feb 2013
Location: the land of oz
Posted: 12th Mar 2019 00:46 Edited at: 12th Mar 2019 00:48
I have made some very small progress;
1. I was not subtracting the header info length from the level length (gravity/fuel/oxygen and pallette, 222 bytes)
2. STRINGS CANNOT CONTAIN NULLS!!!!!!! So i have changed my output buffer to an array

So now i get a length of 2022 when it should be 2018. A difference of 4 bytes which is a little suspicious




This guy has written one in .JS but it may as well be martian to me
fubarpk
Retired Moderator
19
Years of Service
User Offline
Joined: 11th Jan 2005
Playing: AGK is my friend
Posted: 12th Mar 2019 01:02
your making some great progress on this

Martian might be a good language to learn one day, you just never know when you will need it

have you compared the values your getting for gravity/fuel/oxygen and the palette
to the original by using an emulator for some kind of window comparison to what
you have. maybe some screen grabs might help not sure but you sound very close
strange about the 4 byte difference tho
fubar
blink0k
Moderator
11
Years of Service
User Offline
Joined: 22nd Feb 2013
Location: the land of oz
Posted: 12th Mar 2019 01:08
Quote: "have you compared the values your getting for gravity/fuel/oxygen and the palette"

Yes they look good (As far as i can tell)
Gravity:8
Fuel:130
Oxygen:60

Quote: "the palette"

Yes. If you look at the palette i posted earlier, and then have a look at a youtube play through (of the first level), you can see the colours from the palette match the video (Not perfectly because i've converted it from VGA to 24 bit)

fubarpk
Retired Moderator
19
Years of Service
User Offline
Joined: 11th Jan 2005
Playing: AGK is my friend
Posted: 12th Mar 2019 01:14
how do you know it is meant to be 2018 and not 2022 sorry still trying to catch up on this thread
but if the 2018 is taken from a filesize and 2022 is what you need to calculate it that might be correct
after all. just a thought still examining thread
fubar
fubarpk
Retired Moderator
19
Years of Service
User Offline
Joined: 11th Jan 2005
Playing: AGK is my friend
Posted: 12th Mar 2019 01:20
Just a note before KrazyGrandprix I downloaded allsorts trying to extract the track data for stuntcar
even found a program that can save objects in memory none of them worked with anything I had, I gave
up but wiki might've been the better bet but some of those old hat programmers we really should
take our hats off to them what they had and what they did with it was amazing
fubar
blink0k
Moderator
11
Years of Service
User Offline
Joined: 22nd Feb 2013
Location: the land of oz
Posted: 12th Mar 2019 01:20 Edited at: 12th Mar 2019 01:22
The wiki says there is a table at the beginning of the roads.lzs file. Each element contains two 16 bit integers that are the offset and length of each level within the file

When i read the first entry of the table i get 124 as the offset and 2240 as the length for the first level

There are 222 leading bytes of information (gravity=2, fuel=2, oxygen=2, palette=72*3)

So i add 222 to the offset which gives me the offset of the road data, and take 222 of the length which gives me the (uncompressed) size of the road data
fubarpk
Retired Moderator
19
Years of Service
User Offline
Joined: 11th Jan 2005
Playing: AGK is my friend
Posted: 12th Mar 2019 01:44 Edited at: 12th Mar 2019 01:57
the java for decompression looks like this


I know you said its like martian but it may help someone viewing thread and it shows the offsets well




one thing I did notice in the js version the reading of bits(size) changes and this may be the discrepancy
fubar
blink0k
Moderator
11
Years of Service
User Offline
Joined: 22nd Feb 2013
Location: the land of oz
Posted: 12th Mar 2019 02:00
Quote: "one thing I did notice in the js version the reading of bits(size) changes and this may be the discrepancy"


Can you show me where this happens please?
Thanks
fubarpk
Retired Moderator
19
Years of Service
User Offline
Joined: 11th Jan 2005
Playing: AGK is my friend
Posted: 12th Mar 2019 02:04 Edited at: 12th Mar 2019 02:07
My java is so rusty but
line 43 and 45
and
line 53 and 55

EDIT and 63

in the decompress file shows it reading a dif bit amount
fubar
blink0k
Moderator
11
Years of Service
User Offline
Joined: 22nd Feb 2013
Location: the land of oz
Posted: 12th Mar 2019 02:08
Do you mean
dist = 2 + input.readBits(width2);
and
dist = 2 + (1 << width2) + input.readBits(width3);

They seem to be consistent with the wiki as far as i can see

fubarpk
Retired Moderator
19
Years of Service
User Offline
Joined: 11th Jan 2005
Playing: AGK is my friend
Posted: 12th Mar 2019 02:21
but the readbits command you pass how many bits you want to read and its passed through the inputstream
in line 63 it grabs a set 8 bits to variables its only in the latter section its actuall passed to its array outbuf and thats
a constant 8 bits. I really need to relearn java but its reading a byte with this input.readBits(1) when its checking
if theres a byte there aswell from my understanding. but im so rusty and had a quick look to see if there was an
explanation of readbits command
fubar
fubarpk
Retired Moderator
19
Years of Service
User Offline
Joined: 11th Jan 2005
Playing: AGK is my friend
Posted: 12th Mar 2019 02:25
Ive prob steered you up the garden path

but hope that helps clarify to you cos not so sure it does to me
fubar
blink0k
Moderator
11
Years of Service
User Offline
Joined: 22nd Feb 2013
Location: the land of oz
Posted: 12th Mar 2019 02:43 Edited at: 12th Mar 2019 02:43
Quote: "Ive prob steered you up the garden path"

Never!

I put the code side by side to see if there are any differences

Attachments

Login to view attachments
blink0k
Moderator
11
Years of Service
User Offline
Joined: 22nd Feb 2013
Location: the land of oz
Posted: 12th Mar 2019 02:55 Edited at: 12th Mar 2019 02:55
There must be something wrong because
2240 - 222 = 2018

The wiki says that the road data consists of an array of UINT16s (2 bytes) which represents 7 values per line.

This implies that the road data should be divisible by 7 and 2018 is NOT divisible by 7. 2016 on the other hand is. So there must be something i am missing
blink0k
Moderator
11
Years of Service
User Offline
Joined: 22nd Feb 2013
Location: the land of oz
Posted: 12th Mar 2019 03:41 Edited at: 12th Mar 2019 03:42
This code is on the wiki page


It appears to flip the bits prior to decompression using this code
%flip the bit order within the compressed section
bytes = [bytes(1:3); bin2dec(fliplr(dec2bin(bytes(4:end))))];


Now if someone understands this can you tell me if it is reversing the bytes within the block
or is it reversing the bits within each byte
or is it totally flipping the bits withing the block
or is it not touching the first three bytes and flipping the rest using one of the above methods

thanks in advance for any help
fubarpk
Retired Moderator
19
Years of Service
User Offline
Joined: 11th Jan 2005
Playing: AGK is my friend
Posted: 12th Mar 2019 03:49
well its got me stumped

fubar
Virtual Nomad
Moderator
18
Years of Service
User Offline
Joined: 14th Dec 2005
Location: SF Bay Area, USA
Posted: 12th Mar 2019 06:27 Edited at: 12th Mar 2019 06:36
blink,

i've been gazing in and out of this thread all day, watching you march forward (and unraveling my mind whenever it realizes that its in a knot)

while i feel you're close to the eureka! moment, consider "reading" the level images and creating your own file format as a back-up plan? (assuming you're locking yourself into the remake time limit)

otherwise, i wish i had more to offer toward your quest but thank you for sharing it

ps: maybe contact the image creator for a nudge? (in dutch)

blink0k
Moderator
11
Years of Service
User Offline
Joined: 22nd Feb 2013
Location: the land of oz
Posted: 12th Mar 2019 07:42
Thanks Mr Nomad. I really appreciate the support. Those links look good too. Nice backup option
blink0k
Moderator
11
Years of Service
User Offline
Joined: 22nd Feb 2013
Location: the land of oz
Posted: 13th Mar 2019 02:40
I think i've sorted this. I'll elaborate when i have something to show

Login to post a reply

Server time is: 2024-11-23 02:37:41
Your offset time is: 2024-11-23 02:37:41