write dirblock / read fileblock usage help

Author

Message

Naphier

15

Years of Service

User Offline

Joined: 2nd Oct 2010

Location: St Petersburg, Florida

Posted: 6th Nov 2011 05:52

Link

I'm working through Hands on DBPro Vol 1 and reached this curious point that is not working for me.
I create a package with write dirblock and then try to extract the files in that directory with read fileblock.
All I get is a 2 byte file with hex 0F 00 in it.

What am I doing wrong?

Creating the dirblock:

+ Code Snippet

open to write 1,"d:\backup.pak"

write dirblock 1,"d:\test"
close file 1
print "operation complete"
wait key 
end

Extracting the files with read fileblock:

+ Code Snippet

open to read 1,"d:\backup.pak"
read fileblock 1,"d:\test\header.txt"
read fileblock 1,"d:\test\GreenSpace.jpg"
read fileblock 1,"d:\test\It-Came-From-Above.jpg"

close file 1
print "extraction done"
wait key 
end

Note that header.txt was not a file in the directory copied with write dirblock but was my attempt at putting the first block of erroneous data into a separate file. No such luck.

All help is greatly appreciated.

~Aspiring~

Back to top

Profile PM Email Website

Naphier

15

Years of Service

User Offline

Joined: 2nd Oct 2010

Location: St Petersburg, Florida

Posted: 6th Nov 2011 06:00

Link

Note that read dirblock extracts both of the files perfectly fine...

~Aspiring~

Back to top

Profile PM Email Website

IanM

Retired Moderator

23

Years of Service

User Offline

Joined: 11th Sep 2002

Location: In my moon base

Posted: 6th Nov 2011 13:51

Link

Dirblocks are not quite the same as Fileblocks but with care, Dirblocks are convertible to Fileblocks.

Dirblock:

+ Code Snippet

integer = Number of files
Repeat for each file
  integer = Filename length
  byte * Filename length = Filename
  integer = File data length
  byte * File data length = File data

Fileblock:

+ Code Snippet

integer = File data length
byte * File data length = File data

So when reading a Dirblock as a Fileblock, you need to skip those first four bytes (number of files), then for each file in the Dirblock, you need to read the next 4 bytes, then skip that number of bytes. At that point you can use the READ FILEBLOCK command to access the data.

Get the latest Matrix1Utility plug-ins (11-SEPT-2011)

Back to top

Profile PM Email Website

JRNTexas

15

Years of Service

User Offline

Joined: 24th May 2011

Location: Austin, Texas

Posted: 6th Nov 2011 22:03 Edited at: 6th Nov 2011 22:09

Link

@Naphier

Your code should look more like the two following examples:

+ Code Snippet

open to write 1, "backup.pak"
if file open(1)=1
  write dirblock 1, "myDirectory"
  close file 1
endif


open to read 1,"backup.pak"
if file open(1)
  read dirblock 1,"myDirectory"
  close file 1
endif

Once the directory is back, you can read your files from the directory. Then you can delete the directory because is still exists inside "backup.pak".

Also, here is a link to a complete program that both reads and writes using dirblock:

DirBlock Article

Back to top

Profile PM Email Website

Naphier

15

Years of Service

User Offline

Joined: 2nd Oct 2010

Location: St Petersburg, Florida

Posted: 6th Nov 2011 22:41

Link

Thanks, JRNTexas, but read/write dirblock is working just fine for me as is stated in the addendum to my first post.

My problem is using write dirblock with read fileblock hence trying to extract single files out of the package created with dirblock.

IanM - I completely catch the concept you're illustrating. One problem I see is that you cannot pass the file size to READ FILEBLOCK.
I'm not sure why Hands On DB Pro is using the example I posted when it doesn't even come close to working.

Does anyone have a concrete example of code that creates a package with WRITE DIRBLOCK then extracts a single file of many files in the package by using READ FILEBLOCK?

Thx

~Aspiring~

Back to top

Profile PM Email Website

Naphier

15

Years of Service

User Offline

Joined: 2nd Oct 2010

Location: St Petersburg, Florida

Posted: 7th Nov 2011 01:18 Edited at: 15th Nov 2011 03:04

Link

With some DIRection from IanM (thanks!) I now have a solution:

Packages created by DIRBLOCK have the following structure:
1). First 4 bytes make up an integer = the # of files in the package
- These bytes need to be READ to advance the READ command's byte index

2). The next block of the package must be read with READ FILEBLOCK and will contain the name of the first file in the package.
- I imagine we could read the text in this first file and use it to name our first extracted file. I tried just reading it with READ STRING and it did not work. Might be able to do READ BYTE and put those bytes together to make up the string for the filename.

3).Now when we run READ FILEBLOCK again we will finally get our first file.

4).Repeat steps 2 and 3 for how ever many files you want to extract.

+ Code Snippet

open to read 1,"d:\backup.pak"

read byte 1,c
read byte 1,d
read byte 1,d
read byte 1,d

print c," files contained in d:\backup.pak"


read fileblock 1,"d:\test\filename1.txt"
read fileblock 1,"d:\GreenSpace.jpg"
read fileblock 1,"d:\test\filename2.txt"
read fileblock 1,"d:\test\It-Came-From-Above.jpg"
read fileblock 1,"d:\test\filename3.txt"
read fileblock 1,"d:\test\mmm.jpg"

close file 1
print "extraction done"
wait key 
end

Note that you do not need to reference the original directory structure or the original filename.

I imagine I could create from this a sort of random access algorithm for getting to just the file I need. Though it seems I would have a lot of files that would be created and then need to be deleted.

Although I enjoy Jesse Redman's article I'm concerned that putting files in a DIRBLOCK is not the best method of protecting files. Any DB Pro user should be able to get to them with the above method. However I suppose if you name the files with incorrect extensions then it will make it a little tougher to figure out what is what. Also if they had your game they could simply know to look for the files while the game is running.

I guess if I want better protection I'd need to create some sort of encryption algorithm to scramble the data in the DIRBLOCK file. A bit beyond me right now, so I will stick with DIRBLOCK.

Thanks to you both for the help.
I've learned much from this!

~Napland Games~

Back to top

Profile PM Email Website

Naphier

15

Years of Service

User Offline

Joined: 2nd Oct 2010

Location: St Petersburg, Florida

Posted: 7th Nov 2011 02:29

Link

And just because... Here's code that essentially does the same that READ DIRBLOCK does, reads the filename from the DIRBLOCK package then extracts that file with the correct name.

+ Code Snippet

global filename$

open to read 1,"d:\backup.pak" //your package filename here

read byte 1,c
read byte 1,null
read byte 1,null
read byte 1,null

print c," files contained in d:\backup.pak"
print

for a = 1 to c
    PrintFilename()
    ExtractFile(filename$)
    filename$ = ""
next c

close file 1
print "Extraction done."
wait key 
end

function PrintFilename()
    
    for adv = 1 to 4
        read byte 1,null
    next adv
    
    repeat        
        read byte 1, char
        filename$ = filename$ + chr$(char)
    until char = 0
    
    print filename$
    print

endfunction

function ExtractFile(filename$)

    read fileblock 1,filename$

endfunction

Pretty boring, but it does the job. Next step would be to allow random access to a file by simply selecting the file's number, deleting unneeded files immediately after creation, load the sought after file into memory, and then delete it after it is loaded to memory or no longer needed.
Thus not all of your program's files are available at once and remain a bit more hidden than if you use READ DIRBLOCK.

Hoping there's better ways to load a file to memory directly from a packaged file.

~Aspiring~

Back to top

Profile PM Email Website

IanM

Retired Moderator

23

Years of Service

User Offline

Joined: 11th Sep 2002

Location: In my moon base

Posted: 7th Nov 2011 14:52

Link

Quote: "I imagine I could create from this a sort of random access algorithm for getting to just the file I need."

Yes, you only need 3 bits of information from the dirblock to access the data for each file directly:

1. The filename
2. The file size
3. The start point of the data

Put those into a UDT array and then save them alongside the dirblock file. Then when you want to get the data from one file, load the array, locate your file plus the size and start point. Then either open and read the file directly, or map the file into a memory bank.

Get the latest Matrix1Utility plug-ins (11-SEPT-2011)

Back to top

Profile PM Email Website

Naphier

15

Years of Service

User Offline

Joined: 2nd Oct 2010

Location: St Petersburg, Florida

Posted: 9th Nov 2011 04:39

Link

Perfect. I'm really glad I'm getting some useful answers here. Usually us beginners are treated with such disdain.

I now have some really solid ideas on asset protection, encryption algorithms, and even packing files into rar and zip for protection.

Now if I just stop side tracking and get to my learning, maybe I'll be able to bring one of my game ideas to fruition before the year's up.

Here we go!

Back to top

Profile PM Email Website

Sorry your browser is not supported!

DarkBASIC Professional Discussion / write dirblock / read fileblock usage help