Sorry your browser is not supported!

Link to Newsletter Article

I will be monitoring this thread so if you have comments or questions about the article, post them in this thread!

Here is a code update:

I re-wrote the "make file" routine to both write and read the file. It also renames the directory with an "x" on the end when it makes the file from the directory, just to keep the directory from accidentally being deleted by your application.

Please don't consider this an example in how to write code. It's just a quick version of a tool. Again, this isn't a snippet, it's a complete program.



I hope you find it useful.

I was working through Hands On DBPro Vol 1 and it happened to be somewhat aligned with this article from Jesse.
I was attempting to extract files from a DIRBLOCK package using READ FILEBLOCK. I got it to work and it sort of illustrates a loop hole in Jesse's asset protection.
Read more here
http://forum.thegamecreators.com/?m=forum_view&t=191104&b=1

One thing I have to point out with this article is you do not have to know the original directory structure to get files out of one of these packages. So this is unfortunately not protecting your assets very well. Anyone with DB can extract the files and anyone with a binary file editor could do so as will, albeit a little difficult.

I'd really love to see an article to build on this one and encrypt the package file. Or showing us how to load a file directly to memory from one of these package files.

Thanks for the article and helping me to learn something!

@Naphier

It's kind of like watching a magic show and going "how did he do that?" Then going back stage and having the magician show you. Then you have to ask yourself "that was so simple, why didn't I immediately catch on?"

First, I agree that the file gives limited protection. But who are you protecting it from and what is their level of knowledge about such things? If they are very informed, there may not be much you can do regardless of what you try. Most, but not all sophisticated codes can be broken, given enough time.

But it actually depends which end you start from. If I ask you to guess a number between 1 and 100,000 you'd have a hard time guessing it. However if I told you to take 3 multiply it by 16 add 12, divide by 8, add 3,000 and multiply by 30 and that is always the number, you'd say that's easy. There's no guess work at all, it's a formula. Going at it this way, the file is relatively easy to disassemble.

If someone hadn't just given you the complete instructions for disassembling the file, and the file was 75MB, you had never seen the extension type before and you had NO instructions what so ever, and you had no idea even IF it was a single file or a hundred files and you had no clue how long each file was, what the true purpose of the file was, how the program used it or any other "inside" information, THEN it's a little harder than it looks to someone with a complete set of instructions.

I can devise another scheme that has the same level of difficulty and you wouldn't think it was so easy, not knowing the use of the file or any other information.

Could it still be broken? Certainly. What's the level of difficulty? Depends on who you are and what tools you use. And for those with the right tools and experience, so can almost anything you can come up with.

The article is only really a starting place. For me it provides just the amount of protection that I am typically looking for. Most users will not understand the purpose of the file and won't bother trying to extracting 85 files from a 75MB conglomerate. And they either won't know to come here to look for the answer or wouldn't come here, even if they knew that somewhere on this site was the answer. I'm not sure that even someone with experience in such things, without knowing the format, would bother. They might but it has to be a percentage much less than 50% of those with knowledge. A large percentage of those with the ability will probably never even know about your program.

If, you take every file you have, make it a "memblock" then divide it in half, write it out in two parts, and you do this to 50 files, you have 100 files each without "header" info. Each could have a different extension<EDIT> or no extension</EDIT>. Put those into a directory and use dirblock to obscure them and... You get the idea.

Take what you learn and add your personal twist to it.

I could use a specific extension, that is not used elsewhere, for all of my dirblock files. I could put data in 25% of them and put pure trash in 75% of them. I could put 25% real data in 25% of them and make the other 75% of content in the 25% of the files trash and make the other 75% of the files complete trash. Where does this end...?

I will still be using dirblock and I doubt that I personally will go much beyond using dirblock. After all, it's up to the individual what kind of files are in the directory, inside the file. The files may or may not be usable in any meaningful way by whoever wants to pull them out.

I can come up with a hundred other schemes to obscure the data but again...you get the idea.

<EDIT>Oh, and for me, the effort of writing code to load the files from the dirblock without making them back into a directory takes more time than it's worth. My directories, extracted from a dirblock file have a very short life and they are opened at different times so finding one open and accessible is a crap shoot anyway</EDIT>

I agree, DIRBLOCK is still quite handy. I will likely use it to package some files (maybe installation or data files). But for media I will likely use some method of scrambling.

I admit this, as a new programmer, this is the first time I've thought about these things and I thought that DBPro had some built in encryption for files.

1 week ago I had no idea how to do basic file protection and now (thanks to your article, IanM's guidance, and a faux pas in Hands on DBPro - though maybe it wasn't meant to work right) I have a pretty solid idea on how to do basic file packaging like you've described and also how to come up with my own algorithm to scramble/encrypt my files. I even think I've got some ideas on file compression (though I doubt I'll bother since WinRAR is so solid).

Thanks again for some great inspiration and lessons!

@Naphier



As a new programmer, your getting fairly deep into it and that's a good thing!

The TGC's forums have a lot of people willing to help out. Along with TGC's development tools, Dark Basic Pro and AppGameKit and others, you have a lot of resources that are not only good while you are learning, but after 35+ years of development, I still find very useful.

@JRNTexas - I'll post what I've got tomorrow when I'm more awake. I've gone back a few months and dug out my unrar code. Some of the feature(s) that I've got in the plugin so far just from tonight's coding...

. Extract a file from .rar file to memblock or file on disk (with or without password protection/encryption on the rar file) - (WORKS)

memblockID = RAR FILE TO MEMBLOCK( rarfile$, filename$, password$ ) - returns 0 if failed or ID of memblock

success = RAR FILE TO FILE( rarfile$, filename$, password$ ) - returns 0 if failed, 1 if successful

Okay two so far, but it doesn't need any extras. The rar files are created using upto version 2.90 of rar (and have a password). Plus the trial version of all rar software only lasts for 40 days until you register it.

TO DO: Find another easy enough to work with compression/encryption lib.

I'm working on this anyway at the moment but I'll produce any future plugins for DBPro if there's any interest in protecting media this way.

@WLGfx

This looks great! I'm interested and I am sure others would be interested in your plugins for directly manipulating RAR files in DBPro.

Please keep us updated on your results!

Here's the RAR version with an example DPRo project attached. You can't open the media.rar file without using the password "WLGfx"...

It has two commands as listed above and is as simple and quick to use:



Hope this helps anyone...

PLEASE NOTE: You will need WinRar2.90 to make the rar files. Anything above will not work...

I've found some old code lying around the web for unzipping password protected files to disk or memory so today I'll be working on that for another DBP plugin. The UNRAR plugin works nice and fast, so I'm expecting the same speed if not faster for the zip extraction. After some testing I'll post the updates...

This will be much easier as any zip program can be used to create the zip file with a password. It will then be up to the programmer to encrypt their password in their own code...

Will let you know later how things are going...

And now for the ZIP version... (attached)

I've tested it with a WinZip 2.0 compatible password protected archive.


The above code unzips a single file from the password protected archive...



EDIT: Apologise for multiple posts. I got carried away...

The actual function in the plugin for unzipping a file in case anybody is curious, this is it:

@WLGfx
Awesome.

@Naphier - Thanks. I'm hoping it will come in useful as the subject has been mentioned a lot lately. So what better than the ability to grab files quickly from a password protected archive?

I doubt there's much more I can add to the plugins, plus they're independent of any other resource and only have a small foot print on your code. 28k for the rar extractor and 38k for the zip extractor. Nothing compared to the extra protection and compression it gives...

Another question in this realm:
What exactly constitutes a proprietary file format?

Can you simply change an extension and say it's without changing how the data is stored?

Can you just add a header block to a file of your own and call it proprietary?

Can you take an open format and encrypt it then call it your own proprietary format?

Does anyone have references for this (I've been searching all day with minimal results)?

Thanks!

@Naphier - From what I'm aware of and read around all my searches for free media, objects, codes nippets, etc, is that as long as they are not on show (ie visible in a directory for easy access), then you're fine to use them. Also, some media and snippets sometimes requires that you insert somewhere, whether in credits in your software or credits in documentation about the author or where it came from. So long as you follow those steps then that's fine for almost all freely available media or code.

When it comes to altering the proprietary status of such code or media, then no, you can't do that. As I've explained above, some do require that you at least give credit somewhere. That's even true if you encrypt, compress, add headers or embed.

The one thing I do tend to forget is when I download such things, some of them don't come with a doc file and I forget where they came from. Only really lately (hands up to that) I've just started making notes of the media and snippets I've acquired.

Also, it's helpful to take note of the difference between "free for non-commercial use" and "free for commercial and personal use". Some things may require a license purchase if you're planning on making money out of a creation and including someone else's property. In cases like this be careful, especially if you are planning on selling your product.

Protecting other peoples property within your product always helps too, whether you embed, compress, encrypt. I suppose that's where my dead simple plugin comes in handy...

I'm not worried about having permission to use proprietary file formats, I'm more interested in what actually makes a proprietary file format.

For example, WAV file format is a non-proprietary format.
If you encrypt a WAV file and then give it a random extension is it then your own proprietary file format?
What is the minimum alteration to a non-proprietary file format to make it your own proprietary format?

Another example: I create a model in blender and convert it to DBO. I have legal permission to commercially use DBO (and blender files in that case). Can I alter a DBO file and call it my own proprietary format if the legal terms of the DBO converter allows it, and what is the minimum alteration?

Again I'm just talking about the file format, not the actual media itself. I'm not looking to scrub someone else's work to make my own, just wondering if you can take something like an MP3, encrypt it and call it MPP (or whatever) and claim that's your proprietary format... or do you have to go to the WAV format, or straight to the raw data (not sure if it gets more raw than WAV...).

Or another example: I can't imagine the txt file format is proprietary at all... if I simply scramble that data and call it by a different extension, is that my format?

I'd always give credit to predecessors, but just wondering how one goes about making their own proprietary file format.

Sorry if that wasn't clear or if I'm reading you wrong.
Thx

I suppose in that case then, lets say I have a batch of files, I compress each one individually and throw them in a single file. That single file would have a header telling me how many files, their names, offsets, etc. That would essentially be my own file format and I could license it.

But, the one part of the licensing that I may come a cropper on in the above example, is if I used someone else's compression technique before anything else that needed some kind of recognition.

I suppose that's why I throw all my stuff out there for free anyway. If the day comes and I do something worth while then I'd have to take things a little more serious...

Yeah can't imagine I'd ever try to license my own file format, but I really wondered what makes one proprietary.
It didn't seem right that you can take, for example, a RAR file and just rename the extension to call it proprietary. But maybe it is just that: take an open format and convert it to one of your own by whatever method you like other than just changing the extension.

Curiosity satiated.

Thanks!