Sorry your browser is not supported!

You are using an outdated browser that does not support modern web technologies, in order to use this site please update to a new browser.

Browsers supported include Chrome, FireFox, Safari, Opera, Internet Explorer 10+ or Microsoft Edge.

AppGameKit/AppGameKit Studio Showcase / [WIP] Beta release of WadPacker: Single file, cross platform protected media for T1

Author
Message
nz0
AGK Developer
17
Years of Service
User Offline
Joined: 13th Jun 2007
Location: Cheshire,UK
Posted: 28th Jul 2017 00:02 Edited at: 17th Jan 2018 01:55
This is the first beta release of WadPacker.
You are now able to protect your media files files with this easy to use system. Most common file types are included (image, objects, sounds, text files, shaders etc.), with more features to come.

Included:

Compressor program to generate Wads
Library to include for loading assets
Example project
Documentation

Updated: version 0.2
Improved packer
#option_explicit supported
music files supported

Updated: version 0.3
Added case sensitivity constant
Added Wad_GetManifest() function
Added Wad_LoadSkeleton()

Updated: version 0.4
Added command line functionality to packer.
Simpler to keep your wad file up-to-date and/or include in build process.

Updated: version 0.5
Added fast text file loader (NULL terminated files)
Partial support for LoadFont() - pending fix by TGC - didn't get fixed in 2017.12.12 release
Some bugfixes

Attachments

Login to view attachments
Ian Rees
AGK Developer
12
Years of Service
User Offline
Joined: 3rd May 2012
Location: Wales, UK
Posted: 28th Jul 2017 08:43
Thanks for this nz0!!

Just wrote a quick test app with my own media and it worked beautifully - great job! This will be extremely useful.

What is this strange word that you've used - 'documentation'?

Thanks again - the documentation and sample code is great!!
E
AreCustomize
10
Years of Service
User Offline
Joined: 22nd Dec 2013
Location: Cyberspace
Posted: 28th Jul 2017 19:26 Edited at: 28th Jul 2017 19:29
Hi!

I tried to open ray.wad made from testmedia but I got the following error message.
I use AppGameKit Version 2017.5.15
"Anyone who has never made a mistake has never tried anything new."
-Albert Einstein

Attachments

Login to view attachments
nz0
AGK Developer
17
Years of Service
User Offline
Joined: 13th Jun 2007
Location: Cheshire,UK
Posted: 28th Jul 2017 20:08
The function has returned you -1 which usually means the file name / reference name name you passed to the function doesn't exist.
Can you show a source example?


AreCustomize
10
Years of Service
User Offline
Joined: 22nd Dec 2013
Location: Cyberspace
Posted: 28th Jul 2017 20:54
Quote: "The function has returned you -1 which usually means the file name / reference name name you passed to the function doesn't exist.
Can you show a source example?"


Oh. Problem solved. My bad.
I made .wad file again and first time I apparently selected just /WadPacker0.1/ folder instead of /WadPacker0.1/testmedia/

So now it works like it should.
"Anyone who has never made a mistake has never tried anything new."
-Albert Einstein
nz0
AGK Developer
17
Years of Service
User Offline
Joined: 13th Jun 2007
Location: Cheshire,UK
Posted: 28th Jul 2017 23:52
Ah good. The root folder you select in the packer program becomes the root folder you use for referencing the individual media.
PartTimeCoder
AGK Tool Maker
9
Years of Service
User Offline
Joined: 9th Mar 2015
Location: London UK
Posted: 29th Jul 2017 19:40
This is a neat little tool

When I first run the packer I get a "Could not create file" error, if I close the app and reopen and pack again with the same settings loaded from INI I get "I/O Error 145" but the wad is created and loads in AGK

Every time I run the app the same result, set file>>could not create error>>restart>>i/o error>>wad created, but it works
nz0
AGK Developer
17
Years of Service
User Offline
Joined: 13th Jun 2007
Location: Cheshire,UK
Posted: 30th Jul 2017 00:15
It seems like you do not have sufficient access permissions on the folder where you installed WadPacker.

The errors signify that the program is unable to delete or modify files there.
Are you installing to a restricted location?

The program needs to have full access to it's own folder.
PartTimeCoder
AGK Tool Maker
9
Years of Service
User Offline
Joined: 9th Mar 2015
Location: London UK
Posted: 30th Jul 2017 01:26
Its running from the desktop, it should have sufficient permissions from there but I move it into my AppGameKit projects in documents and same result, I have full access in security, I even tried running as admin to no avail.

Got to love windows, when it should work it doesn't and when it reports it doesn't work it does!.
nz0
AGK Developer
17
Years of Service
User Offline
Joined: 13th Jun 2007
Location: Cheshire,UK
Posted: 30th Jul 2017 01:50
It might appear to work, but I am concerned there will be a problem at some point.

The program creates a temporary folder where WadPacker.exe is located called "_wadpack" when you start the build.
In the final pass of building the wad file, as each compressed media file is added to the wad, they are deleted from the "_wadpack" folder and at the end the folder is removed.
The error 145 is "directory not empty" so there will be files left in there, assuming because the delete failed on the files within.

If there are files left in there (you can check) then they may get added to subsequent wad files * checks code * or not.. actually, it only adds the files from the new scan, so not as serious a problem as I first thought, just don't like temp files lying around

I'll put a few more checks in the next version (I have a few, but it's the first release).

The other error "could not create file" on first run: is that as soon as the program loads or when you hit the pack button?
Would you be so kind as to check a debug version?
Supertino
7
Years of Service
User Offline
Joined: 22nd Jun 2017
Location: Behind you!
Posted: 8th Aug 2017 09:35 Edited at: 8th Aug 2017 09:36
@ nz0 this is great, I just used it to pack the media for my game and it works a treat (windows 10), the only issue I have is I use #option_explicit so I'll need to go through and make some changes to your source so I can continue to use the option. I'll be testing the .wad on mac later but I don't see why it would not work.
nz0
AGK Developer
17
Years of Service
User Offline
Joined: 13th Jun 2007
Location: Cheshire,UK
Posted: 8th Aug 2017 15:57
^ Glad to hear it's working out for you. Looking forward to seeing it tested on mac and other platforms.

I did start making it #option_explicit compliant, but didn't finish it.
I've now done so and I've made some more improvements, so should see an update here in the next few hours.
nz0
AGK Developer
17
Years of Service
User Offline
Joined: 13th Jun 2007
Location: Cheshire,UK
Posted: 8th Aug 2017 20:52
Version 0.2 updated - see first post
@PartTimeCoder there seems to be an issue if you create a Wad file in a folder with a space in the path name, although it seems to work the 2nd time you repeat the steps!? The filename is correctly quoted, so I don't know why!
Cybermind
Valued Member
21
Years of Service
User Offline
Joined: 28th Nov 2002
Location: Denmark
Posted: 9th Aug 2017 08:57
Hello nz0 I will try to implement your solution into my project. I was working on my own packer, but I have a problem with the MakeFolder() command and no one has confirmed whether it is a bug with AGK2 or a bug 40 cm in front of the screen Can I credit you on the credits screen?
13/0
PartTimeCoder
AGK Tool Maker
9
Years of Service
User Offline
Joined: 9th Mar 2015
Location: London UK
Posted: 9th Aug 2017 17:40
Quote: "The other error "could not create file" on first run: is that as soon as the program loads or when you hit the pack button?
Would you be so kind as to check a debug version?"


I get the error when hitting the pack button, and sure yea I'll try a debug version, I tried v0.2 and still the same issue, you are right it only seems to happen with a path with spaces which is odd, and inconvenient lol
Supertino
7
Years of Service
User Offline
Joined: 22nd Jun 2017
Location: Behind you!
Posted: 9th Aug 2017 21:07 Edited at: 9th Aug 2017 21:09
@ nz0 - wad loaded fine on MAC FYI . I did get an issue trying to use OGG files (windows and mac), I tried a 10MB file and 1MB file and I got "array index out of bounds" error pointing to a func in your lib, I could load and use the same ogg file using AppGameKit LoadSound() but error using Wad_LoadSoundOGG().
nz0
AGK Developer
17
Years of Service
User Offline
Joined: 13th Jun 2007
Location: Cheshire,UK
Posted: 9th Aug 2017 21:18 Edited at: 9th Aug 2017 21:19
Can you show me an example so I can fix it?
Did the enclosed example ogg work for you?
Thanks
Supertino
7
Years of Service
User Offline
Joined: 22nd Jun 2017
Location: Behind you!
Posted: 10th Aug 2017 08:04
Sure, I'll get what you need when I get home this evening, I never tried your example as it was easy enough to apply but I'll try that too and let you know.
Supertino
7
Years of Service
User Offline
Joined: 22nd Jun 2017
Location: Behind you!
Posted: 10th Aug 2017 20:29 Edited at: 10th Aug 2017 20:34
@ nz0 okay, found the issue seems the file names are case sensitive, not sure if this is by design? Using your orginal example and media I can mix up the case when using WAD_Loadsound() or WAD_Loadimage() etc and if it's different to the filename it will error out at line 254 for audio and effective not be able to find the image for images.
nz0
AGK Developer
17
Years of Service
User Offline
Joined: 13th Jun 2007
Location: Cheshire,UK
Posted: 11th Aug 2017 01:35
Hi,

First off, you mean it's case sensitive only on the Mac?

I'll do a pass on the packer part which forces the case on any files to be all uppercase and see if that helps?
nz0
AGK Developer
17
Years of Service
User Offline
Joined: 13th Jun 2007
Location: Cheshire,UK
Posted: 11th Aug 2017 02:19
I just checked and it should be no different on a case sensitive platform when loading files normally or with Wadpacker.
It will try and load the files using the same case as what the original names where, so:

if your media source folder contained a file called "images/MyImage.png" it would try and load the file using the same case (on a case sensitive platform)
If you tried to load "images/myimage.png" it would fail as expected on a case sensitive platform - wadpacker or not.

I've no experience with Mac, but on Android filenames are case sensitive, so it's not what I'm doing, but the target platform.

In fact, if you look at the loader code, it simplifies the process somewhat by stripping the leading path info when it instantiates the file for loading into AppGameKit in the background, so if you had a complex source folder like:

images/title/character/Player.png as your source file (your source files which you packed with the packer)

Then, the loader only extracts the interim file as Player.png (doesn't bother with all the folder structure as it's not needed)
The folder structure is only retained virtually for referencing the file in the manifest, so you still pass the full virtual path + filename to the function to make sure it selects the correct file.

You can use the error codes supplied by the functions and it wouldn't get to an error in the loader code because you will have got a -1 from the function (e.g. "name not found")
I also made it easy for you to write your media loader in such a way where you can use the "real" folder during development and switch over to the packed version when you make a release, e.g. just have a conditional constant which you use for loading stuff one way or the other. It will always be the same filename passed each time.

I've added a helper function for the next version which returns you the structured manifest as a string so you can see all the actual file names you should be using to reference, including the original case.

Do you think it would be helpful if I made the loader case insensitive so it makes things easier (transparent) on such platforms where case is important?
nz0
AGK Developer
17
Years of Service
User Offline
Joined: 13th Jun 2007
Location: Cheshire,UK
Posted: 11th Aug 2017 02:32
@Cybermind Until I add an encryption to the loader, it probably won't be a good idea to advertise which "protection" system is being used.
Anyone who can see the AppGameKit loader code posted on here with the package will be able to reverse engineer the WAD file if they are reasonably competent.

Supertino
7
Years of Service
User Offline
Joined: 22nd Jun 2017
Location: Behind you!
Posted: 11th Aug 2017 07:40 Edited at: 11th Aug 2017 08:39
Hi nz0 I was doing this on Windows 10 and got the error, insensitive case would be nice but I don't think it s major issue once it's known.

As I said I was getting this error on windows and I can repeat the process to get the same;

1. Download your 0.2 attachment
2. Extract to the desktop (also tried c:\ - a path with no spaces just on a whim)
3. followed your read me to create the WAD and place it in the AGKCode\Wadloader\Media folder
4. open the AGKCode\Wadloader\Wadloader.agk file and run it - WORKS!
5. go to line 41 and change the filename from "ingame2.ogg" to "Ingame2.ogg" and run it - ERROR! "Array index out of bounds, Index: -1, Array Length: 12 in wadlib0.2.agc at line 254"
nz0
AGK Developer
17
Years of Service
User Offline
Joined: 13th Jun 2007
Location: Cheshire,UK
Posted: 13th Aug 2017 16:51
Ah OK, I see what you mean now.

The manifest lookup is case sensitive, as this is what the AppGameKit arrayname.find() function does.
I have added a config switch for the next version which lets you specify if you want to use case sensitivity or not.
Wilf
Valued Member
18
Years of Service
User Offline
Joined: 1st Jun 2006
Location: Gone to Unity.
Posted: 14th Aug 2017 13:00
Hey nz0, really nice tool. It works when packing images and sounds but seems to have a bug with Wad_LoadSpriteShader when the shader is in a sub-folder ("/shaders")
Quote: "
---------------------------
Message
---------------------------
Array index out of bounds, Index: -1, Array Length: 14 in wadlib0.2.agc at line 254
---------------------------
OK
---------------------------
"


Putting that same shader in the root of the pack works (well, has a problem on my machine but its found at least)
Supertino
7
Years of Service
User Offline
Joined: 22nd Jun 2017
Location: Behind you!
Posted: 14th Aug 2017 14:24
@ nz0 - As Wilf mention.... I too forgot to mention that some of the WAD_Load functions are missing the step where it looks-up the file in the manifest and returns the correct object, if you look at WAD_LoadImage() you can see what needs to be added to WAD_LoadSpriteShader() for example.
nz0
AGK Developer
17
Years of Service
User Offline
Joined: 13th Jun 2007
Location: Cheshire,UK
Posted: 15th Aug 2017 00:09
Cheers for that guys. Fixed in 0.3
Wilf
Valued Member
18
Years of Service
User Offline
Joined: 1st Jun 2006
Location: Gone to Unity.
Posted: 16th Aug 2017 06:39
Oh yes Supertino, clear in retrospect! nz0, awesome. Is any chance of adding Spine skeleton support? Something like:



Test assets attached

BTW AppGameKit seems to have a bug when cleaning up atlas files loaded more than once. Deleting the .atlas on disk removes the data from the animation of all instances, leaving just a white sprite. Couldn't repro this when loading directly from a folder so I think its a file-system bug.

Attachments

Login to view attachments
nz0
AGK Developer
17
Years of Service
User Offline
Joined: 13th Jun 2007
Location: Cheshire,UK
Posted: 16th Aug 2017 18:48
Can we get some confirmation from Paul on this atlas file bug?
Wilf
Valued Member
18
Years of Service
User Offline
Joined: 1st Jun 2006
Location: Gone to Unity.
Posted: 17th Aug 2017 16:10
He did help, here are the amended functions:



Basically keeps the image creation/atlas stuff in one place instead of split across two.
nz0
AGK Developer
17
Years of Service
User Offline
Joined: 13th Jun 2007
Location: Cheshire,UK
Posted: 17th Aug 2017 21:52
I see what you've done there, but I would rather compress the atlas file in the same package, assuming:

The atlas file behaves like the subimages.txt, in that the file name of the atlas (and the JSON) are always the same name as the associated image file or just each other?

The change you made there won't work because the atlas file is extracted and deleted without loading it?

Has it been cleared up that the atlas file can be removed after loading then?

nz0
AGK Developer
17
Years of Service
User Offline
Joined: 13th Jun 2007
Location: Cheshire,UK
Posted: 17th Aug 2017 21:56
Right, I've just seen your bug thread which Paul has responded on. This also answers my questions above, so I'll make the appropriate amendments.
nz0
AGK Developer
17
Years of Service
User Offline
Joined: 13th Jun 2007
Location: Cheshire,UK
Posted: 18th Aug 2017 20:44
Version 0.3 updated - see first post
Wilf
Valued Member
18
Years of Service
User Offline
Joined: 1st Jun 2006
Location: Gone to Unity.
Posted: 19th Aug 2017 05:53
nz0
AGK Developer
17
Years of Service
User Offline
Joined: 13th Jun 2007
Location: Cheshire,UK
Posted: 26th Aug 2017 01:34
Hey Wilf, is the Spine loading stuff working OK for you?

Any other methods need adding or changing?
Wilf
Valued Member
18
Years of Service
User Offline
Joined: 1st Jun 2006
Location: Gone to Unity.
Posted: 26th Aug 2017 16:12
Works like a charm, thanks. Nothing else occurs to me right now, I was going to ask for .tmx files to be added before I realised you already have that covered with Wad_LoadTextFile()
Yuhe
7
Years of Service
User Offline
Joined: 27th Aug 2017
Location:
Posted: 27th Aug 2017 04:43
Thanks You!!
quartz
21
Years of Service
User Offline
Joined: 5th Jan 2003
Location:
Posted: 27th Aug 2017 12:05
Tested v0.3, works great.
Great program.
Tumira
9
Years of Service
User Offline
Joined: 28th Oct 2015
Location:
Posted: 3rd Oct 2017 02:03
This is an awesome tool for agk. Will you add support for the new AppGameKit Visual Editor ?
nz0
AGK Developer
17
Years of Service
User Offline
Joined: 13th Jun 2007
Location: Cheshire,UK
Posted: 3rd Oct 2017 20:27
I don't know - are there specific load commands for it?
nz0
AGK Developer
17
Years of Service
User Offline
Joined: 13th Jun 2007
Location: Cheshire,UK
Posted: 10th Oct 2017 00:52
Updated: version 0.4
Added command line functionality to packer.
Simpler to keep your wad file up-to-date and/or include in build process.

It gets a bit annoying to have to keep running it, even though it remembers paths, I was using it for 2 different projects at the same time, so it made sense to add this option.
Supertino
7
Years of Service
User Offline
Joined: 22nd Jun 2017
Location: Behind you!
Posted: 10th Oct 2017 08:59
nz0
AGK Developer
17
Years of Service
User Offline
Joined: 13th Jun 2007
Location: Cheshire,UK
Posted: 14th Oct 2017 00:57
Another small update (see top post)

I noticed that SetTextFont() calls after LoadFont() still access the font file. This will need to be fixed for the function to properly be usable in WadLoader.
Some bugs fixed also (manifest lookup)
nz0
AGK Developer
17
Years of Service
User Offline
Joined: 13th Jun 2007
Location: Cheshire,UK
Posted: 2nd Nov 2017 00:40
Probably not a WIP anymore. It does everything I can think of now, but I don't know if there's somewhere this can go so it's available to all?
Certainly, I'll keep updating it, but not sure if forum mods can suggest where this should live now?
Supertino
7
Years of Service
User Offline
Joined: 22nd Jun 2017
Location: Behind you!
janbo
16
Years of Service
User Offline
Joined: 10th Nov 2008
Location: Germany
Posted: 5th Nov 2017 20:16 Edited at: 5th Nov 2017 20:31
@nz0 You mixed the Vertex and PixelShader String parameter from the Wad_LoadShader function ?
It's not very intuitive to have them the other way around
Needed a few minutes to realize where the error was.
nz0
AGK Developer
17
Years of Service
User Offline
Joined: 13th Jun 2007
Location: Cheshire,UK
Posted: 16th Nov 2017 10:49
Yes, I'd already fixed that, but wasn't enough to make another release. I was hoping the LoadFont() issue was solved before I release the next version
janbo
16
Years of Service
User Offline
Joined: 10th Nov 2008
Location: Germany
Posted: 16th Nov 2017 16:21
I now used your lib a bit and I like it.
While trying to understand what it does, I stumbled over the line: if GetFileSize(_t)>0 inside Wad_Extract function.
But you can get the size of the file from the memblock itself, so you don't need to use GetFileSize().
I don't like the fact that it creates the files to read them back in again.
You need a system that utilizes Create Sound/Image FromMemblock or directly read the string for shaders, so it doesn't need to create files.
nz0
AGK Developer
17
Years of Service
User Offline
Joined: 13th Jun 2007
Location: Cheshire,UK
Posted: 17th Nov 2017 00:19
I think it's mainly down to the zipping of the data. There's no unzip to memblock which i can use.
I could probably do it with a plugin (to memory) but no cross platform then
janbo
16
Years of Service
User Offline
Joined: 10th Nov 2008
Location: Germany
Posted: 19th Nov 2017 18:35 Edited at: 19th Nov 2017 22:32
A unzip to memory would be a great feature request
Do you need the zipping or can you just wrap the files into one encoded file written byte for byte.
Actually I made a lib for that myself but I like the idea of one library everyone is using and contributing to.

Login to post a reply

Server time is: 2024-11-27 05:24:04
Your offset time is: 2024-11-27 05:24:04