Sorry your browser is not supported!

You are using an outdated browser that does not support modern web technologies, in order to use this site please update to a new browser.

Browsers supported include Chrome, FireFox, Safari, Opera, Internet Explorer 10+ or Microsoft Edge.

AppGameKit Classic Chat / Best way for transparent JPEG

Author
Message
Xaby
FPSC Reloaded TGC Backer
12
Years of Service
User Offline
Joined: 17th Apr 2007
Location: Berlin
Posted: 8th Feb 2019 09:50
What could be the best way to use transparent JPEGs?
Why JPEGs? If you have large images, photographies and such, PNG files need a long time to load and need much space on the disk. So JPEGs are the way to go. But! They have no transparency layer.
The idea is, to combine the benefits of both in some way.

First I tried to create a program that reads the transparency data out of an 24+8 Bit PNG and creates a JPEG and a PNG out of that. Maybe I don't need the memblocks Pixel by Pixel, maybe I could use it also with some Mask-Commands, don't know.
It is a prototype.

So the result has to be more tiny in space and better looking in visual terms than what you could get out of the PNG optimised directly from tinypng or such programs. And in some way, this does.




So the question is: Is that the best way? Could it be in some way faster? Or Would it better, to slice big pictures in smaller parts, create more sprites and have only transparency on the edges? So maybe we could have a JPEG in the middle, but at the corner, there are some PNGs with transparency saved in it or so.

And how much would the performance suffer from this? Maybe there is a program, that does, what I tried to do. Maybe it does it much better

For what would that be a use case? For animations with real people, or for unique visuals not only created out of tiles.

Attachments

Login to view attachments
puzzler2018
User Banned
Posted: 8th Feb 2019 09:59
Excellent work here - ill take a good look a little later after work.
Xaby
FPSC Reloaded TGC Backer
12
Years of Service
User Offline
Joined: 17th Apr 2007
Location: Berlin
Posted: 22nd Feb 2019 12:41
@puzzler2018

could you test it, or had any idea, to improve it maybe?

I am thinking, if it would be possible, to have a bunch of unique pictures and layers, maybe in Photoshop or InkScape or somethink like that, build the complete level and be able, to load it in like a Stream.

[B] [B] [B]
[B] [X] [B]
[B] [B] [B]

For Full HD we would need an image buffer about (1920*1080*9*4)/1024/1024 ~ 72 MB. + the Assets we would put onto it to create the 9 times FullHD image. But these images, we could maybe loadin, draw, and free from the memory.

Maybe a combination of tile-sets and big sprites or so has to be created to use not so much frame buffer or memory at all.

fubarpk
Moderator
14
Years of Service
Recently Online
Joined: 11th Jan 2005
Location: Adelaide
Posted: 22nd Feb 2019 16:27
That's great @Xaby
but couldn't you reserve a colour for transparency ie black
and use SetImageTransparentColor(img,0,0,0) that way wouldn't
have to save again as png
fubar
Xaby
FPSC Reloaded TGC Backer
12
Years of Service
User Offline
Joined: 17th Apr 2007
Location: Berlin
Posted: 22nd Feb 2019 17:44 Edited at: 22nd Feb 2019 17:45
@fubarpk

with this method, I have alpha transparency 0..255

So for edges it could be needed, to have not only color or transparency.
And only the JPEG would not work, because JPEG colors are compressed. The problem here would be, that I could not use black in the image itself, and if I would use e.g. magenta, the problem would be, that it could be magenta, after compression, but only maybe

The number of colors is not 24 Bit. So depending on the image itself, every image would be maybe have a 8x8 block with our needed color inside or not.

Attachments

Login to view attachments
fubarpk
Moderator
14
Years of Service
Recently Online
Joined: 11th Jan 2005
Location: Adelaide
Posted: 22nd Feb 2019 18:02
Fair enough just thought having to save as png and then reload might've saved space but not loading times
as html is bad for that and I seem to recall something about html but I have probably misunderstood but
still a great piece of code
fubar
Xaby
FPSC Reloaded TGC Backer
12
Years of Service
User Offline
Joined: 17th Apr 2007
Location: Berlin
Posted: 22nd Feb 2019 19:22
The question is, if we would save our PNGs to JPEG + Alpha, and instead of PNG files, load only the new "Format", would we save space and loading time?

Maybe all is good, like it is and we would PNG only for pictures with some colors and photographies without transparency are still JPEG.

With that "code" we could also test, if there is really transparency in a picture. I tried to convert the old DDS (DXT5) files to PNG and than split them to JPEG files and PNG files. Because AppGameKit could not load DDS itself.
The split was JPEG for textures without transparency and texture with transparency are saved as PNG.

And with something like TinyPNG or RIOT oder SuperPNG and so on, they could be compressed some more.

The biggest problems are still LOADING-Time and MEMORY usage for the Image data itself. I know, maybe with Vulkan and App Game Kit Studio all that could be solved with ASTC format for textures, but not so with an Open GL ES 2.0 renderer.

It would be also possible to have JPEG + JPEG, 24 Bit JPEG + 8 Bit Grayscale JPEG for Alpha. But I guess, it would harm the picture more, than it helps.

Also JPEGs 8x8 blocks need more space even if they are the same in the picture itself. If you have an animation e.g. with side by side pictures, let's say 256 x 256 on a 4096 x 4096 texture, and the 256 x 256 pictures look nearly the same, the JPEG would have to save every 8x8 block for itself,
even if this block will be there about 256 times.

4096 x 4096 ~ 16 Megapixel (64 MB in memory)
256 x 256 ~ 64 kiloPixel (256 kByte in memory)

4096 / 256 = 16

so 16 x 16 x (256 x 256) = 16 Megapixel, or 256 textures with a size of 256 x 256 are the same as one texture 4K x 4K

So on one hand it is better, to have textures combined to one texture, it loads faster and so on. But JPEG helps not much with blocks, that look nearly the same.





janbo
10
Years of Service
User Offline
Joined: 10th Nov 2008
Location: Germany
Posted: 23rd Feb 2019 12:02 Edited at: 23rd Feb 2019 12:12
Hm how about creating your own format ?
Or maybe create a shader that takes RGB from one JPEG texture and the Alpha value from the Red channel of a second JPEG so you can get rid of those png's
But you still have 2 channel's left. Well you could just create 3 shaders for extracting the alpha of a sprite from one of the channels, but than you'll probably want to make an editor too...
You could go a step further and encode only the alpha changes instead of the actual values for each row, so you are left with much more space in your misused alpha mask JPEG texture monster thing
Also take a look at SetImageMask()
Xaby
FPSC Reloaded TGC Backer
12
Years of Service
User Offline
Joined: 17th Apr 2007
Location: Berlin
Posted: 26th Feb 2019 19:57
We could make a combination of 3 JPEGs + 3 ALPHAS, so our texture would be more like 2 x 2, so 4 JPEGs combined or one "big" JPEG.
Santman
7
Years of Service
User Offline
Joined: 15th Sep 2011
Location: Inverness
Posted: 26th Feb 2019 21:55
I'm still confused.

What's wrong with PNGs?
blink0k
AGK Developer
Gold Codemaster
6
Years of Service
User Offline
Joined: 22nd Feb 2013
Location: the land of oz
Posted: 26th Feb 2019 22:00
Quote: "What's wrong with PNGs?"


Nothing other than they can be quite large in size compared to .jpgs
I know adobe flash had a transparent .jpg format
Santman
7
Years of Service
User Offline
Joined: 15th Sep 2011
Location: Inverness
Posted: 26th Feb 2019 22:31
Hmmm, but jpegs are only smaller as they compress.....so if you're going for that level of quality, I'd personally just go for a png, if it needs transparency as well.

Try processing an image in a memblock, then save it in a JPEG.....not the same image at all.
puzzler2018
User Banned
Posted: 26th Feb 2019 22:35 Edited at: 26th Feb 2019 22:37
Can we load in or save as JPGs then - i didnt know

But still will be the same memory consumption.

The memblock that derives for any loaded image will still be the same construction for AppGameKit for working with.

Tell me im wrong
Tier 1 Developer
Ortu
DBPro Master
11
Years of Service
User Offline
Joined: 21st Nov 2007
Location: Austin, TX
Posted: 26th Feb 2019 22:39
If file size is the concern, why not just use png with the zip commands?

https://www.appgamekit.com/documentation/Reference/File/ExtractZip.htm

Extract file, load to memory, delete extracted file
http://games.joshkirklin.com/sulium

A single player RPG featuring a branching, player driven storyline of meaningful choices and multiple endings alongside challenging active combat and intelligent AI.
Santman
7
Years of Service
User Offline
Joined: 15th Sep 2011
Location: Inverness
Posted: 26th Feb 2019 22:41
Oh and for the record, PNGs don't load much slower than jpegs in a fair amount of testing I Did.
puzzler2018
User Banned
Posted: 26th Feb 2019 22:45
Lets calm the opnions on what things do and head back to what Xaby really needs to acheive. otherwise be forever going round in circles and never getting anywhere lol
Tier 1 Developer
Xaby
FPSC Reloaded TGC Backer
12
Years of Service
User Offline
Joined: 17th Apr 2007
Location: Berlin
Posted: 27th Feb 2019 00:21 Edited at: 27th Feb 2019 00:31
PNG is nearly the same as ZIP. The thing is, JPEG can be decoded in hardware from ARM based microconsoles, PNG is slow as hell.

You could try, testing 4K by 4K PNGs and JPEGs, and maybe it feels not like much on a PC with power, but on a low end laptop, the difference is about 10 sec. for the PNGs and maybe only 2 sec. for the JPEG. Depends on how many textures you have to unpack / decode.

@blink0k Yes! I know, that Game Maker Studio don't understand JPEG, but PNG, but also SWFs with transparent JPEGs in them. So it is funny and a workaround for saving file space.
http://www.swftools.org/about.html

http://yowindow.com/png_to_swf.php (result for download)

Maybe we could decode this kind of SWF. Maybe it is also only a ZIP with PNG and JPEG inside or so.

I tried the PNG and used 70% JPEG compression, the resulted SWF is here (you can view the SWF with IrfanView)

Attachments

Login to view attachments
Xaby
FPSC Reloaded TGC Backer
12
Years of Service
User Offline
Joined: 17th Apr 2007
Location: Berlin
Posted: 27th Feb 2019 00:37
It would be also nice, if we could STREAM JPEG or PNGs into an decoder / encoder.
https://github.com/Gizeta/swf-image-extractor/blob/master/src/extractor.js

I mean, if we would have a JPEG-stream, in memory, and we could say: JPEGfromMemblockToImage or something like that. At this moment we need to save a memblock as Image, create a Image-JPEG file and can only load this file. But what if, we would have 10 JPEG files one after another in one file, we would have to create 10 seperate files first and load them again into memory.
Santman
7
Years of Service
User Offline
Joined: 15th Sep 2011
Location: Inverness
Posted: 27th Feb 2019 05:21
I did a lot of work on this thread, check it out - though even on mobile I didn't see that kind of slow down with a png.

https://forum.thegamecreators.com/thread/219750

So basically, is file size an issue? If not then memblocks are many times faster than either image type, but are obviously much larger than either too.

If your going for speed and small size, then I'd probably go with what Janbo suggested and use two jpegs and a simple customs shader....but again, the JPEG compression could be your issue, and probably part of the reason why they don't have transparency in the first place...decompression tends to create a blur, so your transparency may not be accurate. You could use the lowest compression, but then the file size can be equal to png almost.

Or you could combine the two - store ONLY the alpha in a memblock or text file or something, then load that and the JPEG and combine them in a memblock, but it may not be much faster or even slowest then just loading a png.
fubarpk
Moderator
14
Years of Service
Recently Online
Joined: 11th Jan 2005
Location: Adelaide
Posted: 27th Feb 2019 07:49 Edited at: 27th Feb 2019 07:51
png has these save options
1>interlacing
2>save background color
3>save gamma
4>save layer offset
5>save resolution
6>save creation time
7>save comment
8>save color values from transparent pixels
9>pixel format (ie grayscale rgb etc)
10>Compression level
11>save exif data
12>save xmp data
13>save jptc data
14>save thumbnail

jpg has the following save options
1>Quality
2>Optimize ....smoothing
3>Progressive
4>Save exif data
5>Save thumbnail
6>Save xmp data
7>Save IPTC data
8>Use quality settings from original image
9>Save Comment

depending on how you are saving your images in respect to the above settings a png can be the same size as a jpg
it just has the possibility of more save options. It can even be smaller than a jpg and load much faster
fubar
Xaby
FPSC Reloaded TGC Backer
12
Years of Service
User Offline
Joined: 17th Apr 2007
Location: Berlin
Posted: 27th Feb 2019 09:06 Edited at: 27th Feb 2019 12:21
@Santman

in your post, you also stated, that your 50 PNGs needed more time to decode and load instead of the 50 JPEGs. Also you use a 8 GB RAM i3 computer, maybe with Windows optimising something in the background to boost the same files when loading. Some Intel Chips, I guess, can do JPEG in hardware decoding.

So I am aiming more for the Raspberry Pi. The best solution for games would be ASTC, its the best combination of compression and fastest loading times. Because it is compressed memory. It would be faster then only memblocks, because the same dimensions of pixels needed less memory, so the file would be much smaller.
Also no extra decompression would be required. But!!! The Raspberry Pi only supports OpenGL ES 2.0 and there for it only supports ETC1.

On low end Smartphones, or TV-Sticks like Amazon FireTV stick, there is not much space for apps. Also the Android App Store only supports 100 MB for apps in an easy way. And we also can't tell, if we would have enough space to decompile all assets first to the disk or flash drive first. So a memory dump would be only possible, if we had really much space.

JPEG could be also lossless. Mostly we are using the JIFF compression, which can be blurry sometimes.

If we are talking about memblock-file-loadings, maybe we would have to create something like an "tileset" and reconstruct the image into a memblock while loading the memblock.

Maybe we would end with something like that:
Santman
7
Years of Service
User Offline
Joined: 15th Sep 2011
Location: Inverness
Posted: 27th Feb 2019 10:12
That's a fascinating video......I'm quite glad were not generally as limited.

I think a lot of that work is already done though by image compression techniques in JPEG and png, and others. Without knowing in detail what your project is, I can't comment on whether a tiling system would work. My thread focussed more on loading images faster, and in streaming them in the background.....which clearly resulted in massive storage implications. I dont think a modern 100gb game uses all compressed assets but probably a mix of both and custom compression and threaded decompression.

In my current project I use atlas tiles for landscape textures, which are controlled by mask images.....that allows me to "paint" completely unique looking landscapes with no obvious tiling element, but in terms of it helping loading speed and file size I'm not sure it would help, though you could cut a memblock into chunks.

Either way, a memblock is a completely uncompressed memory dump so there's unlikely to be a faster method of loading data, assuming the storage media keeps up. In my thread, I saw little difference between a powerful desktop and a mid range phone, so how AppGameKit does things internally also becomes a factor potentially.
Santman
7
Years of Service
User Offline
Joined: 15th Sep 2011
Location: Inverness
Posted: 27th Feb 2019 10:16
Going back to your top post, you could however do your idea of splitting up the image into transparent and non transparent parts, then saving non transparent as a JPEG then transparent as a memblock. That would probably be fairly quick to load, but again use space.

Overall, I'd stick with Janbo and say two jpegs per image, the second using a colour channel to store the alpha (It's all just data to a gpu) and a custom shader. But that's still loading decoding two images for every transparent image, so I'm not convinced it would be faster or better for space than a png, and will use twice the memory when loaded into memory.

Login to post a reply

Server time is: 2019-04-23 20:02:51
Your offset time is: 2019-04-23 20:02:51