Sorry your browser is not supported!

You are using an outdated browser that does not support modern web technologies, in order to use this site please update to a new browser.

Browsers supported include Chrome, FireFox, Safari, Opera, Internet Explorer 10+ or Microsoft Edge.

AppGameKit Classic Chat / Multiple videos and transparent videos?

Author
Message
Xaby
FPSC Reloaded TGC Backer
17
Years of Service
User Offline
Joined: 17th Apr 2007
Location: Berlin
Posted: 14th Jun 2017 14:53 Edited at: 14th Jun 2017 16:17
Works in Chrome.

Demo for the result I want to get in AGK.
http://simpl.info/videoalpha/
(video : http://simpl.info/videoalpha/video/soccer1.webm )

Under Windows:
you may need: https://xiph.org/dshow/

For getting the color from the video I used "Just Color Picker"
http://annystudio.com/software/colorpicker/#jcp-download


My code so far:



I am missing some in AGK.

Why is it not: vidID = LoadVideo() and PlayVideo(vidID) or PlayVideoToImage(vidID, imgID)

Also WebM has Alpha. But I don't understand, why the Image (video) has the gray background.

It could be interessting for Point-and-click-adventures e.g.

Or do I somehow has to create two! videos!. 2 ... more than one! And use SetImageMask() to create something like that
https://jakearchibald.com/scratch/alphavid/

But how? If I only have one video?
Jack
20
Years of Service
User Offline
Joined: 4th Oct 2004
Location: [Germany]
Posted: 14th Jun 2017 18:15 Edited at: 14th Jun 2017 18:17
vidID = LoadVideo("soccer1.webm")

Nope, the LoadVideo breaks the standard logic. it does not return a vidID, you can only load one video at a time, so there is no video ID at all.

What you want is a special impementation.

Windows, Mac and Linux would work well with multiple alpha based videos. Mobile OS not.
For example the second video android has to handle will not be rendered on the GPU and has, depending on your resolution and bitrate, about 1-5 fps.
I am even not shure about alpha videos on mobile devices.

You can realize this in AppGameKit T1 by working out a seperate OpenGL Video command set.
Let me bet, you will already find your needed lib somewhere on github.
So the basic problem would be how to implement it into an AppGameKit plugin.

If you go for D3d with your plugin though, you will be bound to the windows platform,
so I would prefer to stay with OpenGL.

[/url]
Xaby
FPSC Reloaded TGC Backer
17
Years of Service
User Offline
Joined: 17th Apr 2007
Location: Berlin
Posted: 16th Jun 2017 13:00
@jack

Okay, I could live with one video at a time. And maybe transparent video will not work on iOS with AGK. But WebM is a Google format and Android 4.4 supports it.

The question would be, would it be possible, to render a transparent video to maybe a memblock or image -> memblock -> save ... and preserve the transparent color for maybe later use.

I think, I stuck with some PNG sprites. But I testet days back in 2011 with my iPod and Flash Professional some FLV and F4V videos with transparent backgrounds. WebM and F4V are both some Sorenson V7/V8/V9 codec thingly.

The problem with video in apps is streaming/or not streaming.

Maybe I could do a PNG/JPEG checkerboard combination with right timing and create my own "video-format"

Thanks for help.
Jack
20
Years of Service
User Offline
Joined: 4th Oct 2004
Location: [Germany]
Posted: 16th Jun 2017 16:06 Edited at: 16th Jun 2017 16:08
Quote: "The question would be, would it be possible, to render a transparent video to maybe a memblock or image -> memblock -> save ... and preserve the transparent color for maybe later use. "

Yes, it would. Or you could simply render your video out as a .png sequence and load them into a memblock.

Quote: "
Maybe I could do a PNG/JPEG checkerboard combination with right timing and create my own "video-format""


This is true, as you can request an image from your video and combine this image with a mask- sequence.
I would recommend to create a custom shader where the color (rgb) are taken from the video image and the alpha value is (r*g*b/3.) from the mask image.
Should be done in about two to four additional lines from the native AppGameKit shader example.

With a .PNG video sequence you would go faster.

[/url]
Xaby
FPSC Reloaded TGC Backer
17
Years of Service
User Offline
Joined: 17th Apr 2007
Location: Berlin
Posted: 25th Oct 2018 13:42 Edited at: 25th Oct 2018 13:42
Maybe we could figure it out together.



I wanted to have only the person on the video. I created this video with DaVinci Resolve, so we have the Alpha-Mask also at the same time in the same video.
But what next? Shader? Memblock-Tricks?

Attachments

Login to view attachments
fubarpk
Retired Moderator
19
Years of Service
User Offline
Joined: 11th Jan 2005
Playing: AGK is my friend
Posted: 25th Oct 2018 16:01 Edited at: 25th Oct 2018 16:05
In pixelated labrynth the flames are a video


code snippet the order is important
fubar
Xaby
FPSC Reloaded TGC Backer
17
Years of Service
User Offline
Joined: 17th Apr 2007
Location: Berlin
Posted: 25th Oct 2018 18:02
Definition
SetSpriteTransparency( iSpriteIndex, mode )

Parameters
iSpriteIndex - The ID of the sprite to modify.
mode - The transparency mode for this sprite, 0=off, 1=alpha transparency, 2=additive blending

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

If I use mode 2 like in your example, the person gets white, the blackborder is transparent.
If I am using mode 1, I see no difference. I used a color-picker and the black color is 0, 0, 0 in the video.

So, I don't get the effect I wanted.
CJB
Valued Member
20
Years of Service
User Offline
Joined: 10th Feb 2004
Location: Essex, UK
Posted: 25th Oct 2018 21:37 Edited at: 25th Oct 2018 21:38
Something like this (screenshot):



Attachments

Login to view attachments
Xaby
FPSC Reloaded TGC Backer
17
Years of Service
User Offline
Joined: 17th Apr 2007
Location: Berlin
Posted: 26th Oct 2018 09:50
@CJB,

you are my hero. I thought about something like that, but didn't knew the commands. Was looking for "Blending" and "Texture Masking"
CJB
Valued Member
20
Years of Service
User Offline
Joined: 10th Feb 2004
Location: Essex, UK
Posted: 26th Oct 2018 13:27
It's not a very efficient way of doing things as you are re-creating a new image mask every frame, but it gets the job done.
Xaby
FPSC Reloaded TGC Backer
17
Years of Service
User Offline
Joined: 17th Apr 2007
Location: Berlin
Posted: 26th Oct 2018 15:16
@CJB,

Could there be a better solution? I read that "This is a slow command and should not be called every frame."
But every frame the image mask would be another one.

And I guess, they also didn't do that realtime


I also was looking into that a little bit, handling MemBlocks, but I guess, it would also not faster
https://forum.thegamecreators.com/thread/213539

So what would be a more elegant solution, if money doesn't matter? But we still use AppGameKit I guess, I would 3D-scan the person, rig the person, have a low polycount, bake the textures and so on and make motion caputure for the same movement ...
blink0k
Moderator
11
Years of Service
User Offline
Joined: 22nd Feb 2013
Location: the land of oz
Posted: 26th Oct 2018 21:07
I think a shader is the way to go with this.
CJB
Valued Member
20
Years of Service
User Offline
Joined: 10th Feb 2004
Location: Essex, UK
Posted: 27th Oct 2018 13:10
Good idea! Here's my first attempt:

alphavid.ps:


agk code:


Set _cutoff_level to the brightness level required of each pixel (in the range 0.0 to 1.0 (0 = rgb(0,0,0), 1 = rgb(255,255,255)) - the shader will only render pixels that have their r, g and b values above this level, otherwise the pixel is discarded.

Attachments

Login to view attachments
CJB
Valued Member
20
Years of Service
User Offline
Joined: 10th Feb 2004
Location: Essex, UK
Posted: 27th Oct 2018 17:35 Edited at: 27th Oct 2018 17:37
With a little tweak, it now blends in the dark areas by adding alpha inversely to the level of darkness below the _cutoff_level. Makes the edges smoother:

shader:


agk example:


We could use this same technique to create a chroma-keying shader

Attachments

Login to view attachments
Jack
20
Years of Service
User Offline
Joined: 4th Oct 2004
Location: [Germany]
Posted: 28th Oct 2018 03:15
Xaby
FPSC Reloaded TGC Backer
17
Years of Service
User Offline
Joined: 17th Apr 2007
Location: Berlin
Posted: 16th Nov 2018 10:40
Could this method for Video-streaming "make" sense also for static JPEGs or could we use this for maybe Sprite-Animations as well, but with JPEG instead of PNG or would it loss its benefits?

I am often thinking about texture compression on 3D models and at this point I seperated transparent textures from non-transparent and compressed the non-transparend as JPEG (because no ATSC or DXT or so) and the transparent ones as PNGs.
In Memory, I guess, it's the same later, but PNG is also very slow in decompression.
Xaby
FPSC Reloaded TGC Backer
17
Years of Service
User Offline
Joined: 17th Apr 2007
Location: Berlin
Posted: 23rd Nov 2019 01:43
@CJB

your shader is the way to go. Tested it over a year later on the OUYA. And with the masking method I had under 10 FPS, with the shader thingly ~ 60+ FPS, can't be more accurate.
What I saw is, that there are some washed out hair. But not on the white mask.

But!, you are my hero. And man, how fast a year passed by.
Xaby
FPSC Reloaded TGC Backer
17
Years of Service
User Offline
Joined: 17th Apr 2007
Location: Berlin
Posted: 23rd Nov 2019 14:11 Edited at: 23rd Nov 2019 14:11
How can I use a pixel-shader on an 3D object?

fubarpk
Retired Moderator
19
Years of Service
User Offline
Joined: 11th Jan 2005
Playing: AGK is my friend
Posted: 23rd Nov 2019 17:33
the ps shader may need modification
and you would need a vs shader
fubarpk
fubarpk on Itch...………...https://fubarpk.itch.io/
fubarpk on googleplay..https://play.google.com/store/apps/developer?id=fubarpk
Xaby
FPSC Reloaded TGC Backer
17
Years of Service
User Offline
Joined: 17th Apr 2007
Location: Berlin
Posted: 25th Nov 2019 13:07
@fubarpk,

yes, but how

so how would my "empty" VS shader look like?
Bengismo
7
Years of Service
User Offline
Joined: 20th Nov 2017
Location: Yorkshire, England
Posted: 25th Nov 2019 13:14 Edited at: 25th Nov 2019 13:14
https://www.appgamekit.com/documentation/guides/13_shaders.htm <--Its in the documentation

....including standard "empty" 3D vertex shader
Xaby
FPSC Reloaded TGC Backer
17
Years of Service
User Offline
Joined: 17th Apr 2007
Location: Berlin
Posted: 30th Nov 2019 19:45
@Bengismo, I still don't get it.

My video on the plane is not seen anymore.


Shaders:

"Standard2D.vs"


"alphavid.ps"


I tried the shaders from the help, but most of them promted me with an error. The only one, that I got to work, is the one, I posted here.
fubarpk
Retired Moderator
19
Years of Service
User Offline
Joined: 11th Jan 2005
Playing: AGK is my friend
Posted: 30th Nov 2019 20:18 Edited at: 30th Nov 2019 20:27
To get to work with sprites you have to load a sprite shader which is just the ps shader seems to work
to get to work with obects you load both the vs and ps shader altho I got it visible I couldn't get it to
duplicate

standard vs shader

ps shader


fubarpk
fubarpk on Itch...………...https://fubarpk.itch.io/
fubarpk on googleplay..https://play.google.com/store/apps/developer?id=fubarpk
Xaby
FPSC Reloaded TGC Backer
17
Years of Service
User Offline
Joined: 17th Apr 2007
Location: Berlin
Posted: 30th Nov 2019 23:25
@fubarpk

I still get this error:
---------------------------
Message
---------------------------
Error: Pixel shader "alphavid.ps" has varying "vec4 colorVarying;" that does not exist in the vertex shader in main.agc at line 67
---------------------------
OK
---------------------------
I changed the names to mine, but used your code for vs and ps and my video-file.

I added the line "varying vec4 colorVarying;" to the vs. But I guess, something has to be done with the transparent effect. Setting objects to some blending mode maybe.

Or maybe using the shader flips the UV coordinates and we are looking from behind to the object now?
Bengismo
7
Years of Service
User Offline
Joined: 20th Nov 2017
Location: Yorkshire, England
Posted: 1st Dec 2019 17:41 Edited at: 1st Dec 2019 17:43
If you have a pixel shader that uses a "varying" variable then it must be passed from the vertex shader otherwise it has no value to work with.

As its an 3D object that being drawn, the colour varying is typically taken from the built in uniform value agk_MeshDiffuse, which is basically the objects colour set by SetObjectColor(). Its doubtful that you want the object colour though right now anyway.

If you set a uniform transparent colour then calculate how far away the colour you are rendering is away from that colour then you can set any colour to be transparent and also set a level of tolerance.



In this example I have made the clear colur purple to show the masking in action. The video is the standard one that comes with ADGK in the projects driectory.

A slightly more advanced technique is to set a blend zone so the edges can be blurred between masked an not....
Xaby
FPSC Reloaded TGC Backer
17
Years of Service
User Offline
Joined: 17th Apr 2007
Location: Berlin
Posted: 2nd Dec 2019 08:29
@Bengismo

your result looks like the result I wanted to achieve.
At the moment I look at the shader code and it feels like an illiterate. Maybe I overtired or it is because of Christmas.

How do I get "agk_MeshDiffuse" from one shader to the other? I am glad, that I understand AppGameKit in the basics. The last bug costed me about some days in my project to figure out, that it was not my fault and to figure out a workaround.
I guess, I can't be a specialist in all parts of effects, programming, and so on.

So in the AppGameKit code you also used nearly the same VS and PS like @fubarpk and so I?

And the next question is, so if I am using this shader-thingly, I would lose the option to use SetObjectColor() and would have to build a thingly for changing diffuse color of the object?
fubarpk
Retired Moderator
19
Years of Service
User Offline
Joined: 11th Jan 2005
Playing: AGK is my friend
Posted: 2nd Dec 2019 08:48
It really depends on what ya need might have something like this
depending on the one your using


but im very much a noob to shaders but think it would be better if you made it not to set the colors
fubarpk
fubarpk on Itch...………...https://fubarpk.itch.io/
fubarpk on googleplay..https://play.google.com/store/apps/developer?id=fubarpk

Login to post a reply

Server time is: 2024-11-22 04:53:21
Your offset time is: 2024-11-22 04:53:21