Using SetRenderToImage() for Higher resolution and faster VR?

Author

Message

Stab in the Dark software

Valued Member

21

Years of Service

User Offline

Joined: 12th Dec 2002

Playing: Badges, I don't need no stinkin badges

Posted: 4th Jun 2017 16:22 Edited at: 4th Jun 2017 16:53

Link

Has anyone had success using CreateRenderImage() and SetRenderToImage() commands to apply
the image to an object? I know that it works when applying to a sprite. What I am trying to accomplish
is creating a faster way to render VR on a mobile device. This method is using 2 object planes that are
distorted with a barrel vertex shader. Then I need to apply the render image to the object planes.
My testing so far shows that SetRenderToImage() does not work when the imagine is on an object.
The documentation says that it is supposed to work for an object. I think this may be a bug with
SetRenderToImage() when used with objects so if any one has an example of it working on an object
please post. If it would work I could post a complete example for a faster and higher resolution for VR.

I am basing this method on this article.
https://www.imgtec.com/blog/speeding-up-gpu-barrel-distortion-correction-in-mobile-vr/

Here is just a basic layout of the code.

+ Code Snippet

vr_LeftMesh = LoadObject( "objects/VR Mesh.x" )
SetObjectShader( vr_LeftMesh, vr_ShadersID )
vr_LeftRenderImageID = CreateRenderImage( GetVirtualWidth(), GetVirtualHeight(), 0,0 )

SetObjectImage( vr_LeftMesh, vr_LeftRenderImageID, 0 )

do
    Print( ScreenFPS() )
  
    VR_Update()
loop

function VR_Update()
	
    Update(0)
    SetCameraPosition(1, 1850, 560,-3650 )
    SetCameraLookAt( 1, 0.0, 0.0, 0.0, 0.0 )

    SetRenderToImage(vr_LeftRenderImageID, 0)
    ClearScreen()
    Render()
	
    SetRenderToScreen()
    Swap() 
    
    SetCameraPosition ( 1, 0.0, 0.0, -35.0 )
    SetCameraLookAt( 1, 0.0, 0.0, 0.0, 0.0 )	
endfunction

The coffee is lovely dark and deep,and I have code to write before I sleep.

Back to top

Profile PM Email Website

MikeMax

AGK Academic Backer

12

Years of Service

User Offline

Joined: 13th Dec 2011

Location: Paris

Posted: 4th Jun 2017 17:28

Link

have you tried :

+ Code Snippet

SetRenderToImage(vr_LeftRenderImageID, -1) // -1 instead of 0

--------------------------------
Join us on dedicated AppGameKit WeeKChat :

https://week.chat/room/AppGameKit

Back to top

Profile PM Email

Stab in the Dark software

Valued Member

21

Years of Service

User Offline

Joined: 12th Dec 2002

Playing: Badges, I don't need no stinkin badges

Posted: 4th Jun 2017 18:08

Link

Yes I have it still does not work.

The coffee is lovely dark and deep,and I have code to write before I sleep.

Back to top

Profile PM Email Website

MikeMax

AGK Academic Backer

12

Years of Service

User Offline

Joined: 13th Dec 2011

Location: Paris

Posted: 4th Jun 2017 18:21 Edited at: 4th Jun 2017 18:22

Link

are you sure about the UVs of your mesh ?

--------------------------------
Join us on dedicated AppGameKit WeeKChat :

https://week.chat/room/AppGameKit

Back to top

Profile PM Email

Stab in the Dark software

Valued Member

21

Years of Service

User Offline

Joined: 12th Dec 2002

Playing: Badges, I don't need no stinkin badges

Posted: 4th Jun 2017 19:50

Link

Yes, I can get it to work with the GetImage() command but it is too slow as it not recommended to be called every frame.

Quote: "Calling GetImage is a slow command and it is not recommended that it be called every frame."

The coffee is lovely dark and deep,and I have code to write before I sleep.

Back to top

Profile PM Email Website

MikeMax

AGK Academic Backer

12

Years of Service

User Offline

Joined: 13th Dec 2011

Location: Paris

Posted: 4th Jun 2017 20:45 Edited at: 4th Jun 2017 20:45

Link

Very strange.

For my own VR experiences i prefered to use a fullscreen shader for barrel distortion on 2 rendered sprite (leftEye, RightEye) which produces some good performances :

I wanted to release a VR example with my home-made Lib but for the moment i'm on my Multiplayer Linux server

--------------------------------
Join us on dedicated AppGameKit WeeKChat :

https://week.chat/room/AppGameKit

Attachments

Login to view attachments

Back to top

Profile PM Email

Stab in the Dark software

Valued Member

21

Years of Service

User Offline

Joined: 12th Dec 2002

Playing: Badges, I don't need no stinkin badges

Posted: 5th Jun 2017 20:11

Link

Hopefully Paul will look into this since it seems to be a bug.

The coffee is lovely dark and deep,and I have code to write before I sleep.

Back to top

Profile PM Email Website

Paul Johnston

TGC Developer

21

Years of Service

User Offline

Joined: 16th Nov 2002

Location: United Kingdom

Posted: 5th Jun 2017 21:26

Link

AGK comes with an example called RenderToImage that will use a rendered image on a 3D object, so it should work. It may be because the object still has the image attached when Render is called, which will try to draw the object to the image it is currently using. Something like this may work

+ Code Snippet

function VR_Update()
     
    Update(0)
    SetCameraPosition(1, 1850, 560,-3650 )
    SetCameraLookAt( 1, 0.0, 0.0, 0.0, 0.0 )

    SetObjectVisible(vr_LeftMesh,0) 
    SetRenderToImage(vr_LeftRenderImageID, 0)
    ClearScreen()
    Render()
     
    SetRenderToScreen()
    SetObjectVisible(vr_LeftMesh,1)
    DrawObject(vr_LeftMesh)
    Swap() 
     
    SetCameraPosition ( 1, 0.0, 0.0, -35.0 )
    SetCameraLookAt( 1, 0.0, 0.0, 0.0, 0.0 )    
endfunction

Back to top

Profile PM Email

Stab in the Dark software

Valued Member

21

Years of Service

User Offline

Joined: 12th Dec 2002

Playing: Badges, I don't need no stinkin badges

Posted: 6th Jun 2017 15:28 Edited at: 6th Jun 2017 15:30

Link

Thanks for your insight Paul, I was unaware of the demo.
I examined the demo and discovered what I was doing wrong.
The demo does not move the camera like I am so I had to move the
SetCameraPosition() call to before the Swap() and add another Render().
I can now continue with my development on this method. Now I just need to
properly deform a mesh in a modeling program to match the distortion of the
lenses in the VR headset.

+ Code Snippet

function VR_Update()
    Update(0)
    SetCameraPosition(1, 1850, 560,-3650 )
    SetCameraLookAt( 1, 0.0, 0.0, 0.0, 0.0 )
 
    SetRenderToImage(vr_LeftRenderImageID, -1)
    ClearScreen()
    Render()
  
    SetRenderToScreen()
    SetCameraPosition ( 1, 0.0, 0.0, -35.0 )
    SetCameraLookAt( 1, 0.0, 0.0, 0.0, 0.0 )    
    Render()
    Swap() 
endfunction

The coffee is lovely dark and deep,and I have code to write before I sleep.

Back to top

Profile PM Email Website

Stab in the Dark software

Valued Member

21

Years of Service

User Offline

Joined: 12th Dec 2002

Playing: Badges, I don't need no stinkin badges

Posted: 8th Jun 2017 19:08

Link

I have made some progress, I have been able to correctly distort a mesh using Wings3D.
I now have it properly rendering the image on the mesh. Unfortunately I have run into a glitch.

I need to change the aspect ratio of the camera before I render to the image and then change it back to normal afterwards.
It seems that in AppGameKit I can not change the aspect ratio in the loop. If I do not change the aspect ratio it results in a distorted
image when the camera is rotated on the roll axis. See attached images. Two solutions could solve this, more than one camera
or being able to change the aspect ratio. In my opinion AppGameKit should have more then camera especially since all the commands
actually take a camera number already. This pre distorted mesh method is better and faster than the barrel pixel shader method
or my original testing using a barrel vertex shader to distort the mesh.

I base this on my testing results and the information from this article.
http://smus.com/vr-lens-distortion/

Paul, should it be possible to change the aspect ratio like this?

function VR_Update()
	
	SetCameraRotationQuat(1,GetRawRotationVectorW2(),GetRawRotationVectorX2(),GetRawRotationVectorY2(),GetRawRotationVectorZ2())

Update(0)
	SetCameraFOV(1,90.0) //Set a larger field of view for the render grab
	SetCameraAspect( 1, ( GetVirtualWidth() *0.5 ) / GetVirtualHeight() )
    SetCameraPosition(1, 2000.0, 560.0, -3250.0 )
	SetRenderToImage(vr_RenderImageID, -1)
	ClearScreen()
	Render()

SetRenderToScreen()

SetCameraFOV(1,70.0) //Set the field of view back to normal for the viewing of the VR render object planes
	SetCameraAspect( 1, GetVirtualWidth() / GetVirtualHeight() )
	SetCameraPosition ( 1, 0.0, 0.0, -97.5 ) //Set the camera back to location where VR render object planes are located.
	SetCameraLookAt( 1, 0.0, 0.0, 0.0, 0.0 )	//Make sure the camera looks at VR render object planes squarely.

//Draws the center line on screen
	drawline( getvirtualwidth() / 2 - 3, 0, getvirtualwidth() / 2 - 3, GetVirtualHeight(), 255, 0, 0)	
	drawline( getvirtualwidth() / 2, 0, getvirtualwidth() / 2, GetVirtualHeight(), 255, 255, 255)
	drawline( getvirtualwidth() / 2 + 3, 0, getvirtualwidth() / 2 + 3, GetVirtualHeight(), 0, 255, 0)

Render()
	Swap() 
endfunction

+ Code Snippet

function VR_Update()
	
	SetCameraRotationQuat(1,GetRawRotationVectorW2(),GetRawRotationVectorX2(),GetRawRotationVectorY2(),GetRawRotationVectorZ2())

	Update(0)
	SetCameraFOV(1,90.0) //Set a larger field of view for the render grab
	SetCameraAspect( 1, ( GetVirtualWidth() *0.5 ) / GetVirtualHeight() )
    SetCameraPosition(1, 2000.0, 560.0, -3250.0 )
	SetRenderToImage(vr_RenderImageID, -1)
	ClearScreen()
	Render()

    SetRenderToScreen()

	SetCameraFOV(1,70.0) //Set the field of view back to normal for the viewing of the VR render object planes
	SetCameraAspect( 1, GetVirtualWidth() / GetVirtualHeight() )
	SetCameraPosition ( 1, 0.0, 0.0, -97.5 ) //Set the camera back to location where VR render object planes are located.
	SetCameraLookAt( 1, 0.0, 0.0, 0.0, 0.0 )	//Make sure the camera looks at VR render object planes squarely.

	//Draws the center line on screen
	drawline( getvirtualwidth() / 2 - 3, 0, getvirtualwidth() / 2 - 3, GetVirtualHeight(), 255, 0, 0)	
	drawline( getvirtualwidth() / 2, 0, getvirtualwidth() / 2, GetVirtualHeight(), 255, 255, 255)
	drawline( getvirtualwidth() / 2 + 3, 0, getvirtualwidth() / 2 + 3, GetVirtualHeight(), 0, 255, 0)

	Render()
	Swap() 
endfunction

The coffee is lovely dark and deep,and I have code to write before I sleep.

Attachments

Login to view attachments

Back to top

Profile PM Email Website

Stab in the Dark software

Valued Member

21

Years of Service

User Offline

Joined: 12th Dec 2002

Playing: Badges, I don't need no stinkin badges

Posted: 8th Jun 2017 21:08

Link

It seems that you can change the aspect ratio in in the loop, AppGameKit did not like these lines.

+ Code Snippet

SetCameraAspect( 1, ( GetVirtualWidth() *0.5 ) / GetVirtualHeight() ) 
SetCameraAspect( 1, GetVirtualWidth() / GetVirtualHeight() )

It needed to be like this.

+ Code Snippet

SetCameraAspect( 1, 0.888888 )
SetCameraAspect( 1, 1.7777777 )

I will post the complete demo as soon as I get it cleaned up.
I am getting very good results on an LG Stylo 3 with a Tzumi Dream Vision headset.

The coffee is lovely dark and deep,and I have code to write before I sleep.

Back to top

Profile PM Email Website

Sorry your browser is not supported!

AppGameKit Classic Chat / Using SetRenderToImage() for Higher resolution and faster VR?

Attachments

Attachments