Alright!
There is actually some progress:
the red camera is the original sensor fusion from android passed to AppGameKit as "GetRawRotationVectorW2, X2, Y2, Z2", the green camera represents the custom rotation.
As you can see, the new method will eliminate the whole wobble and can even help to reduce gyro drift, if the phone is shaking. There are still some small bugs in the quaternion calculation code it will result in a failed frustrum culling so objects that are actually on screen get culled out, the precision of the quat calculation has to be perfect in order to get this done. Maybe floating point errors, this should be easy to fix.
The next thing on the task list would be to get the magnetometer working again, so the camera will always point in the same direction, once initialized. I have to create a new concept for this using the magnetometer raw data and the custom game_rotation_vector with a custom kalman filter. Maybe I should try it with the old rotation vector first, this one has already the magnetometer build in, but due to the rotation delay, it's quite possible that I have to code a custom implementation for this one too.
I bet I could even improve the accelerometer data into this custom fusion, but it will be hard work though.
I have also taken a look inside the oculus SDK. They are actually using different sensors build in the gear vr headset. This sensors are much better, running on 1000Hz. There is actually a chance to get them in AppGameKit, but I have still no clue how to combine the oculus SDK with AGK. There are also good news: both of them are using the same Android SDK and can be compiled using the same NDK.
I don`t have the skills right now to fuse the oculus SDK into AppGameKit, I have still to learn a lot on this segment.
Paul, do you have data about the speed difference of code written in AppGameKit or NDK? And what is about the timer() command in agk? The docs mentioned
"Calling this command will internally make a system call to get the amount of time passed, so avoid calling this command unnecessarily. " - how does it affect the performance, and are there faster methods to get the frame time?
Are all OpenGL calls in a seperate thread and commands like sin(), abs() in another? Like the music that doesn't get slower, even if the FPS is dropping?
I need this information to improve the performance of the working code