Sorry your browser is not supported!

You are using an outdated browser that does not support modern web technologies, in order to use this site please update to a new browser.

Browsers supported include Chrome, FireFox, Safari, Opera, Internet Explorer 10+ or Microsoft Edge.

Geek Culture / Video Motion Tracking / Compositing

Author
Message
RUCCUS
20
Years of Service
User Offline
Joined: 11th Dec 2004
Location: Canada
Posted: 23rd Oct 2009 17:56 Edited at: 27th Oct 2009 17:25
Link
I've finally gotten off my ass and learnt how to perform good motion tracking in videos, with the help of a little perk called Boujou that my school has offered to me for the year .

Here's the first experiment;
http://www.youtube.com/watch?v=h6PQTabzQ5A&feature=player_embedded

Second one;
http://www.youtube.com/watch?v=dxuEcnZJMmY

Third one,
http://www.youtube.com/watch?v=H8YdTbt_XXY

Fourth,
http://www.youtube.com/watch?v=b1YODFuPNaA

On a side note of awesome news - incase anyones interested - Canadian Tire is buying the rights to one of my logos I designed in school last year and putting me in the newspaper , they aren't offering me much so I may try to bump the price up a bit, but regardless its still awesome for a portfolio.

- RUC'
Back to top
Profile PM Email Website
Drew Cameron
21
Years of Service
User Offline
Joined: 30th Jan 2004
Location: Scotland
Posted: 23rd Oct 2009 18:06
Link
AMAZING!!!!!

Do you mind if I add you on MSN to discuss this shot?

Thanks in advance,
Drew

Back to top
Profile PM Email Website
RUCCUS
20
Years of Service
User Offline
Joined: 11th Dec 2004
Location: Canada
Posted: 23rd Oct 2009 18:10
Link
Sure, email is chiken [dash] foot [at] hotmail [dot] com. Dont ask, been my email since I was 10. Make sure you spell it chiken, not chicken, and use a dash, not an underscore.
Back to top
Profile PM Email Website
Ron Erickson
Moderator
22
Years of Service
User Offline
Joined: 6th Dec 2002
Location: Pittsburgh, PA, USA
Posted: 23rd Oct 2009 18:24
Link
Very cool video Ruccus!
You should write-up a tutorial on doing this.


a.k.a WOLF!
Back to top
Profile PM Email
Fallout
22
Years of Service
User Offline
Joined: 1st Sep 2002
Location: Basingstoke, England
Posted: 23rd Oct 2009 18:25 Edited at: 23rd Oct 2009 18:32
Link
Yeah, that is quality. I can see Drew's little filming braincells churn into action, as have mine! Well, when I say Drew's braincells, I'm obviously flattering him.

Edit: hehehe. Boujou costs £10k! I'm sure I could write a DBP app to do something similar (although clearly no way as advanced) using coloured balls for queues and Rons EZRotate. Although whether it's worth it, to add DBP rendering to a movie?!? And the difficulty in recapture and recoding .....

Back to top
Profile PM
Ron Erickson
Moderator
22
Years of Service
User Offline
Joined: 6th Dec 2002
Location: Pittsburgh, PA, USA
Posted: 23rd Oct 2009 18:50
Link
@Fallout:

Quote: "Edit: hehehe. Boujou costs £10k!"

LOL! I saw that too. Ah well. It looked like something FUN to play with.

Quote: " I'm sure I could write a DBP app to do something similar (although clearly no way as advanced) using coloured balls for queues and Rons EZRotate. Although whether it's worth it, to add DBP rendering to a movie?!? "

Go for it! I know I would play around with it. The rendering wouldn't be great (although some nice things can be done with Shaders), but it would be a cheap, dirty, fun solution.

Quote: "And the difficulty in recapture and recoding ....."

I'm not familiar with the feature set of Dark Video, but that would be the FIRST place that I'd suggest looking if you're interested in attempting this.


a.k.a WOLF!
Back to top
Profile PM Email
Drew Cameron
21
Years of Service
User Offline
Joined: 30th Jan 2004
Location: Scotland
Posted: 23rd Oct 2009 19:15
Link
If anyone is interested, I frequently use DBP for 3D bits in the animated project I'm working on.

But that is ANIMATION and it doesn't have the prerequisite of having to look photo real.

In addition I'm only working at 1280x720 which is bare bones HD.

Back to top
Profile PM Email Website
RUCCUS
20
Years of Service
User Offline
Joined: 11th Dec 2004
Location: Canada
Posted: 23rd Oct 2009 19:31
Link
The maths required in camera tracking are a little more complex than you would think . Ive looked into it.
Back to top
Profile PM Email Website
Ron Erickson
Moderator
22
Years of Service
User Offline
Joined: 6th Dec 2002
Location: Pittsburgh, PA, USA
Posted: 23rd Oct 2009 19:38
Link
Quote: "The maths required in camera tracking are a little more complex than you would think"


Sounds like FUN to me


a.k.a WOLF!
Back to top
Profile PM Email
RUCCUS
20
Years of Service
User Offline
Joined: 11th Dec 2004
Location: Canada
Posted: 23rd Oct 2009 19:53
Link
Go for it ron.

If you could actually get it working, you could make games where theres a pre-recorderded video thats been tracked displayed in the background, and in DBP you could have enemies or whatever running around in this 3D environment, you'd just follow along the camera path shooting guys. Add some masking methods so they can come out from behind walls and it'd be pretty cool.
Back to top
Profile PM Email Website
Ron Erickson
Moderator
22
Years of Service
User Offline
Joined: 6th Dec 2002
Location: Pittsburgh, PA, USA
Posted: 23rd Oct 2009 20:10
Link
It would be awesome to get something like that working in combination with some green screen capability. I don't know what would be funner to play with: Adding 3D models to REAL environments (as you did in your video) -OR- adding video to 3D environments. Much fun could be had with BOTH!


a.k.a WOLF!
Back to top
Profile PM Email
RUCCUS
20
Years of Service
User Offline
Joined: 11th Dec 2004
Location: Canada
Posted: 23rd Oct 2009 20:24 Edited at: 23rd Oct 2009 20:26
Link
The only issue is everything would have to be pre-solved, which isn't that much of an issue really, just a limit. For example, the video I showed above was only 768 frames, and it took boujou almost an hour to solve (excluding the time it took to find the track points, filter out bad points, correct for lens distortion, etc.)

I think my next test will be using a model I made a year or so ago of an xbox 360 controller that could turn into a robot, here it is:

http://forum.thegamecreators.com/?m=forum_view&t=120991&b=3

Ill take that, film a normal 360 controller, switch it with this model in a frame change, then animate the model to morph itself into a robot. Should be cool.
Back to top
Profile PM Email Website
Fallout
22
Years of Service
User Offline
Joined: 1st Sep 2002
Location: Basingstoke, England
Posted: 23rd Oct 2009 20:49 Edited at: 23rd Oct 2009 20:51
Link
Quote: "The maths required in camera tracking are a little more complex than you would think . Ive looked into it."


I'm sure their system is very complex! But my initial thoughts are a MUCH simplified system.

-You have 3 coloured balls, in primary colours.
-You put them anywhere in your scene, then measure exactly how far they are apart on all 3 axis (get your tape measure out!). You effectively give them 3D coordinates.
-You store these in your DBP program and you now have a 3D triangle that represents your base orientation
-You now code an image processor that looks for these primary colours in each frame of footage, and figures out where the center of each ball is in the frame (obviously the balls will always have to be in shot and you'll need to work hard on this code to make it as reliable as possible)
-You can now calculate the new triangle.
-You always assume you've above the triangle on the y, so you can get exact rotations and use EZRotate (to make life easier!) to calculate the offset of rotations the shot has to the base rotations, as well as the zoom and position (I personally think this would be relatively straight forward)
-Position the camera in 3D space accordingly

Initial problems I see:
- Accurate recognition of the balls
- Having 3 coloured balls in your scene! You'd have to put 3D objects over the top of them in your compositing
- Error handling and smoothing of incorrect angle calculations etc.

I'm sure it'd be tricky and fiddly, but I dont think it's that complicated. I think the genius in their software would be the reliability, and their much more complex cue recognition. Still, I reckon it's doable. The main thing that'd stop me having a try is I can't see a real use for it except for messing around, since DBP isn't geared towards photo-realistic rendering.

Back to top
Profile PM
RUCCUS
20
Years of Service
User Offline
Joined: 11th Dec 2004
Location: Canada
Posted: 23rd Oct 2009 21:13
Link
Thats what I had initially assumed as well. The issue is you'll never get a smooth track with only 3 tracking points, regardless of how easily the system can recognize them. Lens distortion plays a big role in it. It would work, you'd have to get the size of a ball at x-units from the camera, then determine the size of the ball in the video, divide this by the pre-determined size when the ball is say 1 metre from the camera, and you'll find the distance to the ball. It just won't be a smooth enough track though, the models will roughly follow along, but they'll still jitter around and not be firmly placed on the ground unless you can get many other track points and average their movements together for each frame.

I was actually looking into this for about a week trying to implement it into DBP but when I began finding articles about solving all of the issues that come with doing this, there ends up being a heavy load of math involved to get the right camera solve.

Im not trying to discourage you, if you get the time by all means go for it Fallout, I know you've got the skill to implement a system like you're saying, I just know that you wont get a completely reliable, perfectly smooth track out of it (I know you're not saying you will either, Im just reinforcing the fact).
Back to top
Profile PM Email Website
jasonhtml
21
Years of Service
User Offline
Joined: 20th Mar 2004
Location: OC, California, USA
Posted: 23rd Oct 2009 21:22
Link
very impressive!

Back to top
Profile PM Email Website
Drew Cameron
21
Years of Service
User Offline
Joined: 30th Jan 2004
Location: Scotland
Posted: 24th Oct 2009 00:13
Link
Quote: "Im not trying to discourage you, if you get the time by all means go for it Fallout, I know you've got the skill to implement a system like you're saying, I just know that you wont get a completely reliable, perfectly smooth track out of it (I know you're not saying you will either, Im just reinforcing the fact). "


I'm guessing it may just be easier to do it manually shot by shot...

Back to top
Profile PM Email Website
NeX the Fairly Fast Ferret
20
Years of Service
User Offline
Joined: 10th Apr 2005
Location: The Fifth Plane of Oblivion
Posted: 24th Oct 2009 00:19 Edited at: 24th Oct 2009 00:20
Link
http://www.roborealm.com/

Aww man, this used to be free software!

Athlon64 2.7gHz->OC 3.9gHz, 31C, MSi 9500GT->OC 1gHz core/2gHz memory, 48C, 4Gb DDR2 667, 500Gb Seagate + 80Gb Maxtor + 40Gb Maxtor = 620Gb, XP Home
Air cooled, total cost £160
Back to top
Profile PM Website
RUCCUS
20
Years of Service
User Offline
Joined: 11th Dec 2004
Location: Canada
Posted: 24th Oct 2009 06:26
Link
Got another one done, this time I was trying to get a good track with a moving vehicle and person in the shot, along with quite a shakey camera. It took a little tweaking, but it worked in the end.

http://www.youtube.com/watch?v=dxuEcnZJMmY
Back to top
Profile PM Email Website
Jeku
Moderator
21
Years of Service
User Offline
Joined: 4th Jul 2003
Location: Vancouver, British Columbia, Canada
Posted: 24th Oct 2009 08:41
Link
Wow, that's cool


Senior Web Developer - Nokia
Back to top
Profile PM Website
Agent Dink
21
Years of Service
User Offline
Joined: 30th Mar 2004
Location:
Posted: 24th Oct 2009 09:55
Link
That is indeed awesome!!! I need to get a camcorder!

MISoft Studios - Silver-Dawn Gorilda is lost!

Back to top
Profile PM
AaronG
18
Years of Service
User Offline
Joined: 28th Oct 2006
Location: Millstone, NJ
Posted: 24th Oct 2009 23:29
Link
Dude your stuff is amazing. Just work on lighting and texturing (of course I cannot speak for my skills are HORRIBLE.) But man, you've got so much potential, I'm so jealous.



Back to top
Profile PM Email
RUCCUS
20
Years of Service
User Offline
Joined: 11th Dec 2004
Location: Canada
Posted: 25th Oct 2009 00:28
Link
The lighting and texturing are easy, those models were just downloaded quickly from turbosquid for testing. I've already experimented with adding atmospherics as well like fog in after effects to give them more depth. Thanks guys.
Back to top
Profile PM Email Website
RUCCUS
20
Years of Service
User Offline
Joined: 11th Dec 2004
Location: Canada
Posted: 25th Oct 2009 04:39
Link
New video, experimenting with atmospherics and masks to put the models behind objects in the video:

http://www.youtube.com/watch?v=H8YdTbt_XXY
Back to top
Profile PM Email Website
Gunn3r
18
Years of Service
User Offline
Joined: 12th Jun 2006
Location: Portland, OR
Posted: 25th Oct 2009 04:42
Link
Lemme guess, your next big project is a Cloverfield style movie? Looks great, RUCCUS. I wish I had a camcorder and knew how to edit video like that.

Back to top
Profile PM Email Website
Fallout
22
Years of Service
User Offline
Joined: 1st Sep 2002
Location: Basingstoke, England
Posted: 25th Oct 2009 16:45
Link
@Ruccus

The Star Trek one looks brilliant. Now the potential is showing through. Using the amateur camera footage approach works well too.

I know what you're saying about a DIY solution to this. It would be really tricky to get just right. I don't think I would ever measure the size of the ball though. I would measure the size of the triangle, and would enforce some sort of setup rule, like the use of an equalateral triangle in order to make calculations easier. I'm hoping I could ignore the size of the balls, and just detect where they are, create a 2D triangle from them, then calculate the lengths and angles and perform a 3D conversion.

Still, realistically I am just speculating. I don't think there's enough value in it to warrent the effort. Oh well. It's a fun toy for you though matey. I hope you come up with some good projects with it.

Back to top
Profile PM
Diggsey
19
Years of Service
User Offline
Joined: 24th Apr 2006
Location: On this web page.
Posted: 25th Oct 2009 17:04
Link
I had a look at the boujou website, and it explains the basic techniques behind it.

From each frame, they take a number of sample points, which are places of high contrast. They then track each of these points over as many frames as possible, by finding nearby points which look similar. Using statistical analysis of the movement, they can remove erroneous positions, and can get an idea of the 2d movement in every part of the screen for every frame.

By taking groups of these points, and looking at their movement, it is possible to work out the equivalent 3d movement which they describe. Once they have calculated this, they can again use statistics to find the most common 3d movement in the frame. (Obviously they use some more advanced methods look over many frames to make sure that the movements they find are common in all the frames for the points).

They can then use this movement as the movement of stationary points in the scene, and the camera movement is the inverse of this.

Once they have the camera movement, they can untransform the other points by the camera movement. Because they know the camera movement, and they know the stationary points, they can calculate the depths into the screen of the stationary parts.

They can also group together moving points which are moving in a similar way to find objects in the scene.

I think the main technique they use is statistics. However good the algorithms are for tracking points, there will always be a high percentage of errors, and so using statistical analysis lets you cut out most of the errors.


New DBPro IDE + integrated debugger!
Back to top
Profile PM Email Website
dab
20
Years of Service
User Offline
Joined: 22nd Sep 2004
Location: Your Temp Folder!
Posted: 26th Oct 2009 01:01
Link
Holy cow! You have got to show me how you do that!

Back to top
Profile PM Email Website
RUCCUS
20
Years of Service
User Offline
Joined: 11th Dec 2004
Location: Canada
Posted: 27th Oct 2009 17:24 Edited at: 27th Oct 2009 17:25
Link
Latest video;

http://www.youtube.com/watch?v=b1YODFuPNaA

I think Im about ready to move onto something bigger and longer / more detailed now. I now know how to do the motion tracking pretty succesfully, render shadows, motion blur, and lighting effects from 3DS Max, and alter the graininess, tones, lighting, and opacity in after effects as well as mask out portions of the footage behind scene objects. I think my next test will be making something fall from the sky (an asteroid or a bomb or an escape pod) and crash into the street making a crater / hole, then having another shot look into the hole at the pod.
Back to top
Profile PM Email Website
Drew Cameron
21
Years of Service
User Offline
Joined: 30th Jan 2004
Location: Scotland
Posted: 27th Oct 2009 18:02
Link
RUCCUS, that last video for instance; how long did that take you from scratch? Very impressive.

Nazi Zombies Clone. Click here for info page.
Back to top
Profile PM Email Website
RUCCUS
20
Years of Service
User Offline
Joined: 11th Dec 2004
Location: Canada
Posted: 27th Oct 2009 18:10 Edited at: 27th Oct 2009 18:13
Link
Well because Im doing such short film segments, the tracking isn't taking too long, but its still quite long for how short they are. For that last video, Boujou took roughly 10 minutes to track the features, then around half an hour to do the camera solve. After adjusting the solve with a few of boujou's tools to get a cleaner final product overall I would say I spent around 50 minutes in boujou. Inside max it was pretty quick, I found a model of a jet, animated it on a flight path, duplicated the jet a few times to make the other ones, moved them around to give them different depths, etc. All the animation time was about 10 minutes, and to render the scene (keep in mind I had no lighting in that scene, only motion blur applied it took 12 minutes, so roughly 25 minutes in max.

In after effects its pretty straight forward, import the two videos, overlay on onto the other, it took all of a few minutes to get the scene set up, and then doing the masking took around 5 minutes as well. A few minutes for the exporting of the video... so roughly 15 minutes in after effects give or take.

So all in all... 50 + 25 + 15... an hour and a half to do the whole thing. The filming time was next to nothing, I didnt really plan the scene, just took my camera out and filmed a quick shot while I was walking to the bus from work.


It doesnt seem too bad... but thinking about it, thats quite long for a 20 second video like that. Id imagine in the industry they have their software set up on a network to allow multiple computers to solve sections of a video and then string them all together to cut down on the time needed. I could always do a faster solve in boujou, but theres a chance it wont be as smooth. Its better to just put the processing time in, go have a coffee or something and come back to a proper solve that you dont have to much about with later on.
Back to top
Profile PM Email Website
Drew Cameron
21
Years of Service
User Offline
Joined: 30th Jan 2004
Location: Scotland
Posted: 27th Oct 2009 18:18 Edited at: 27th Oct 2009 18:22
Link
Quote: "thats quite long for a 20 second video like that"


In the animated project I'm doing it takes about a week and a half to do one minute ...I thought this would have taken you the better part of a day so I'm surprised!! [At quick rendering times etc, not that I'd expect you to be slow or anything bad]

Nazi Zombies Clone. Click here for info page.
Back to top
Profile PM Email Website
Back to top

Login to post a reply

Server time is: 2025-06-04 21:00:14
Your offset time is: 2025-06-04 21:00:14