Sorry your browser is not supported!

You are using an outdated browser that does not support modern web technologies, in order to use this site please update to a new browser.

Browsers supported include Chrome, FireFox, Safari, Opera, Internet Explorer 10+ or Microsoft Edge.

Program Announcements / 3D Midi Wizard

Years of Service
User Offline
Joined: 26th Jul 2008
Location: one of Cybertrons moons
Posted: 9th Aug 2009 17:57 Edited at: 9th Aug 2009 18:04
Hi guys. I just wanted to let you know about my first and only DarkBasic Pro project.

This is a little program I made for progressive rock band "Dream Theater" for their latest tour (I doubt anyone here has heard of them).

The program is basically a 3D character that plays along with whatever you play via MIDI in realtime. It is a wizard character with an octagonal keyboard, and it will attempt to play whatever notes it receives through MIDI as plausibly as possible. The challenge was getting it to sort the notes and figure out which hand and which finger to play which, and perform it as best it can.
The program has already been used for about 12 tour dates up the East coast of the US, then will be used across the rest of the US, then Europe, then across the rest of the world for about 100 shows in total. The program was also demonstrated at this year's NAMM show at the Korg booth.

Here are some screenshots and a video-

Since it's only designed for one specific high end machine, I splashed out on graphics, since it didn't need to be compatible with low end hardware. The video is older and so is without the graphical effects.

Technical notes-
The room uses a parallax mapping shader, modified to include specular mapping and light mapping. Everything else in the scene is also lightmapped, and there is a glow shader applied to the screen to add some softness to the scene.

The musical staff shows what notes are being played in realtime, and uses a custom made vertex shader to ripple the staff based on playing speed.

All of the character animation is done in realtime through code with no pre-made animation. I programmed some basic Inverse Kinematics into the arms so I could position the hands in the correct spot on the keyboard. You'll also notice a cel shading effect, although I'm only using the cartoon outline, and not the shading aspect.

I've also used a shadow shader so the fingers cast a shadow onto the keys, and I've cube mapped the keys to make them shiny.

I also have some video of the program being used live, which gets displayed on a 55" plasma screen behind the keyboardist, and also sometimes used on the large projection screen.
The band's keyboardist also does a solo spot where he duels with the 3D character, and then does a duet, but I don't have video of that unfortunately.

Here are some live clips of the program being used- - at 8:42, it is displayed on the large projection screen. Ignore the dancing girl! - from 0:55 in, program is displayed for most of the song behind the keyboardist. Also notice that when the keyboardist rotates his keyboard, the 3D character rotates his keyboard too, thanks to a custom made "MIDI Compass" device that translates rotation to a MIDI controller value.

I'd like to give a huge thanks to Rudolpho for making this program possible with his fantastic MIDI plugin for DBP, and for fixing up the problems and making it absolutely bulletproof and able to endure a whole live show of battering.
Also thanks to this forum and the creators of DB Pro for making it possible for me to make this project happen quickly and easily. This forum has been very helpful with all of my problems.

Sorry I can't post any demos, but I'm sure you understand the situation.
Retired Moderator
Years of Service
User Offline
Joined: 2nd Dec 2002
Location: United States
Posted: 9th Aug 2009 20:22
Very awesome stuff BlobVanDam, thanks for sharing. I am very impressed by the animations, and how you have had to program so much from scratch. Can you recommend any good books on programming inverse kinematics? It's something I've always been interested in programming.

A book? I hate book. Book is stupid.
(Formerly Yellow)
Years of Service
User Offline
Joined: 8th Oct 2008
Posted: 9th Aug 2009 23:36
THAT is impressive stuff! Used on 12 tours so far? Beastly. And I've heard of Dream Theater

Your signature has been erased by a mod
Virtual Nomad
Years of Service
Recently Online
Joined: 14th Dec 2005
Location: SF Bay Area, USA
Posted: 10th Aug 2009 00:04 Edited at: 10th Aug 2009 00:22
good stuff! congratz on a job well done. that's quite an accomplishment!

question: what happens when 2+ people play the keyboard? can the wizard conjure up some help?

UMMMM! i've just visited your site and your work is amazing. i'm a major sabbath fan, so... you just made my day in a big way! (and the justice vid was vastly appreciated, too!) THANKS!

Years of Service
User Offline
Joined: 8th Nov 2008
Posted: 10th Aug 2009 03:47
Very impressive. I'd imagine working with the finger animations must have been a real pain. Great to see a high quality application made with DBPro.
Years of Service
User Offline
Joined: 26th Jul 2008
Location: one of Cybertrons moons
Posted: 10th Aug 2009 09:34
MikeS - I didn't use any help to program the IK. It's a very limited low tech solution that's fudged in a few areas, and relies on the hierarchy being put together and used a specific way. The top of each arm has 3 dummy objects, so I can use a different dummy for each axis to make rotations easier.
Then it uses very basic trigonometry to figure out the angles.
I'd be happy to post some code etc if you want, but it's probably not the most pretty thing to look at and decipher!
dark coder
Years of Service
User Offline
Joined: 6th Oct 2002
Location: Japan
Posted: 10th Aug 2009 10:50
Very impressive, it's rare to see a project around here with this much polish! Nice to see it being put to good use too .

Dr UnCognizant
Years of Service
User Offline
Joined: 10th Aug 2009
Posted: 10th Aug 2009 16:35
Quote: "This is a little program I made for progressive rock band "Dream Theater" for their latest tour (I doubt anyone here has heard of them).

Never heard of them?, im going to see them :p.

This look extreamly impressive, the graphic types
Dark Dragon
Years of Service
User Offline
Joined: 22nd Jun 2007
Location: In the ring, Kickin\' *donkeybutt*.
Posted: 11th Aug 2009 20:13


(O.o ) / WORLD DOMINATION!!!!!!!!!!
(> < )
Tom J
Years of Service
User Offline
Joined: 4th Aug 2005
Location: Essex, England
Posted: 11th Aug 2009 22:22
This is newsletter worthy stuff
Pincho Paxton
Years of Service
User Offline
Joined: 8th Dec 2002
Posted: 12th Aug 2009 21:25
I play the keyboard, and.....You know you want to give me the demo!

I could watch the wizard on my 50in TV!

Years of Service
User Offline
Joined: 20th Dec 2004
Location: Scotland
Posted: 14th Aug 2009 13:41
I didn't know John Petrucci still had long hair.

On topic though;

This is sweet stuff! Nice to see DarkBasic used more publicly. Good to see that you came to it too when asked for this to be made.

Login to post a reply

Server time is: 2023-06-06 13:50:19
Your offset time is: 2023-06-06 13:50:19