Sorry your browser is not supported!

You are using an outdated browser that does not support modern web technologies, in order to use this site please update to a new browser.

Browsers supported include Chrome, FireFox, Safari, Opera, Internet Explorer 10+ or Microsoft Edge.

DarkBASIC Professional Discussion / is there a way to have dbpro understand what you say?

Author
Message
sovr
16
Years of Service
User Offline
Joined: 2nd Jan 2010
Location: USA
Posted: 24th Jan 2012 19:02
Hello, I have been trying to find a way to have dbpro understand what I say... but have been having a really hard time figuring out how to do it. Is there is anybody out there that has done something close to it or did it? if not maybe I can at leat get started with some technique of some sort? thank you a lot!

sov the game creator!
TheComet
18
Years of Service
User Offline
Joined: 18th Oct 2007
Location: I`m under ur bridge eating ur goatz.
Posted: 24th Jan 2012 21:09
The way it's done professionally is in 3 steps:

1) Convert recorded voice into a 3D spectrum and store in an array (x=time, y=frequency, z=amplitude)

2) Normalize the data in the array so time, frequency, and amplitude become relative. This eliminates the problem of slow/fast speaking people, quite/loud speaking people, and low/high voices.

3) Interpret the array using artificial neural networks trained to recognise different words.

Ta-dah

TheComet

CumQuaT
AGK Master
16
Years of Service
User Offline
Joined: 28th Apr 2010
Location: Tasmania, Australia
Posted: 25th Jan 2012 00:34
What you're after is Speech Recognition.

Much as I'm a fan of TGC software, if you're writing software for robotics applications you'd be better off using Visual Studio... But if you're really determined...

DBpro has the ability to interact with third party DLLs, such as the ones provided with software packages such as Dragon Naturally Speaking or even Windows 7. These will give you the functionality that you need, though I'm not sure on the specifics.

Or... You know... As TheComet said... Neural networks...


sovr
16
Years of Service
User Offline
Joined: 2nd Jan 2010
Location: USA
Posted: 25th Jan 2012 01:14 Edited at: 25th Jan 2012 01:16
Well I can use dll's but if needed I can make multiple programming languages communicate to each other also... Say java and dbpro or agk and c++, etc. but I would need to do a lot of reasearch on voice recognition!

Edit: if I do develop anything I will keep this thread updated!

sov the game creator!
MrValentine
AGK Backer
15
Years of Service
User Offline
Joined: 5th Dec 2010
Playing: FFVII
Posted: 25th Jan 2012 04:02
Interesting... keeping tracked...

revenant chaos
DBPro Master
19
Years of Service
User Offline
Joined: 21st Mar 2007
Location: Robbinsdale, MN
Posted: 25th Jan 2012 04:23
I played around with something like this in the past, but my solution was to use window's "speech to text" to pass my input to dbpro using the entry buffer. It wasn't as reliable as I had hoped but it did work (though I suppose I could have spent more time training the speech recognition).
If from there you wrote a system to compare strings you could increase the accuracy a bit more, but that would be up to you to account for errors (misinterpreting and executing the wrong command).
MrValentine
AGK Backer
15
Years of Service
User Offline
Joined: 5th Dec 2010
Playing: FFVII
Posted: 25th Jan 2012 04:29
Ahh look into Tom Clancys END WAR think it was they used speech recognition in there... maybe if you look for developer videos you might find out how they did it...

sovr
16
Years of Service
User Offline
Joined: 2nd Jan 2010
Location: USA
Posted: 25th Jan 2012 18:52
@revenant choas

Do you have any form of source code I can work off of? Again if I update it or anything I will post it. But as soon as I got time i will figure this stuff out!

sov the game creator!
revenant chaos
DBPro Master
19
Years of Service
User Offline
Joined: 21st Mar 2007
Location: Robbinsdale, MN
Posted: 26th Jan 2012 04:45 Edited at: 26th Jan 2012 04:46
Here it is, but there is really not much to show (put this in your main loop):

The real magic happens within window's speech-to-text (which you have to setup yourself).
zenassem
23
Years of Service
User Offline
Joined: 10th Mar 2003
Location: Long Island, NY
Posted: 26th Jan 2012 23:45
I'm still stuck on DBpro understanding what I type!!!!

~ZENassem
Leonid
User Banned
Posted: 31st Jan 2012 22:40
Sovr, you must know how live brain recognizes sounds. In real, we listen all sounds as MIDI, not a soup of PCM. I don't know, but believe, MP3 converts WAVs, using digital resonance for every listenable frequency. Write 10 - 12 digital "receivers" for several (important for understanding of sayed word) frequencies - and learn your gomuskulus.

My old cellular phone with gray screen can recognize my own commands, but often writes "Dismatch. Say again.".

If you are from secret services, you will never ask this, because it already works.

Let's work together, because I searching the same.
TheComet
18
Years of Service
User Offline
Joined: 18th Oct 2007
Location: I`m under ur bridge eating ur goatz.
Posted: 31st Jan 2012 23:19
Quote: "I don't know, but believe, MP3 converts WAVs, using digital resonance for every listenable frequency."


Not quiet. MP3 compresses it's samples at lower frequencies. I think OGG is the one you're thinking about.

The Fast Fourier Transform algorithm I described in the first post I made is exactly what picks apart each frequency.

TheComet

Chris Tate
DBPro Master
17
Years of Service
User Offline
Joined: 29th Aug 2008
Location: London, England
Posted: 1st Feb 2012 00:44 Edited at: 1st Feb 2012 00:44
I have never considered doing this, but wouldn't the native Record Sound command work? Is it fast enough?

It has a duration parameter. If you keep recording 100 milliseconds or so into a sound that can be converted into a memblock every 100ms or so; using make memblock from sound, then you could examine the sound wave in code.


Leonid
User Banned
Posted: 1st Feb 2012 18:48 Edited at: 1st Feb 2012 18:53
To recognize sound, first, we need to detect each frequency from mixed data. I still thinking, how to do it better. See, how 2 blended frequencies looks together. (I dont know how place a picture - press red button "View").
MrValentine
AGK Backer
15
Years of Service
User Offline
Joined: 5th Dec 2010
Playing: FFVII
Posted: 1st Feb 2012 18:50
Have you not looked at Dark Voices¿

Chris Tate
DBPro Master
17
Years of Service
User Offline
Joined: 29th Aug 2008
Location: London, England
Posted: 1st Feb 2012 19:01
Dark Voices appears to detect changes in volumes which indicate how wide to open the character's mouth. I could be wrong.

sovr
16
Years of Service
User Offline
Joined: 2nd Jan 2010
Location: USA
Posted: 2nd Feb 2012 02:17
there is a chunk of code done by someone (using the record sound command) in the code base that uses memblocks (if I remember correctly) to be compared from a previous recording. But the dark voices seem good! lol right now I am in the macosx software so I can not check, but is there a command that could get the volumes of each segment of a recording? If not we should try to look into making a command (or even look more into dark voices) to get something started

sov the game creator!
Leonid
User Banned
Posted: 2nd Feb 2012 02:48 Edited at: 2nd Feb 2012 03:26
Quote: "there is a chunk of code done by someone (using the record sound command) in the code base that uses memblocks (if I remember correctly) to be compared from a previous recording."

Where is the code? How I think, my gray phone did the same comparation, but I tired to repeat the command again and again, untill phone will compare succesfully. And, this program will never recognize a voice of another player. I need my program will understand words as brain does.

My not clever solution (click red buttons below). If yellow phases repeats 4 times (1 2 3 4) - it is already resonance. Sovr, do you want to continue think about frequency detection and offer something too?
sovr
16
Years of Service
User Offline
Joined: 2nd Jan 2010
Location: USA
Posted: 2nd Feb 2012 23:23
Here is the link of the code... But it would not be as good as a dll file.

http://www.thegamecreators.com/?m=codebase_view&i=87d131af0a40c67db69a918567153d92

sov the game creator!
sovr
16
Years of Service
User Offline
Joined: 2nd Jan 2010
Location: USA
Posted: 3rd Feb 2012 05:49 Edited at: 3rd Feb 2012 06:19
Well I am to start on working with c++ to find a way to do this and make it into a
Dll for dbpro... I should have some time this weekend


edit: here is a good place to start

http://msdn.microsoft.com/en-us/library/system.speech.recognition.aspx


double edit: and here is an example code:
http://msdn.microsoft.com/en-us/library/system.speech.recognition.speechrecognitionengine.speechrecognized.aspx

sov the game creator!
Leonid
User Banned
Posted: 3rd Feb 2012 21:35 Edited at: 3rd Feb 2012 21:45
I was thinking a little: spending my time for other's fun. I want to use only DBPro (memblocks). Why you need C++ ? It works faster? Or it doing recognition in background mode? What you think about logic of recognition? (Yourself made program.) If DBPro's recognition is very hard to do, I think, I will go away.

My methods of recognition:

1. Detect the highest frequency and store the amplitude of modulated (by this frequency) logical signal in the new memblock.

2. Erase this frequency using the filter of high-speed changes in phase.

3. Detect the next frequency (lower than first) and store the amplitude of modulated (by this frequency) logical signal.

4. Erase this frequency too (like in article 2).

5. Make the spectrum lines in time of speaking.

6. Say into microphone many words with every sound.

7. Find the differencies between fallings and risings from "letter" to "letter" (press the red button View).

8. Now you need only compare risings and fallings between letters after recording a sound. No matter what speed and frequency of speech will be.

Correct me if I wrong.
sovr
16
Years of Service
User Offline
Joined: 2nd Jan 2010
Location: USA
Posted: 3rd Feb 2012 22:45 Edited at: 3rd Feb 2012 22:48
the process you just said sounds very good to me and almost sounds like a process I was trying, but more into it. The whole C++ thing is to make it so I could add commands to dbpro using a dll file. The voice recognition class built into VS C++ allows me to easily make a voice recognition command in dbpro (I think... didn't get into it that much). But from previous experience I do know how to make simple and fairly complex dbpro commands from a dll. If I were to strictly go with only C++ it would be faster and I could do different threads... but that is not what I am trying to do.

edit: I do see memblocks working, but I would want an easier and (maybe) more accurate way of doing it.

sov the game creator!
Leonid
User Banned
Posted: 4th Feb 2012 01:27
Do you use the same method of speech recognition or have your's own?

Forgot to say. 8. No matter what amplitude will be too (Need to use a percents). 9. And, how I am thinking now, any way, my method has a bad side: program must know whole words' fallings and risings, so the program must have a big database: several numeric values with type Byte per word; 2000 words will must be at least.
Chris Tate
DBPro Master
17
Years of Service
User Offline
Joined: 29th Aug 2008
Location: London, England
Posted: 4th Feb 2012 01:58 Edited at: 4th Feb 2012 01:58
I'm just throwing something into the hat. Both of your ideas make good sense so don't take me wrong, this is just idea 3.

Another way is to get DBPRO to study many recordings of words, and save the average of the analysis to a memblock, bank or array file. By study, I mean save wave data every so often from a wave file.

Then scan the live audio for patterns similar to each word played back at different speeds. To scan, compare the difference between integers from a recusive range in the recording to the recursive range in the pre-recorded data.

sovr
16
Years of Service
User Offline
Joined: 2nd Jan 2010
Location: USA
Posted: 4th Feb 2012 03:36 Edited at: 4th Feb 2012 18:12
Ok I have looked at the C++ stuff and thought that would take me a very long time to do... so I decided to try it with memblocks in dbpro first. here is the code for the program... don't know if it works I just did it and since it is night time over here I would not want to do any recording. I will test this out tomorrow, but try it out yourselves if there is a bug I will edit the code (on the post also). Lol I might of made a mistake with reading the memblocks with bytes (that's my thought).

code:


Ohh and chris this code is just like your idea, maybe slightly different.


edit: about the code! you need a sound file called hello.wav, which is the same word and recording duration as the word you are about to say. when you press a key to continue just say the word and wait. there should be a 0 printed on the screen, now you press key 1 to get a percentage of how similar they are. This might work fine or really bad, but I suspect the accuracy to be bad.



*another thought is you could have one massive sound file (that has 1000's of words with spaces between them) that could be scanned once with the small sound file to be scanned many times. get what I am saying?


*yet another edit: I have added a second method of comparing to sounds file using arrays which tend to be a lot more accurate!

sov the game creator!
Captain Chucky
14
Years of Service
User Offline
Joined: 5th Feb 2012
Location:
Posted: 5th Feb 2012 09:47
Use Visual Studio system.speech and convert it to a DBP plugin. DBP Plugins can be created in C++, C# or vb.net...though i'm still experimenting on using .net libraries within DBP. Start with the "Help" in DBP where it mentions "Third Party Plugins". Other details for conversion are all over the forums.
sovr
16
Years of Service
User Offline
Joined: 2nd Jan 2010
Location: USA
Posted: 7th Feb 2012 01:43
doing a plugin is very good, and I might do one when I have time. I did find a good voice recognition software that does hot keys so I could sync it to the keystate() command in dbpro. I have heard of people integrating this software with skyrim so you say the shouts to make it happen in the game. So that is what I will be using in the mean time.

*if anyone is interested here is the link:
http://www.dwvac.com/download.html

sov the game creator!

Login to post a reply

Server time is: 2026-07-10 08:04:19
Your offset time is: 2026-07-10 08:04:19