Trying to get better at Neural Networks

Author

Message

Pincho Paxton

23

Years of Service

User Offline

Joined: 8th Dec 2002

Location:

Posted: 11th Feb 2012 14:23 Edited at: 11th Feb 2012 16:37

Link

I have had a few attempts at Neural Networks...

http://forum.thegamecreators.com/?m=forum_view&t=168058&b=8&p=1

I am now working on a Robot. But I never really got the Tank working, so first I need to try to understand the code properly.

Quote from link....

Quote: "Someone correct me if I'm wrong, but I could have sworn that || was a standard OR operator, not an XOR operator. "

My first question is... is this correct... || or should it be ~~

Code here...

+ Code Snippet

Function Set_Left_Nibble(bt as byte,value as byte)
   value = value << 4
   bt = bt || value
Endfunction bt

Function Set_Right_Nibble(bt as byte,value as byte)
   value = value << 4
   value = value >> 4
   bt = bt || value
Endfunction bt

Back to top

Profile PM Email Website

BatVink

Moderator

23

Years of Service

User Offline

Joined: 4th Apr 2003

Location: Gods own County, UK

Posted: 11th Feb 2012 20:08

Link

|| is OR
~~ is XOR

Back to top

Profile PM Email

Pincho Paxton

23

Years of Service

User Offline

Joined: 8th Dec 2002

Location:

Posted: 12th Feb 2012 01:26

Link

OK, but do Neural Networks use OR... or do they use XOR?

The thing about my tank is that is scores for hitting the mines, and is supposed to hit the mines. But eventually it gets really good at dodging between the mines.. I can train it to dodge the mines by scoring it to hit the mines. Which is backwards. So I don't know why the learning is back to front. I don't understand OR/XOR. So I am wondering if that is the fault.

I want to understand the code properly. My reason for the project is not that I am any good at maths, it is because I have an idea that alters the method.

Back to top

Profile PM Email Website

Pincho Paxton

23

Years of Service

User Offline

Joined: 8th Dec 2002

Location:

Posted: 12th Feb 2012 01:56

Link

Ok changed the OR for XOR, and the movements look more complex, so I think it is a good sign. Now I will lengthen the epochs.

Back to top

Profile PM Email Website

DVader

22

Years of Service

User Offline

Joined: 28th Jan 2004

Location:

Posted: 12th Feb 2012 02:10

Link

Hi Pincho, long time no see. Good luck with this, always trying an interesting project as normal I see

I'm busy with AppGameKit myself having fun going back to Amiga style games. Heres a link to my latest demo if you want a peek.
http://forum.thegamecreators.com/?m=forum_view&t=194270&b=48

Back to top

Profile PM Website

Pincho Paxton

23

Years of Service

User Offline

Joined: 8th Dec 2002

Location:

Posted: 12th Feb 2012 03:21

Link

Yeah I remember a Psygnosis game like that. I thought it was Menace with a red ship, but Menace didn't have a red ship.

Back to top

Profile PM Email Website

Pincho Paxton

23

Years of Service

User Offline

Joined: 8th Dec 2002

Location:

Posted: 13th Feb 2012 16:36 Edited at: 13th Feb 2012 17:20

Link

I have decided to try a different method, because the last method seems to only remember 15 bits, and I can't figure it out. So now I am trying to convert VB code to DBPro code. They have different outputs however, and I'm not sure if the DBPro version is correct, or not... DBPro...

EDIT: Fixed it

`Backprop neural network by Richard Gardner
`rgardner@rgsoftware.com
`http://www.rgsoftware.com

Global LEARNING_RATE# As float = 0.5
Dim Weights#(3,3) As Float
Global net1#
Global net2#
Global net3#
Global Weights#
Global Out#
Global V1#
Global V2#
`Global i as integer
`Global J
Dim Deltas#(3) As Float
Global Deltas#
Do
Gosub test
WAIT KEY

end

loop

Test:

Reset()

For i = 1 To 3000
        Train(0, 0, 0)
        Train(0, 1, 1)
        Train(1, 0, 1)
        Train(1, 1, 0)
    Next i

Run(0,0): Print Out#
    Run(0,1): print Out#
    Run(1,0): print Out#
    Run(1,1): print Out#

Return

Function Reset()
    `Reset the weights with random values

Randomize timer()
    For i = 1 To 3
        For j = 1 To 3
            Weights#(i, j) = Rnd(2)
        Next j
    Next i
EndFunction

Function Train(Input1# As Float, Input2# As Float, Target# As Float)

`Calculate the net values for the hidden layer neurons with activations
    net1# = Activation(Weights#(1, 1) + Input1# * Weights#(2, 1) + Input2# * Weights#(3, 1))
    net2# = Activation(Weights#(1, 2) + Input1# * Weights#(2, 2) + Input2# * Weights#(3, 2))
     `Now, calculate the net for the final output layer.
    net3# = Activation(Weights#(1, 3) + net1# * Weights#(2, 3) + net2# * Weights#(3, 3))
    `We have to calculate the deltas for the two layers.
    `Remember, we have to calculate the errors backwards
    `from the output layer to the hidden layer (that's why
    `it's called BACK-propagation).
    Deltas#(3) = net3# * (1 - net3#) * (Target# - net3#)
    Deltas#(2) = net2# * (1 - net2#) * (Weights#(3, 3)) * (Deltas#(3))
    Deltas#(1) = net1# * (1 - net1#) * (Weights#(2, 3)) * (Deltas#(3))
    `Now, alter the weights accordingly.
    v1# = Input1#
    v2# = Input2#
    For i = 1 To 3
        `Change the values for the output layer, if necessary.
        If i = 3
            v1# = net1#
            v2# = net2#
        EndIf
        Weights#(1, i) = Weights#(1, i) + LEARNING_RATE# * 1.0 * Deltas#(i) `Bias
        Weights#(2, i) = Weights#(2, i) + LEARNING_RATE# * v1# * Deltas#(i)
        Weights#(3, i) = Weights#(3, i) + LEARNING_RATE# * v2# * Deltas#(i)
    Next i

Out# = Net3#
EndFunction Out#

Function Run(Input1# As Float, Input2# As Float)

net1# = Activation(Weights#(1, 1) + Input1# * Weights#(2, 1) + Input2# * Weights#(3, 1))
    net2# = Activation(Weights#(1, 2) + Input1# * Weights#(2, 2) + Input2# * Weights#(3, 2))
    net3# = Activation(Weights#(1, 3) + net1# * Weights#(2, 3) + net2# * Weights#(3, 3))
Out# = net3#
EndFunction Out#

Function Activation(Value# As Float)
    J# = (1.0 / (1.0 + Exp(Value# * -1.0)))
EndFunction J#

+ Code Snippet

`Backprop neural network by Richard Gardner
`[email protected]
`http://www.rgsoftware.com

Global LEARNING_RATE# As float = 0.5
Dim Weights#(3,3) As Float
Global net1#
Global net2#
Global net3#
Global Weights#
Global Out#
Global V1#
Global V2#
`Global i as integer
`Global J
Dim Deltas#(3) As Float
Global Deltas#
Do
Gosub test
WAIT KEY

end

loop

Test:


    Reset()

    For i = 1 To 3000
        Train(0, 0, 0)
        Train(0, 1, 1)
        Train(1, 0, 1)
        Train(1, 1, 0)
    Next i

    Run(0,0): Print Out#
    Run(0,1): print Out#
    Run(1,0): print Out#
    Run(1,1): print Out#


Return

Function Reset()
    `Reset the weights with random values

    Randomize timer()
    For i = 1 To 3
        For j = 1 To 3
            Weights#(i, j) = Rnd(2)
        Next j
    Next i
EndFunction

Function Train(Input1# As Float, Input2# As Float, Target# As Float)

    `Calculate the net values for the hidden layer neurons with activations
    net1# = Activation(Weights#(1, 1) + Input1# * Weights#(2, 1) + Input2# * Weights#(3, 1))
    net2# = Activation(Weights#(1, 2) + Input1# * Weights#(2, 2) + Input2# * Weights#(3, 2))
     `Now, calculate the net for the final output layer.
    net3# = Activation(Weights#(1, 3) + net1# * Weights#(2, 3) + net2# * Weights#(3, 3))
    `We have to calculate the deltas for the two layers.
    `Remember, we have to calculate the errors backwards
    `from the output layer to the hidden layer (that's why
    `it's called BACK-propagation).
    Deltas#(3) = net3# * (1 - net3#) * (Target# - net3#)
    Deltas#(2) = net2# * (1 - net2#) * (Weights#(3, 3)) * (Deltas#(3))
    Deltas#(1) = net1# * (1 - net1#) * (Weights#(2, 3)) * (Deltas#(3))
    `Now, alter the weights accordingly.
    v1# = Input1#
    v2# = Input2#
    For i = 1 To 3
        `Change the values for the output layer, if necessary.
        If i = 3
            v1# = net1#
            v2# = net2#
        EndIf
        Weights#(1, i) = Weights#(1, i) + LEARNING_RATE# * 1.0 * Deltas#(i) `Bias
        Weights#(2, i) = Weights#(2, i) + LEARNING_RATE# * v1# * Deltas#(i)
        Weights#(3, i) = Weights#(3, i) + LEARNING_RATE# * v2# * Deltas#(i)
    Next i

Out# = Net3#
EndFunction Out#

Function Run(Input1# As Float, Input2# As Float)

    net1# = Activation(Weights#(1, 1) + Input1# * Weights#(2, 1) + Input2# * Weights#(3, 1))
    net2# = Activation(Weights#(1, 2) + Input1# * Weights#(2, 2) + Input2# * Weights#(3, 2))
    net3# = Activation(Weights#(1, 3) + net1# * Weights#(2, 3) + net2# * Weights#(3, 3))
Out# = net3#
EndFunction Out#

Function Activation(Value# As Float)
    J# = (1.0 / (1.0 + Exp(Value# * -1.0)))
EndFunction J#

VB.....

'Backprop neural network by Richard Gardner
'rgardner@rgsoftware.com
'http://www.rgsoftware.com

Const LEARNING_RATE As Double = 0.5

Private Weights(1 To 3, 1 To 3) As Double

Public Function Reset()
    'Reset the weights with random values
    Dim i As Integer
    Dim j As Integer
    Randomize
    For i = 1 To 3
        For j = 1 To 3
            Weights(i, j) = (Rnd * 2) - 1
        Next j
    Next i
End Function

Public Function Train(Input1 As Double, Input2 As Double, Target As Double)
    Dim Deltas(1 To 3) As Double
    Dim net1 As Double
    Dim net2 As Double
    Dim net3 As Double
    Dim v1 As Double
    Dim v2 As Double
    Dim i As Integer
    Dim out As Double
    'Calculate the net values for the hidden layer neurons with activations
    net1 = Activation(Weights(1, 1) + Input1 * Weights(2, 1) + Input2 * Weights(3, 1))
    net2 = Activation(Weights(1, 2) + Input1 * Weights(2, 2) + Input2 * Weights(3, 2))
     'Now, calculate the net for the final output layer.
    net3 = Activation(Weights(1, 3) + net1 * Weights(2, 3) + net2 * Weights(3, 3))
    'We have to calculate the deltas for the two layers.
    'Remember, we have to calculate the errors backwards
    'from the output layer to the hidden layer (that's why
    'it's called BACK-propagation).
    Deltas(3) = net3 * (1 - net3) * (Target - net3)
    Deltas(2) = net2 * (1 - net2) * (Weights(3, 3)) * (Deltas(3))
    Deltas(1) = net1 * (1 - net1) * (Weights(2, 3)) * (Deltas(3))
    'Now, alter the weights accordingly.
    v1 = Input1
    v2 = Input2
    For i = 1 To 3
        'Change the values for the output layer, if necessary.
        If i = 3 Then
            v1 = net1
            v2 = net2
        End If
        Weights(1, i) = Weights(1, i) + LEARNING_RATE * 1 * Deltas(i) 'Bias
        Weights(2, i) = Weights(2, i) + LEARNING_RATE * v1 * Deltas(i)
        Weights(3, i) = Weights(3, i) + LEARNING_RATE * v2 * Deltas(i)
    Next i
    DoEvents
    Train = net3
End Function

Public Function Run(Input1 As Double, Input2 As Double)
    Dim net1 As Double
    Dim net2 As Double
    Dim net3 As Double
    net1 = Activation(Weights(1, 1) + Input1 * Weights(2, 1) + Input2 * Weights(3, 1))
    net2 = Activation(Weights(1, 2) + Input1 * Weights(2, 2) + Input2 * Weights(3, 2))
    net3 = Activation(Weights(1, 3) + net1 * Weights(2, 3) + net2 * Weights(3, 3))
    Run = net3
End Function

Private Function Activation(Value As Double)
    Activation = (1 / (1 + Exp(Value * -1)))
End Function

+ Code Snippet

'Backprop neural network by Richard Gardner
'[email protected]
'http://www.rgsoftware.com

Const LEARNING_RATE As Double = 0.5

Private Weights(1 To 3, 1 To 3) As Double

Public Function Reset()
    'Reset the weights with random values
    Dim i As Integer
    Dim j As Integer
    Randomize
    For i = 1 To 3
        For j = 1 To 3
            Weights(i, j) = (Rnd * 2) - 1
        Next j
    Next i
End Function

Public Function Train(Input1 As Double, Input2 As Double, Target As Double)
    Dim Deltas(1 To 3) As Double
    Dim net1 As Double
    Dim net2 As Double
    Dim net3 As Double
    Dim v1 As Double
    Dim v2 As Double
    Dim i As Integer
    Dim out As Double
    'Calculate the net values for the hidden layer neurons with activations
    net1 = Activation(Weights(1, 1) + Input1 * Weights(2, 1) + Input2 * Weights(3, 1))
    net2 = Activation(Weights(1, 2) + Input1 * Weights(2, 2) + Input2 * Weights(3, 2))
     'Now, calculate the net for the final output layer.
    net3 = Activation(Weights(1, 3) + net1 * Weights(2, 3) + net2 * Weights(3, 3))
    'We have to calculate the deltas for the two layers.
    'Remember, we have to calculate the errors backwards
    'from the output layer to the hidden layer (that's why
    'it's called BACK-propagation).
    Deltas(3) = net3 * (1 - net3) * (Target - net3)
    Deltas(2) = net2 * (1 - net2) * (Weights(3, 3)) * (Deltas(3))
    Deltas(1) = net1 * (1 - net1) * (Weights(2, 3)) * (Deltas(3))
    'Now, alter the weights accordingly.
    v1 = Input1
    v2 = Input2
    For i = 1 To 3
        'Change the values for the output layer, if necessary.
        If i = 3 Then
            v1 = net1
            v2 = net2
        End If
        Weights(1, i) = Weights(1, i) + LEARNING_RATE * 1 * Deltas(i) 'Bias
        Weights(2, i) = Weights(2, i) + LEARNING_RATE * v1 * Deltas(i)
        Weights(3, i) = Weights(3, i) + LEARNING_RATE * v2 * Deltas(i)
    Next i
    DoEvents
    Train = net3
End Function

Public Function Run(Input1 As Double, Input2 As Double)
    Dim net1 As Double
    Dim net2 As Double
    Dim net3 As Double
    net1 = Activation(Weights(1, 1) + Input1 * Weights(2, 1) + Input2 * Weights(3, 1))
    net2 = Activation(Weights(1, 2) + Input1 * Weights(2, 2) + Input2 * Weights(3, 2))
    net3 = Activation(Weights(1, 3) + net1 * Weights(2, 3) + net2 * Weights(3, 3))
    Run = net3
End Function

Private Function Activation(Value As Double)
    Activation = (1 / (1 + Exp(Value * -1)))
End Function

VB Test Code...

+ Code Snippet

Private Sub cmdTest_Click()

    Dim i As Long

    Reset

    For i = 1 To 3000
        Train 0, 0, 0
        Train 0, 1, 1
        Train 1, 0, 1
        Train 1, 1, 0
    Next i

    Print CCur(Run(0, 0))
    Print CCur(Run(0, 1))
    Print CCur(Run(1, 0))
    Print CCur(Run(1, 1))

End Sub

Back to top

Profile PM Email Website

DVader

22

Years of Service

User Offline

Joined: 28th Jan 2004

Location:

Posted: 14th Feb 2012 01:33

Link

You had a look then? Try it out at all? I am looking for gameplay testing off peeps who try the demo. This neural net business looks a bit complicated for me yikes! What are you hoping to make from it?

Back to top

Profile PM Website

Pincho Paxton

23

Years of Service

User Offline

Joined: 8th Dec 2002

Location:

Posted: 14th Feb 2012 03:23

Link

I hope to make a Robot simulation, but first I need to get my Think Tank working properly...

http://www.youtube.com/watch?v=zBtt-iVqvFE

Back to top

Profile PM Email Website

Penfold

22

Years of Service

User Offline

Joined: 3rd Dec 2003

Location: RED postbox houses of parliment

Posted: 24th Feb 2012 00:12 Edited at: 24th Feb 2012 01:58

Link

This is frankly amazing !!..

Ive just read a ton of stuff on this purely based on your posts.

The youtube video has shown me something your perhaps not aware of.

Your tank *loves* green mines. I think you might be missing the point. Even though the tank seems to be dodgding the red mines this is in fact wrong. Its actually driving towards the green in most if not all cases. It will see the red in view but steer towards green and grab it instead (the only time this changes is if the turning arc on your tank is too big and it spots a closer green and goes for that instead).

If you want to get it to drive towards red instead you have to make the red higher bias for the *fitness* or whatever point system you use.

Fantastic work though.

<edited>

Ive started dissasembling your code from your *old* post , are you still using tis for the basis of yuor newest creation? If so I might be able to help you shrink the code a bit to make it easier to work on (there is a little bit of unneeded code duplication in there amongst other things) fi you need a bit of help with that. Also is the origanl tank obj available for me to borrow so I can see if i can get my modded code to work?

'Ooh 'eck chief'...'crumbs'

Back to top

Profile PM

Penfold

22

Years of Service

User Offline

Joined: 3rd Dec 2003

Location: RED postbox houses of parliment

Posted: 26th Feb 2012 01:14 Edited at: 26th Feb 2012 01:17

Link

Ok so I got RiDii's version and your version smashed them together added a bit of my own spark and ended up with this. its fast I've left the FPS on 0 so it could go as fast as possible. it seems to cap out on intelligence at about 500 epochs but I could be wrong. I got his little blighter doing an average score of 108 which i though was amazing.

Also here's the RAWR code to play with .

Rem ***** Addaptive Neural Coding - Argmentum****
`Based on /using the Riidii and Pincho Paxton code.
`Variable set up - define types

`The Curve type has been added as it is clearly something we will want to "evolve"(is it? arg).
Type Neurons
   Value as byte
   Curve as byte
   Weight as byte
   Threshold as byte
Endtype

Type Genes
   Curve as Integer
   Weight as Integer
   Threshold as Integer
Endtype

Dim Neuron(5) as Neurons

`Gene is the Tank's "genetic code". Typically, the genetic code would record and consider the population.
`In this case though, we only have one Tank and one player, and that type of evolution would take a really long time.
`So we are going to take the population of posibilities and use those instead. Since a neuron can hold a value of
`0 to 15, we will track how well each setting does and keep track over many Epochs.
`We are tracking 3 neurons ( Array Count(Neurons()) ), 2 nibbles (0-1), and 16 possible settings (0-15)
`So our Dim is:
Dim Gene(Array Count(Neuron()),5,15) as Genes
Dim BestGenes(Array Count(Neuron()),5) as Genes

`epoch data storage
Dim Epoch_Data (300)  `change this to store more epochs

Global Tank_Score#
Global Tank_Average#
Global count
Global Epoch_Timer
Global Epoch_Counter
Global Epoch_Data_Counter
Global T
Global Rotate
Global Turn#

Global NearestFuel
Global ed#
Global Lasted#

#Constant LeftEye 1
#Constant RightEye 2
#Constant Tank 1
#Constant Monster 1
#Constant Fuel 50

`----------------------------------------------------------------------------

`initial set up
set display mode 1024,768,32
Sync On: Sync Rate 0: Autocam Off
Null=Make Vector3(1)
Randomize Timer()
set normalization on

`create chomper
Make Object Sphere Monster,5,6,6
Make Mesh From Object Monster,1
Delete Object Monster
Make Object Cube Monster,20
color object Monster,rgb(0,0,0)
Add Limb Monster,LeftEye,1
Offset Limb Monster,LeftEye,-10,5,5
Color Limb Monster,LeftEye,Rgb(255,0,0)
Add Limb Monster,RightEye,1
Offset Limb Monster,RightEye,10,5,5
Color Limb Monster,RightEye,Rgb(255,0,0)
Position Object Monster,0,0,15
Turn Object Right Monster,90

`create world
Make object plain 2,1024,768
Color Object 2,Rgb(20,20,255)
position object 2,0,0,0
Rotate object 2,270,180,0
set object 2,1,1,0

`position camera
position camera 0,900,0
xrotate camera 90

`make inkys and blinkys
For count = 10 to Fuel
Make Object Sphere count,20
Position Object count,rnd(1024)-512,0,rnd(768)-384
Color Object count,Rgb(255,255,0)
Set Object Emissive count,Rgb(100,100,0)
Next count
NearestFuel = 10

Initialize_AI()

`----------------------------------------------------------------------------

`main loop
Do
   Input_Data()
   Process_Data()
   Output()
   Game()
   Scoring()
   Epoch()
   NewPosition()
   ScreenData()
   Debug()
   Sync
Loop

`----------------------------------------------------------------------------

`Functions

`will add deug options to change fps etc.
function Debug()
	text 10,95, " X/z: "+ str$(int(object position x(1)))+" /"+ str$(int(object position z(1)))
	for i = 1 to 300
		Line 200+(i*2),101,200+(i*2),(100-Epoch_Data(i))
	next i
endfunction

`onscreen data
Function ScreenData()
	Set Cursor 0,0
	SET TEXT FONT "verdana"
	set text size 12
 	text 10,15, " Tank Score: " + str$(int(Tank_Score#))
   	text 10,30, " Tank Avg:   " + str$ (int(Tank_Average#))
   	`Just to play, the player's score will be opposite of the Tank's.
   	text 10,50, " Epoch:  " + str$(Epoch_Counter)
   	text 10,65, " Epoch Data:  " + str$(Epoch_Data_Counter)
    text 10,80, " FPS: " + str$(Screen FPS())
    Endfunction

`new chomper position (changed to actual positions rather than screen pos)
Function NewPosition()
	A = object position x(1) `why split?
	B = Object position z(1)
	if object position x(1) > 512 then position object 1,-512,0,B
	if object position x(1) < -512 then position object 1,512,0,B
	if object position z(1) < -384 then position object 1,A,0,384
	if object position z(1) > 384 then position object 1,A,0,-384
Endfunction

`This function is added to set, or reset, our AI randomly.
Function Initialize_AI()
   For i = 0 to Array Count(Neuron())
      Neuron(i).Curve=Set_Left_Nibble(0,rnd(14)+1)
      Neuron(i).Curve=Set_Right_Nibble(Neuron(i).Curve,rnd(14)+1)
      Neuron(i).Weight=Set_Left_Nibble(0,rnd(14)+1)
      Neuron(i).Weight=Set_Right_Nibble(Neuron(i).Weight,rnd(14)+1)
      Neuron(i).Threshold=Set_Left_Nibble(0,rnd(15))
      Neuron(i).Threshold=Set_Right_Nibble(Neuron(i).Threshold,rnd(15))
   Next i
Endfunction

`We need a "game" that our Tank can learn to play.
Function Game()
   lasted# = 9999.0
   for i = 10 to fuel
   	Set Vector3 1,(Object Position X(i)-Object Position X(Tank)),0,(Object Position Z(i)-Object Position Z(Tank))
   	ed#=Length Vector3(1):`Fuel(n) Distance
   	if ed# < Lasted#
   		Lasted# = ed#
   	NearestFuel = i
   endif
   next i
Endfunction

Function Scoring()
   if Lasted# > 150.0 then  Tank_Score# = Tank_Score# - 1
if Lasted#>140.0
    Tank_Score# = Tank_Score#-(0.08+(Lasted#/10000.0)) :`The further the fuel gets, the more points lost.
   Endif
   If Lasted# <= 20.0 `distance to chomp
   Tank_Score# = Tank_Score# + 200.0
  Position Object NearestFuel,rnd(1024)-512,0,rnd(768)-384
   Endif
Endfunction

`Each Epoch (100 frames), the chomper will update it's learning.
Function Epoch()
   `Inc Epoch_Timer
   If T <= Timer()
    Inc Epoch_Counter
    Inc Epoch_Timer
    inc Epoch_Data_Counter
    If Epoch_Data_Counter = 301
    	Epoch_Data_Counter = 1
    Endif
   T = Timer()+ 500

`Endif `random endif?
   `Every 100 frames, the chomper will evaluate its performance and start over. This will eventually be the genetic evolution

`Here's where we score our neuron settings and store them for later:
   For i = 0 to Array Count(Neuron())
      For j = 0 to 1
         `n=Get_nibble(neuron(i).curve,j)
         Gene(i,j,n).curve=Gene(i,j,n).curve+Int(Sigmoidal(Tank_Score#-Tank_Average#,500)*10)
         n=Get_nibble(neuron(i).weight,j)
         Gene(i,j,n).weight=Gene(i,j,n).weight+Int(Sigmoidal(Tank_Score#-Tank_Average#,500)*10)
         n=Get_nibble(neuron(i).threshold,j)
         Gene(i,j,n).threshold=Gene(i,j,n).threshold+Int(Sigmoidal(Tank_Score#-Tank_Average#,500)*10)
      Next j
   Next i

`Initialize the AI with some random values.
   Initialize_AI()
   `Now populate the AI with some best scores. The more Epochs that go by, the less mutation occurs.
   `Note: this is sort of an inverse mutation function. The gene starts off by default as mutated.
   `      Then if the random selection comes up, the mutation is replaced by a successful gene.
   `      The more Epochs that go by, the more likely the mutations will be replaced, but never 100%.
   Get_Best_Genes()
   For i = 0 to Array Count(Neuron())
      If Rnd(100+Epoch_Counter)<Epoch_Counter
         Neuron(i).curve=Set_Left_Nibble(0,BestGenes(i,0).curve)
         Neuron(i).curve=Set_Right_Nibble(Neuron(i).curve,BestGenes(i,0).curve)
      Endif
      If Rnd(100+Epoch_Counter)<Epoch_Counter
         Neuron(i).weight=Set_Left_Nibble(0,BestGenes(i,0).weight)
         Neuron(i).weight=Set_Right_Nibble(Neuron(i).weight,BestGenes(i,0).weight)
      Endif
      If Rnd(100+Epoch_Counter)<Epoch_Counter
         Neuron(i).threshold=Set_Left_Nibble(0,BestGenes(i,0).threshold)
         Neuron(i).threshold=Set_Right_Nibble(Neuron(i).threshold,BestGenes(i,0).threshold)
      Endif

If Rnd(100+Epoch_Counter)<Epoch_Counter
         Neuron(i).curve=Set_Left_Nibble(3,BestGenes(i,3).curve)
         Neuron(i).curve=Set_Right_Nibble(Neuron(i).curve,BestGenes(i,3).curve)
      Endif
      If Rnd(100+Epoch_Counter)<Epoch_Counter
         Neuron(i).weight=Set_Left_Nibble(3,BestGenes(i,3).weight)
         Neuron(i).weight=Set_Right_Nibble(Neuron(i).weight,BestGenes(i,3).weight)
      Endif
      If Rnd(100+Epoch_Counter)<Epoch_Counter
         Neuron(i).threshold=Set_Left_Nibble(3,BestGenes(i,3).threshold)
         Neuron(i).threshold=Set_Right_Nibble(Neuron(i).threshold,BestGenes(i,3).threshold)
      Endif
   Next i

`Calculate the new average score
   `Inc Epoch_Counter
   `Weighted average so the current performance is based on past performance.
   Tank_Average#=((Tank_Average# * (Epoch_Counter-1))+Tank_Score#)/Epoch_Counter
   Epoch_data(Epoch_Data_Counter)= Tank_Average#

Tank_Score#=0
   `Epoch_Timer=0
   Initialize_AI()
   endif
Endfunction

`This function chooses the "best" genes, the genes with the highest scores.
`A feature has been added to typically select 1 of the best 3 (usually) to avoid getting stuck in a niche.
Function Get_Best_Genes()
   For i = 1 to Array Count(Neuron())
      For j=0 to 1
         best_curve1=-9999
         best_weight1=-9999
         best_threshold1=-9999
         For k=0 to 15
            If best_curve1<Gene(i,j,k).curve and rnd(3)=1  :`Typically will choose one of the best 3, but not always.
               best_curve1=Gene(i,j,k).curve
               BC1=k
            Endif
            If best_weight1<Gene(i,j,k).weight and rnd(3)=1
               best_weight1=Gene(i,j,k).weight
               BW1=k
            Endif
            If best_threshold1<Gene(i,j,k).threshold and rnd(3)=1
               best_threshold1=Gene(i,j,k).threshold
               BT1=k
            Endif
         Next k
         BestGenes(i,j).curve=BC1
         BestGenes(i,j).weight=BW1
         BestGenes(i,j).threshold=BT1
      Next j
   Next i

For i = 1 to Array Count(Neuron())
      For j=3 to 4
         best_curve2=-9999
         best_weight2=-9999
         best_threshold2=-9999
         For k=0 to 15
            If best_curve2<Gene(i,j,k).curve and rnd(3)=1  :`Typically will choose one of the best 3, but not always.
               best_curve2=Gene(i,j,k).curve
               BC=k
            Endif
            If best_weight2<Gene(i,j,k).weight and rnd(3)=1
               best_weight2=Gene(i,j,k).weight
               BW=k
            Endif
            If best_threshold2<Gene(i,j,k).threshold and rnd(3)=1
               best_threshold2=Gene(i,j,k).threshold
               BT=k
            Endif
         Next k
         BestGenes(i,j).curve=BC2
         BestGenes(i,j).weight=BW2
         BestGenes(i,j).threshold=BT2
      Next j
   Next i
Endfunction

Function Input_Data()
nearestfuel = 10
if Get_Left_Nibble(Neuron(2).value)-4 >0 :

Look_at_Fuel_Right()
Turn Object Right 1,turn#
`scroll limb texture 1, 3,-0.01,0
move object 1,1.5
`scroll limb texture 1, 3,-0.01,0
`scroll limb texture 1, 4,-0.01,0

else

Look_at_Fuel_Right()
Turn Object Left 1,turn#
`scroll limb texture 1, 4,-0.01,0
move object 1,1.5
`scroll limb texture 1, 3,-0.01,0
`scroll limb texture 1, 4,-0.01,0

Endif

Set Vector3 1,(Object Position X(NearestFuel)-Object Position X(Tank)),0,(Object Position Z(NearestFuel)-Object Position Z(Tank))
   ed#=Length Vector3(1):`Fuel(n) Distance

range#=Sigmoidal((ed#-20.0),curve)*15.0
   Neuron(1).value=Set_Left_Nibble(0,range#)

curve=Get_Right_Nibble(Neuron(1).Curve)
   range#=Sigmoidal((ed#-20.0),curve)*15.0
   Neuron(1).value=Set_Right_Nibble(Neuron(1).value,range#)

if Get_Left_Nibble(Neuron(5).value)-6.5 >0 :

Look_at_Fuel_Right()
`
move object 1,2
`scroll limb texture 1, 3,-0.005,0
`scroll limb texture 1, 4,-0.005,0

else

Look_at_Fuel_Right()

`missing code?

Endif

Set Vector3 1,(Object Position X(NearestFuel)-Object Position X(Tank)),0,(Object Position Z(NearestFuel)-Object Position Z(Tank))
   ed#=Length Vector3(1):`Fuel(n) Distance

`Here we set the player vs. Tank range Neuron
   `curve=Get_Left_Nibble(Neuron(1).Curve)
 range#=Sigmoidal((ed#-20.0),curve)*15.0

`Notice the curve variable has been added to our Sigmoidal function. This will be part of how our AI learns.
   `range#=Sigmoidal((ed#-pd#),Curve)*15.0
   Neuron(4).value=Set_Left_Nibble(3,range#)

`Here we set the Tank vs. Fuel(n) range Neuron
   curve=Get_Right_Nibble(Neuron(4).Curve)
   range#=Sigmoidal((ed#-20.0),curve)*15.0
   Neuron(4).value=Set_Right_Nibble(Neuron(4).value,range#)
Endfunction

Function Process_Data()
   n1=Get_Left_Nibble(Neuron(1).value)
   n2=Get_Right_Nibble(Neuron(1).value)
   curve=Get_Left_Nibble(Neuron(2).Curve)
   factor#=Sigmoidal((n1-n2),curve)*15.0
   Neuron(2).value=Set_Left_Nibble(0,factor#)

n1=Get_Left_Nibble(Neuron(4).value)
   n2=Get_Right_Nibble(Neuron(4).value)
   curve=Get_Left_Nibble(Neuron(5).Curve)
   factor#=Sigmoidal((n1-n2),curve)*15.0
   Neuron(5).value=Set_Left_Nibble(3,factor#)
Endfunction

Function Output()
   Neuron(0).value=0
   Neuron(3).value=0
Endfunction

Function Look_at_Fuel_Right()
best# = 500000.0
For Nearest = 10 to 50
Set Vector3 1,(Object Position X(Nearest)-Limb Position X(Tank,LeftEye)),0,(Object Position Z(Nearest)-Limb Position Z(Tank,LeftEye))
   ln#=Length Vector3(1)
   Set Vector3 1,(Object Position X(Nearest)-Limb Position X(Tank,RightEye)),0,(Object Position Z(Nearest)-Limb Position Z(Tank,RightEye))
   rn#=Length Vector3(1)
   if ln# < best#
   best# = ln#
   NearestFuel = Nearest
   Rotate = 1
   Endif

if rn# < ln#
   best# = rn#
   NearestFuel = Nearest
   Rotate = 2
   Endif
Next Nearest
   Set Vector3 1,(Object Position X(NearestFuel)-Limb Position X(Tank,LeftEye)),0,(Object Position Z(NearestFuel)-Limb Position Z(Tank,LeftEye))
   ld#=Length Vector3(1)
   Set Vector3 1,(Object Position X(NearestFuel)-Limb Position X(Tank,RightEye)),0,(Object Position Z(NearestFuel)-Limb Position Z(Tank,RightEye))
   rd#=Length Vector3(1)
   curve#=Get_Left_Nibble(Neuron(0).Curve)
   Turn#=Sigmoidal((ld#-rd#)*2.0,curve#/10.0)*15.0
   Neuron(0).value=Set_Left_Nibble(Neuron(0).value,turn#)
   turn#=((Get_Left_Nibble(Neuron(0).value)-8.0)/7.0)*4.0
   Rotate = rnd(1)+1
   if Rotate = 1
   Turn Object Left 1,turn#+1
   Turn Object Left 1,0.01

`scroll limb texture 1, 4,-0.01,0
   else
   Turn Object Right 1,turn#+1
   Turn Object Right 1,0.01
   `scroll limb texture 1, 3,-0.01,0
   endif
Endfunction

Function Sigmoidal(value#,curve#)
   `the sigmoidal formula. Graph this on a spreadsheet to see how it works.
   result#=1.0/(1.0+2.7183^-(value#/curve#))
Endfunction result#

`Simple Nibble Functions.
Function Set_Nibble(bt as byte, value as byte, nibble as byte)
   if nibble=0 then bt = Set_Right_Nibble(bt,value)
   if nibble=1 then bt = Set_Left_Nibble(bt,value)
Endfunction bt

Function Set_Left_Nibble(bt as byte,value as byte)
   value = value << 4
   bt = bt || value
Endfunction bt

Function Set_Right_Nibble(bt as byte,value as byte)
   value = value << 4
   value = value >> 4
   bt = bt || value
Endfunction bt

Function Get_Nibble(bt as byte, nibble as byte)
   if nibble=0 then bt=Get_Right_Nibble(bt)
   if nibble=1 then bt=Get_Left_Nibble(bt)
Endfunction bt

Function Get_Left_Nibble(bt as byte)
   bt = bt >> 4
Endfunction bt

Function Get_Right_Nibble(bt as byte)
   bt = bt << 4
   bt = bt >> 4
Endfunction bt

+ Code Snippet

Rem ***** Addaptive Neural Coding - Argmentum****
`Based on /using the Riidii and Pincho Paxton code.
`Variable set up - define types

`The Curve type has been added as it is clearly something we will want to "evolve"(is it? arg).
Type Neurons
   Value as byte
   Curve as byte
   Weight as byte
   Threshold as byte
Endtype

Type Genes
   Curve as Integer
   Weight as Integer
   Threshold as Integer
Endtype

Dim Neuron(5) as Neurons

`Gene is the Tank's "genetic code". Typically, the genetic code would record and consider the population.
`In this case though, we only have one Tank and one player, and that type of evolution would take a really long time.
`So we are going to take the population of posibilities and use those instead. Since a neuron can hold a value of
`0 to 15, we will track how well each setting does and keep track over many Epochs.
`We are tracking 3 neurons ( Array Count(Neurons()) ), 2 nibbles (0-1), and 16 possible settings (0-15)
`So our Dim is:
Dim Gene(Array Count(Neuron()),5,15) as Genes
Dim BestGenes(Array Count(Neuron()),5) as Genes

`epoch data storage
Dim Epoch_Data (300)  `change this to store more epochs

Global Tank_Score#
Global Tank_Average#
Global count
Global Epoch_Timer
Global Epoch_Counter
Global Epoch_Data_Counter
Global T
Global Rotate
Global Turn#

Global NearestFuel
Global ed#
Global Lasted#

#Constant LeftEye 1
#Constant RightEye 2
#Constant Tank 1
#Constant Monster 1
#Constant Fuel 50

`----------------------------------------------------------------------------

`initial set up
set display mode 1024,768,32
Sync On: Sync Rate 0: Autocam Off
Null=Make Vector3(1)
Randomize Timer()
set normalization on

`create chomper
Make Object Sphere Monster,5,6,6
Make Mesh From Object Monster,1
Delete Object Monster
Make Object Cube Monster,20
color object Monster,rgb(0,0,0)
Add Limb Monster,LeftEye,1
Offset Limb Monster,LeftEye,-10,5,5
Color Limb Monster,LeftEye,Rgb(255,0,0)
Add Limb Monster,RightEye,1
Offset Limb Monster,RightEye,10,5,5
Color Limb Monster,RightEye,Rgb(255,0,0)
Position Object Monster,0,0,15
Turn Object Right Monster,90

`create world
Make object plain 2,1024,768
Color Object 2,Rgb(20,20,255)
position object 2,0,0,0
Rotate object 2,270,180,0
set object 2,1,1,0

`position camera
position camera 0,900,0
xrotate camera 90

`make inkys and blinkys
For count = 10 to Fuel
Make Object Sphere count,20
Position Object count,rnd(1024)-512,0,rnd(768)-384
Color Object count,Rgb(255,255,0)
Set Object Emissive count,Rgb(100,100,0)
Next count
NearestFuel = 10

Initialize_AI()

`----------------------------------------------------------------------------

`main loop
Do
   Input_Data()
   Process_Data()
   Output()
   Game()
   Scoring()
   Epoch()
   NewPosition()
   ScreenData()
   Debug()
   Sync
Loop

`----------------------------------------------------------------------------

`Functions

`will add deug options to change fps etc.
function Debug()
	text 10,95, " X/z: "+ str$(int(object position x(1)))+" /"+ str$(int(object position z(1)))
	for i = 1 to 300
		Line 200+(i*2),101,200+(i*2),(100-Epoch_Data(i))
	next i
endfunction

`onscreen data
Function ScreenData()
	Set Cursor 0,0
	SET TEXT FONT "verdana"
	set text size 12
 	text 10,15, " Tank Score: " + str$(int(Tank_Score#))
   	text 10,30, " Tank Avg:   " + str$ (int(Tank_Average#))
   	`Just to play, the player's score will be opposite of the Tank's.
   	text 10,50, " Epoch:  " + str$(Epoch_Counter)
   	text 10,65, " Epoch Data:  " + str$(Epoch_Data_Counter)
    text 10,80, " FPS: " + str$(Screen FPS())
    Endfunction

`new chomper position (changed to actual positions rather than screen pos)
Function NewPosition()
	A = object position x(1) `why split?
	B = Object position z(1)
	if object position x(1) > 512 then position object 1,-512,0,B
	if object position x(1) < -512 then position object 1,512,0,B
	if object position z(1) < -384 then position object 1,A,0,384
	if object position z(1) > 384 then position object 1,A,0,-384
Endfunction

`This function is added to set, or reset, our AI randomly.
Function Initialize_AI()
   For i = 0 to Array Count(Neuron())
      Neuron(i).Curve=Set_Left_Nibble(0,rnd(14)+1)
      Neuron(i).Curve=Set_Right_Nibble(Neuron(i).Curve,rnd(14)+1)
      Neuron(i).Weight=Set_Left_Nibble(0,rnd(14)+1)
      Neuron(i).Weight=Set_Right_Nibble(Neuron(i).Weight,rnd(14)+1)
      Neuron(i).Threshold=Set_Left_Nibble(0,rnd(15))
      Neuron(i).Threshold=Set_Right_Nibble(Neuron(i).Threshold,rnd(15))
   Next i
Endfunction

`We need a "game" that our Tank can learn to play.
Function Game()
   lasted# = 9999.0
   for i = 10 to fuel
   	Set Vector3 1,(Object Position X(i)-Object Position X(Tank)),0,(Object Position Z(i)-Object Position Z(Tank))
   	ed#=Length Vector3(1):`Fuel(n) Distance
   	if ed# < Lasted#
   		Lasted# = ed#
   	NearestFuel = i
   endif
   next i
Endfunction

Function Scoring()
   if Lasted# > 150.0 then  Tank_Score# = Tank_Score# - 1
if Lasted#>140.0
    Tank_Score# = Tank_Score#-(0.08+(Lasted#/10000.0)) :`The further the fuel gets, the more points lost.
   Endif
   If Lasted# <= 20.0 `distance to chomp
   Tank_Score# = Tank_Score# + 200.0
  Position Object NearestFuel,rnd(1024)-512,0,rnd(768)-384
   Endif
Endfunction

`Each Epoch (100 frames), the chomper will update it's learning.
Function Epoch()
   `Inc Epoch_Timer
   If T <= Timer()
    Inc Epoch_Counter
    Inc Epoch_Timer
    inc Epoch_Data_Counter
    If Epoch_Data_Counter = 301
    	Epoch_Data_Counter = 1
    Endif
   T = Timer()+ 500

   `Endif `random endif?
   `Every 100 frames, the chomper will evaluate its performance and start over. This will eventually be the genetic evolution

   `Here's where we score our neuron settings and store them for later:
   For i = 0 to Array Count(Neuron())
      For j = 0 to 1
         `n=Get_nibble(neuron(i).curve,j)
         Gene(i,j,n).curve=Gene(i,j,n).curve+Int(Sigmoidal(Tank_Score#-Tank_Average#,500)*10)
         n=Get_nibble(neuron(i).weight,j)
         Gene(i,j,n).weight=Gene(i,j,n).weight+Int(Sigmoidal(Tank_Score#-Tank_Average#,500)*10)
         n=Get_nibble(neuron(i).threshold,j)
         Gene(i,j,n).threshold=Gene(i,j,n).threshold+Int(Sigmoidal(Tank_Score#-Tank_Average#,500)*10)
      Next j
   Next i



   `Initialize the AI with some random values.
   Initialize_AI()
   `Now populate the AI with some best scores. The more Epochs that go by, the less mutation occurs.
   `Note: this is sort of an inverse mutation function. The gene starts off by default as mutated.
   `      Then if the random selection comes up, the mutation is replaced by a successful gene.
   `      The more Epochs that go by, the more likely the mutations will be replaced, but never 100%.
   Get_Best_Genes()
   For i = 0 to Array Count(Neuron())
      If Rnd(100+Epoch_Counter)<Epoch_Counter
         Neuron(i).curve=Set_Left_Nibble(0,BestGenes(i,0).curve)
         Neuron(i).curve=Set_Right_Nibble(Neuron(i).curve,BestGenes(i,0).curve)
      Endif
      If Rnd(100+Epoch_Counter)<Epoch_Counter
         Neuron(i).weight=Set_Left_Nibble(0,BestGenes(i,0).weight)
         Neuron(i).weight=Set_Right_Nibble(Neuron(i).weight,BestGenes(i,0).weight)
      Endif
      If Rnd(100+Epoch_Counter)<Epoch_Counter
         Neuron(i).threshold=Set_Left_Nibble(0,BestGenes(i,0).threshold)
         Neuron(i).threshold=Set_Right_Nibble(Neuron(i).threshold,BestGenes(i,0).threshold)
      Endif

 If Rnd(100+Epoch_Counter)<Epoch_Counter
         Neuron(i).curve=Set_Left_Nibble(3,BestGenes(i,3).curve)
         Neuron(i).curve=Set_Right_Nibble(Neuron(i).curve,BestGenes(i,3).curve)
      Endif
      If Rnd(100+Epoch_Counter)<Epoch_Counter
         Neuron(i).weight=Set_Left_Nibble(3,BestGenes(i,3).weight)
         Neuron(i).weight=Set_Right_Nibble(Neuron(i).weight,BestGenes(i,3).weight)
      Endif
      If Rnd(100+Epoch_Counter)<Epoch_Counter
         Neuron(i).threshold=Set_Left_Nibble(3,BestGenes(i,3).threshold)
         Neuron(i).threshold=Set_Right_Nibble(Neuron(i).threshold,BestGenes(i,3).threshold)
      Endif
   Next i

   `Calculate the new average score
   `Inc Epoch_Counter
   `Weighted average so the current performance is based on past performance.
   Tank_Average#=((Tank_Average# * (Epoch_Counter-1))+Tank_Score#)/Epoch_Counter
   Epoch_data(Epoch_Data_Counter)= Tank_Average#

   Tank_Score#=0
   `Epoch_Timer=0
   Initialize_AI()
   endif
Endfunction

`This function chooses the "best" genes, the genes with the highest scores.
`A feature has been added to typically select 1 of the best 3 (usually) to avoid getting stuck in a niche.
Function Get_Best_Genes()
   For i = 1 to Array Count(Neuron())
      For j=0 to 1
         best_curve1=-9999
         best_weight1=-9999
         best_threshold1=-9999
         For k=0 to 15
            If best_curve1<Gene(i,j,k).curve and rnd(3)=1  :`Typically will choose one of the best 3, but not always.
               best_curve1=Gene(i,j,k).curve
               BC1=k
            Endif
            If best_weight1<Gene(i,j,k).weight and rnd(3)=1
               best_weight1=Gene(i,j,k).weight
               BW1=k
            Endif
            If best_threshold1<Gene(i,j,k).threshold and rnd(3)=1
               best_threshold1=Gene(i,j,k).threshold
               BT1=k
            Endif
         Next k
         BestGenes(i,j).curve=BC1
         BestGenes(i,j).weight=BW1
         BestGenes(i,j).threshold=BT1
      Next j
   Next i

   For i = 1 to Array Count(Neuron())
      For j=3 to 4
         best_curve2=-9999
         best_weight2=-9999
         best_threshold2=-9999
         For k=0 to 15
            If best_curve2<Gene(i,j,k).curve and rnd(3)=1  :`Typically will choose one of the best 3, but not always.
               best_curve2=Gene(i,j,k).curve
               BC=k
            Endif
            If best_weight2<Gene(i,j,k).weight and rnd(3)=1
               best_weight2=Gene(i,j,k).weight
               BW=k
            Endif
            If best_threshold2<Gene(i,j,k).threshold and rnd(3)=1
               best_threshold2=Gene(i,j,k).threshold
               BT=k
            Endif
         Next k
         BestGenes(i,j).curve=BC2
         BestGenes(i,j).weight=BW2
         BestGenes(i,j).threshold=BT2
      Next j
   Next i
Endfunction


Function Input_Data()
nearestfuel = 10
if Get_Left_Nibble(Neuron(2).value)-4 >0 :

Look_at_Fuel_Right()
Turn Object Right 1,turn#
`scroll limb texture 1, 3,-0.01,0
move object 1,1.5
`scroll limb texture 1, 3,-0.01,0
`scroll limb texture 1, 4,-0.01,0

else

Look_at_Fuel_Right()
Turn Object Left 1,turn#
`scroll limb texture 1, 4,-0.01,0
move object 1,1.5
`scroll limb texture 1, 3,-0.01,0
`scroll limb texture 1, 4,-0.01,0

Endif

   Set Vector3 1,(Object Position X(NearestFuel)-Object Position X(Tank)),0,(Object Position Z(NearestFuel)-Object Position Z(Tank))
   ed#=Length Vector3(1):`Fuel(n) Distance

   range#=Sigmoidal((ed#-20.0),curve)*15.0
   Neuron(1).value=Set_Left_Nibble(0,range#)

   curve=Get_Right_Nibble(Neuron(1).Curve)
   range#=Sigmoidal((ed#-20.0),curve)*15.0
   Neuron(1).value=Set_Right_Nibble(Neuron(1).value,range#)

 if Get_Left_Nibble(Neuron(5).value)-6.5 >0 :

 Look_at_Fuel_Right()
`
move object 1,2
`scroll limb texture 1, 3,-0.005,0
`scroll limb texture 1, 4,-0.005,0


else

Look_at_Fuel_Right()

`missing code?

Endif

 Set Vector3 1,(Object Position X(NearestFuel)-Object Position X(Tank)),0,(Object Position Z(NearestFuel)-Object Position Z(Tank))
   ed#=Length Vector3(1):`Fuel(n) Distance

   `Here we set the player vs. Tank range Neuron
   `curve=Get_Left_Nibble(Neuron(1).Curve)
 range#=Sigmoidal((ed#-20.0),curve)*15.0

   `Notice the curve variable has been added to our Sigmoidal function. This will be part of how our AI learns.
   `range#=Sigmoidal((ed#-pd#),Curve)*15.0
   Neuron(4).value=Set_Left_Nibble(3,range#)

   `Here we set the Tank vs. Fuel(n) range Neuron
   curve=Get_Right_Nibble(Neuron(4).Curve)
   range#=Sigmoidal((ed#-20.0),curve)*15.0
   Neuron(4).value=Set_Right_Nibble(Neuron(4).value,range#)
Endfunction

Function Process_Data()
   n1=Get_Left_Nibble(Neuron(1).value)
   n2=Get_Right_Nibble(Neuron(1).value)
   curve=Get_Left_Nibble(Neuron(2).Curve)
   factor#=Sigmoidal((n1-n2),curve)*15.0
   Neuron(2).value=Set_Left_Nibble(0,factor#)

   n1=Get_Left_Nibble(Neuron(4).value)
   n2=Get_Right_Nibble(Neuron(4).value)
   curve=Get_Left_Nibble(Neuron(5).Curve)
   factor#=Sigmoidal((n1-n2),curve)*15.0
   Neuron(5).value=Set_Left_Nibble(3,factor#)
Endfunction

Function Output()
   Neuron(0).value=0
   Neuron(3).value=0
Endfunction

Function Look_at_Fuel_Right()
best# = 500000.0
For Nearest = 10 to 50
Set Vector3 1,(Object Position X(Nearest)-Limb Position X(Tank,LeftEye)),0,(Object Position Z(Nearest)-Limb Position Z(Tank,LeftEye))
   ln#=Length Vector3(1)
   Set Vector3 1,(Object Position X(Nearest)-Limb Position X(Tank,RightEye)),0,(Object Position Z(Nearest)-Limb Position Z(Tank,RightEye))
   rn#=Length Vector3(1)
   if ln# < best#
   best# = ln#
   NearestFuel = Nearest
   Rotate = 1
   Endif

if rn# < ln#
   best# = rn#
   NearestFuel = Nearest
   Rotate = 2
   Endif
Next Nearest
   Set Vector3 1,(Object Position X(NearestFuel)-Limb Position X(Tank,LeftEye)),0,(Object Position Z(NearestFuel)-Limb Position Z(Tank,LeftEye))
   ld#=Length Vector3(1)
   Set Vector3 1,(Object Position X(NearestFuel)-Limb Position X(Tank,RightEye)),0,(Object Position Z(NearestFuel)-Limb Position Z(Tank,RightEye))
   rd#=Length Vector3(1)
   curve#=Get_Left_Nibble(Neuron(0).Curve)
   Turn#=Sigmoidal((ld#-rd#)*2.0,curve#/10.0)*15.0
   Neuron(0).value=Set_Left_Nibble(Neuron(0).value,turn#)
   turn#=((Get_Left_Nibble(Neuron(0).value)-8.0)/7.0)*4.0
   Rotate = rnd(1)+1
   if Rotate = 1
   Turn Object Left 1,turn#+1
   Turn Object Left 1,0.01

	`scroll limb texture 1, 4,-0.01,0
   else
   Turn Object Right 1,turn#+1
   Turn Object Right 1,0.01
   `scroll limb texture 1, 3,-0.01,0
   endif
Endfunction


Function Sigmoidal(value#,curve#)
   `the sigmoidal formula. Graph this on a spreadsheet to see how it works.
   result#=1.0/(1.0+2.7183^-(value#/curve#))
Endfunction result#

`Simple Nibble Functions.
Function Set_Nibble(bt as byte, value as byte, nibble as byte)
   if nibble=0 then bt = Set_Right_Nibble(bt,value)
   if nibble=1 then bt = Set_Left_Nibble(bt,value)
Endfunction bt

Function Set_Left_Nibble(bt as byte,value as byte)
   value = value << 4
   bt = bt || value
Endfunction bt

Function Set_Right_Nibble(bt as byte,value as byte)
   value = value << 4
   value = value >> 4
   bt = bt || value
Endfunction bt

Function Get_Nibble(bt as byte, nibble as byte)
   if nibble=0 then bt=Get_Right_Nibble(bt)
   if nibble=1 then bt=Get_Left_Nibble(bt)
Endfunction bt

Function Get_Left_Nibble(bt as byte)
   bt = bt >> 4
Endfunction bt

Function Get_Right_Nibble(bt as byte)
   bt = bt << 4
   bt = bt >> 4
Endfunction bt

'Ooh 'eck chief'...'crumbs'

Back to top

Profile PM

Penfold

22

Years of Service

User Offline

Joined: 3rd Dec 2003

Location: RED postbox houses of parliment

Posted: 26th Feb 2012 23:32

Link

Newer cleaner code with a major code issue fixed.

I can get the little fella to average out at about 120 points after about 700 epochs - can anyone get one thats smarter?

Code requires no external media just copy paste n compile.

Rem ***** Addaptive Neural Coding - Argmentum****
`Based on /using the Riidii and Pincho Paxton code.
`Variable set up - define types

`The Curve type has been added as it is clearly something we will want to "evolve"(is it? arg).
Type Neurons
   Value as byte
   Curve as byte
   Weight as byte
   Threshold as byte
Endtype

Type Genes
   Curve as Integer
   Weight as Integer
   Threshold as Integer
Endtype

Dim Neuron(5) as Neurons

`Gene is the Tank's "genetic code". Typically, the genetic code would record and consider the population.
`In this case though, we only have one Tank and one player, and that type of evolution would take a really long time.
`So we are going to take the population of posibilities and use those instead. Since a neuron can hold a value of
`0 to 15, we will track how well each setting does and keep track over many Epochs.
`We are tracking 3 neurons ( Array Count(Neurons()) ), 2 nibbles (0-1), and 16 possible settings (0-15)
`So our Dim is:
Dim Gene(Array Count(Neuron()),5,15) as Genes
Dim BestGenes(Array Count(Neuron()),5) as Genes

`epoch data storage
Dim Epoch_Data (300)  `change this to store more epochs

Global Tank_Score#
Global Tank_Average#
Global count
Global Epoch_Timer
Global Epoch_Counter
Global Epoch_Data_Counter
Global T
Global Rotate
Global Turn#

Global NearestFuel
Global ed#
Global Lasted#

#Constant LeftEye 1
#Constant RightEye 2
#Constant Tank 1
#Constant Monster 1
#Constant Fuel 50

`----------------------------------------------------------------------------

`initial set up
set display mode 1024,768,32
Sync On: Sync Rate 0: Autocam Off
Null=Make Vector3(1)
Randomize Timer()
set normalization on

`create chomper
Make Object Sphere Monster,5,6,6
Make Mesh From Object Monster,1
Delete Object Monster
Make Object Cube Monster,20
color object Monster,rgb(0,0,0)
Add Limb Monster,LeftEye,1
Offset Limb Monster,LeftEye,-10,5,5
Color Limb Monster,LeftEye,Rgb(255,0,0)
Add Limb Monster,RightEye,1
Offset Limb Monster,RightEye,10,5,5
Color Limb Monster,RightEye,Rgb(255,0,0)
Position Object Monster,0,0,15
Turn Object Right Monster,90

`create world
Make object plain 2,1024,768
Color Object 2,Rgb(20,20,255)
position object 2,0,0,0
Rotate object 2,270,180,0
set object 2,1,1,0

`position camera
position camera 0,900,100
xrotate camera 90

`make inkys and blinkys
For count = 10 to Fuel
Make Object Sphere count,20
Position Object count,rnd(1024)-512,0,rnd(768)-384
Color Object count,Rgb(255,255,0)
Set Object Emissive count,Rgb(100,100,0)
Next count
NearestFuel = 10

Initialize_AI()

`----------------------------------------------------------------------------

`main loop
Do
   Input_Data()
   Process_Data()
   Output()
   Game()
   Scoring()
   Epoch()
   NewPosition()
   ScreenData()
   Debug()
   Sync
Loop

`----------------------------------------------------------------------------

`Functions

`will add deug options to change fps etc.
function Debug()
	text 10,95, " X/z: "+ str$(int(object position x(1)))+" /"+ str$(int(object position z(1)))
	for i = 1 to 300
		Line 200+(i*2),151,200+(i*2),(150-Epoch_Data(i))
		if i= Epoch_Data_Counter
			ink rgb(0,255,0),0
			Line 200+(i*2),151,200+(i*2),(150-Epoch_Data(i))
			ink rgb(255,255,255),0
		endif
	next i
	ink rgb(255,0,0),0
	Line 200,50,800,50
	ink rgb(255,255,255),0
endfunction