learning AI [open source] - GameCreators Forum

Author

Message

El Goorf

17

Years of Service

User Offline

Joined: 17th Sep 2006

Location: Uni: Manchester, Home: Dunstable

Posted: 17th Feb 2007 02:13 Edited at: 17th Feb 2007 02:18

Link

about two years ago I tried naughts and crosses, but with a different approach to AI. Instead of programming it to make lines, or block off th eplayers lines, i wanted it to randomly pick which square to play in, and learn over time which moves are good or bad by storing the results of different combinations of moves.

that project went uncompleted but this morning I went back and revisitted the project and have just completed it.

It works by going through a list of all the previous games it's had where the same combinations of moves have been played. for each reamining available square, the ai +'s or -'s the value of the square by counting how many games have been won or lost as a result of making that move, and then going for the one with the greatest value since it has the most chance of winning, judging from history.

and now, the code:

Rem Project: AI crosses
Rem Created: 15/02/2007 20:03:37

Rem ***** Main Source File *****
sync off
randomize timer()

global turn as boolean
global x
global y
global cond$
global id$
global remain

dim grid(2,2) as byte
dim brain$(-1)
dim aichoose(8)

if file exist("brainsize.dat")
   open to read 1,"brainsize.dat"
   read long 1,size
   close file 1
   for t=0 to size
      add to queue brain$(0)
   next t
   load array "brain.dat",brain$(0)
endif

set window on
set window size 600,650
set display mode 600,650,32
set window position 100,100
set window title "1's and 0's: Your turn."

start:
cls
print size
drawui()
remain=8
id$=""
turn = 0
for t = 0 to 8
   aichoose(t)=0
next t
for s=0 to 2
   for t=0 to 2
      grid(s,t)=2
   next t
next s

do
   updatewindowtitle()
   if turn=0
      repeat
         square=waitclick()
      until validmove(x,y)
   endif

if turn=1
      square=aiclever()
      if square=10 then square=airandom(remain)
   endif

aichoose(square)=-10000
   `update info
   grid(x,y)=turn
   center text x*200+100,y*200+100,str$(turn)
   id$=id$+str$(square)
   text 10,620,id$
   dec remain

winner=checkwin()
   if winner<2
      print "winner: ";winner
      print cond$
      if winner=0 then id$=id$+"L" else id$=id$+"W"
      array insert at bottom brain$(0)
      brain$(array count(brain$(0)))=id$
      save array "brain.dat",brain$(0)

delete file "brainsize.dat"
      open to write 1,"brainsize.dat"
      write long 1,array count(brain$(0))
      close file 1

wait key
      goto start
   endif

inc turn : if turn=2 then turn=0
   if remain=-1
      print "draw"
      id$=id$+"D"
      array insert at bottom brain$(0)
      brain$(array count(brain$(0)))=id$
      save array "brain.dat",brain$(0)

delete file "brainsize.dat"
      open to write 1,"brainsize.dat"
      write long 1,array count(brain$(0))
      close file 1

wait key
      goto start
   endif
loop

function drawui()
   line 200,0,200,600
   line 400,0,400,600
   line 0,200,600,200
   line 0,400,600,400
endfunction

function updatewindowtitle()
   if turn = 0 then set window title "1's and 0's: Your turn."
   if turn = 1 then set window title "1's and 0's: AI's turn."
endfunction

function waitclick()
   square=10
   repeat
      if mouseclick()=1 and mousebetween(0,0,600,600)
         x=mousex()/200
         y=mousey()/200
         square=y*3+x
      endif
   until square<10
endfunction square

function validmove(x,y)
   if grid(x,y)=2
      valid=1
   else
      `print "invalid"
   endif
endfunction valid

function airandom(remain)
   x=-1
   y=0
   s=-1
   t=-1
   wait rnd(500) `simulate thinking time
   choose=rnd(remain)
   repeat
      inc t
      inc x
      if x=3 then x=0 : inc y
      if x<3 and y<3
         if grid(x,y)=2 then inc s
      endif
   until s=choose
   square=t
endfunction square

function checkwin()
   win=2
   for a=0 to 2
      if grid(a,0)=grid(a,1) and grid(a,1)=grid(a,2) then win=grid(a,0) : cond$="vert "+str$(a)
      if grid(0,a)=grid(1,a) and grid(1,a)=grid(2,a) then win=grid(0,a) : cond$="hori "+str$(a)
   next a
   if grid(0,0)=grid(1,1) and grid(1,1)=grid(2,2) then win=grid(0,0) : cond$="diag DR"
   if grid(2,0)=grid(1,1) and grid(1,1)=grid(0,2) then win=grid(2,0) : cond$="diag UL"
endfunction win

function aiclever()
   for t = 0 to 8
      if aichoose(t)>-10000 then aichoose(t)=0
   next t

square=10
   for t=0 to array count(brain$(0))
      if left$(brain$(t),len(id$)-1)=left$(id$,len(id$)-1)
         `print brain$(t)
         slot=val(mid$(brain$(t),len(id$)+1))
         if right$(brain$(t),1)="W" then inc aichoose(slot)
         if right$(brain$(t),1)="L" then dec aichoose(slot)
      endif
   next t

`disp results:
   `center text 150-remain*10,130,str$(aichoose(0)) : center text 350-remain*10,130,str$(aichoose(1)) : center text 550-remain*10,130,str$(aichoose(2))
   `center text 150-remain*10,330,str$(aichoose(3)) : center text 350-remain*10,330,str$(aichoose(4)) : center text 550-remain*10,330,str$(aichoose(5))
   `center text 150-remain*10,530,str$(aichoose(6)) : center text 350-remain*10,530,str$(aichoose(7)) : center text 550-remain*10,530,str$(aichoose(8))

`go with best option
   option=0
   for t=0 to 8
      if aichoose(t)>aichoose(option) then option=t
   next t

x=-1
   y=0
   for t=0 to option
      inc x
      if x=3
         x=0
         inc y
      endif
   next t

if grid(x,y)=2
      grid(x,y)=1
      square=option
   else
      `we have a problem
   endif
endfunction square

function mousebetween(x1,y1,x2,y2)
   if mousex()=>x1 and mousex()=<x2
      if mousey()=>y1 and mousey()<=y2
         res=1
      endif
   endif
endfunction res

+ Code Snippet

Rem Project: AI crosses
Rem Created: 15/02/2007 20:03:37

Rem ***** Main Source File *****
sync off
randomize timer()

global turn as boolean
global x
global y
global cond$
global id$
global remain

dim grid(2,2) as byte
dim brain$(-1)
dim aichoose(8)

if file exist("brainsize.dat")
   open to read 1,"brainsize.dat"
   read long 1,size
   close file 1
   for t=0 to size
      add to queue brain$(0)
   next t
   load array "brain.dat",brain$(0)
endif

set window on
set window size 600,650
set display mode 600,650,32
set window position 100,100
set window title "1's and 0's: Your turn."

start:
cls
print size
drawui()
remain=8
id$=""
turn = 0
for t = 0 to 8
   aichoose(t)=0
next t
for s=0 to 2
   for t=0 to 2
      grid(s,t)=2
   next t
next s

do
   updatewindowtitle()
   if turn=0
      repeat
         square=waitclick()
      until validmove(x,y)
   endif

   if turn=1
      square=aiclever()
      if square=10 then square=airandom(remain)
   endif

   aichoose(square)=-10000
   `update info
   grid(x,y)=turn
   center text x*200+100,y*200+100,str$(turn)
   id$=id$+str$(square)
   text 10,620,id$
   dec remain

   winner=checkwin()
   if winner<2
      print "winner: ";winner
      print cond$
      if winner=0 then id$=id$+"L" else id$=id$+"W"
      array insert at bottom brain$(0)
      brain$(array count(brain$(0)))=id$
      save array "brain.dat",brain$(0)

      delete file "brainsize.dat"
      open to write 1,"brainsize.dat"
      write long 1,array count(brain$(0))
      close file 1

      wait key
      goto start
   endif

   inc turn : if turn=2 then turn=0
   if remain=-1
      print "draw"
      id$=id$+"D"
      array insert at bottom brain$(0)
      brain$(array count(brain$(0)))=id$
      save array "brain.dat",brain$(0)

      delete file "brainsize.dat"
      open to write 1,"brainsize.dat"
      write long 1,array count(brain$(0))
      close file 1

      wait key
      goto start
   endif
loop

function drawui()
   line 200,0,200,600
   line 400,0,400,600
   line 0,200,600,200
   line 0,400,600,400
endfunction

function updatewindowtitle()
   if turn = 0 then set window title "1's and 0's: Your turn."
   if turn = 1 then set window title "1's and 0's: AI's turn."
endfunction

function waitclick()
   square=10
   repeat
      if mouseclick()=1 and mousebetween(0,0,600,600)
         x=mousex()/200
         y=mousey()/200
         square=y*3+x
      endif
   until square<10
endfunction square

function validmove(x,y)
   if grid(x,y)=2
      valid=1
   else
      `print "invalid"
   endif
endfunction valid

function airandom(remain)
   x=-1
   y=0
   s=-1
   t=-1
   wait rnd(500) `simulate thinking time
   choose=rnd(remain)
   repeat
      inc t
      inc x
      if x=3 then x=0 : inc y
      if x<3 and y<3
         if grid(x,y)=2 then inc s
      endif
   until s=choose
   square=t
endfunction square

function checkwin()
   win=2
   for a=0 to 2
      if grid(a,0)=grid(a,1) and grid(a,1)=grid(a,2) then win=grid(a,0) : cond$="vert "+str$(a)
      if grid(0,a)=grid(1,a) and grid(1,a)=grid(2,a) then win=grid(0,a) : cond$="hori "+str$(a)
   next a
   if grid(0,0)=grid(1,1) and grid(1,1)=grid(2,2) then win=grid(0,0) : cond$="diag DR"
   if grid(2,0)=grid(1,1) and grid(1,1)=grid(0,2) then win=grid(2,0) : cond$="diag UL"
endfunction win

function aiclever()
   for t = 0 to 8
      if aichoose(t)>-10000 then aichoose(t)=0
   next t

   square=10
   for t=0 to array count(brain$(0))
      if left$(brain$(t),len(id$)-1)=left$(id$,len(id$)-1)
         `print brain$(t)
         slot=val(mid$(brain$(t),len(id$)+1))
         if right$(brain$(t),1)="W" then inc aichoose(slot)
         if right$(brain$(t),1)="L" then dec aichoose(slot)
      endif
   next t

   `disp results:
   `center text 150-remain*10,130,str$(aichoose(0)) : center text 350-remain*10,130,str$(aichoose(1)) : center text 550-remain*10,130,str$(aichoose(2))
   `center text 150-remain*10,330,str$(aichoose(3)) : center text 350-remain*10,330,str$(aichoose(4)) : center text 550-remain*10,330,str$(aichoose(5))
   `center text 150-remain*10,530,str$(aichoose(6)) : center text 350-remain*10,530,str$(aichoose(7)) : center text 550-remain*10,530,str$(aichoose(8))

   `go with best option
   option=0
   for t=0 to 8
      if aichoose(t)>aichoose(option) then option=t
   next t

   x=-1
   y=0
   for t=0 to option
      inc x
      if x=3
         x=0
         inc y
      endif
   next t

   if grid(x,y)=2
      grid(x,y)=1
      square=option
   else
      `we have a problem
   endif
endfunction square

function mousebetween(x1,y1,x2,y2)
   if mousex()=>x1 and mousex()=<x2
      if mousey()=>y1 and mousey()<=y2
         res=1
      endif
   endif
endfunction res

however, at the moment it only judges what to do next by its own history of winning or losing. what i need to do now is also judge which square to go in by observing the player's winnings and losings. for example, if a player has one most by playing in a particular square, then the ai needs to go for that square to stop the player getting it.

Tell me what you think so far

edit: oops forgot to state controls.
click on square to play it, once a game is played, press the any key to start a new game.

Back to top

Profile PM Website

HyperByte

17

Years of Service

User Offline

Joined: 3rd Jan 2007

Location: in a small kola bear costume

Posted: 17th Feb 2007 05:17 Edited at: 17th Feb 2007 05:17

Link

Wow it works pretty good but there are a few problems with the AI but Im sure you'll be able to fix it. But good job so far.

(since I tested yours you should test mine "secret knock")

On a scale from one to ten one being very smart and ten being an absolute genius your none of the above

Back to top

Profile PM Email

El Goorf

17

Years of Service

User Offline

Joined: 17th Sep 2006

Location: Uni: Manchester, Home: Dunstable

Posted: 17th Feb 2007 11:34 Edited at: 18th Feb 2007 17:33

Link

Rem Project: AI crosses
Rem Created: 15/02/2007 20:03:37

Rem ***** Main Source File *****
sync off
randomize timer()

global turn as boolean
global x
global y
global cond$
global id$
global remain

dim grid(2,2) as byte
dim brain$(-1)
dim aichoose(8)

if file exist("brainsize.dat")
   open to read 1,"brainsize.dat"
   read long 1,size
   close file 1
   for t=0 to size
      add to queue brain$(0)
   next t
   load array "brain.dat",brain$(0)
endif

set window on
set window size 600,650
set display mode 600,650,32
set window position 100,100
set window title "1's and 0's: Your turn."

start:
cls
print size
drawui()
remain=8
id$=""
turn = 0
for t = 0 to 8
   aichoose(t)=0
next t
for s=0 to 2
   for t=0 to 2
      grid(s,t)=2
   next t
next s

do
   updatewindowtitle()
   if turn=0
      repeat
         square=waitclick()
      until validmove(x,y)
   endif

if turn=1
      square=aiclever()
      if square=10 then square=airandom(remain)
   endif

aichoose(square)=-10000
   `update info
   grid(x,y)=turn
   center text x*200+100,y*200+100,str$(turn)
   id$=id$+str$(square)
   text 10,620,id$
   dec remain

winner=checkwin()
   if winner<2
      print "winner: ";winner
      print cond$
      if winner=0 then id$=id$+"L" else id$=id$+"W"

add=1
      for t = 0 to array count(brain$(0))
         if brain$(t)=id$ then add=0
      next t

if add=1
         text 10,640,"adding"
         array insert at bottom brain$(0)
         brain$(array count(brain$(0)))=id$
         save array "brain.dat",brain$(0)

delete file "brainsize.dat"
         open to write 1,"brainsize.dat"
         write long 1,array count(brain$(0))
         close file 1
      else
         text 10,640,"combo already done. not adding."
      endif

wait key
      goto start
   endif

inc turn : if turn=2 then turn=0
   if remain=-1
      print "draw"
      id$=id$+"D"
      array insert at bottom brain$(0)
      brain$(array count(brain$(0)))=id$
      save array "brain.dat",brain$(0)

delete file "brainsize.dat"
      open to write 1,"brainsize.dat"
      write long 1,array count(brain$(0))
      close file 1

wait key
      goto start
   endif
loop

function drawui()
   line 200,0,200,600
   line 400,0,400,600
   line 0,200,600,200
   line 0,400,600,400
endfunction

function updatewindowtitle()
   if turn = 0 then set window title "1's and 0's: Your turn."
   if turn = 1 then set window title "1's and 0's: AI's turn."
endfunction

function waitclick()
   square=10
   repeat
      if mouseclick()=1 and mousebetween(0,0,600,600)
         x=mousex()/200
         y=mousey()/200
         square=y*3+x
      endif
   until square<10
endfunction square

function validmove(x,y)
   if grid(x,y)=2
      valid=1
   else
      `print "invalid"
   endif
endfunction valid

function airandom(remain)
   x=-1
   y=0
   s=-1
   t=-1
   wait rnd(500) `simulate thinking time
   choose=rnd(remain)
   repeat
      inc t
      inc x
      if x=3 then x=0 : inc y
      if x<3 and y<3
         if grid(x,y)=2 then inc s
      endif
   until s=choose
   square=t
endfunction square

function checkwin()
   win=2
   for a=0 to 2
      if grid(a,0)=grid(a,1) and grid(a,1)=grid(a,2) then win=grid(a,0) : cond$="vert "+str$(a)
      if grid(0,a)=grid(1,a) and grid(1,a)=grid(2,a) then win=grid(0,a) : cond$="hori "+str$(a)
   next a
   if grid(0,0)=grid(1,1) and grid(1,1)=grid(2,2) then win=grid(0,0) : cond$="diag DR"
   if grid(2,0)=grid(1,1) and grid(1,1)=grid(0,2) then win=grid(2,0) : cond$="diag UL"
endfunction win

function aiclever()
   for t = 0 to 8
      if aichoose(t)>-10000 then aichoose(t)=0
   next t

square=10
   for t=0 to array count(brain$(0))
      if left$(brain$(t),len(id$)-1)=left$(id$,len(id$)-1)
         `print brain$(t)
         slot=val(mid$(brain$(t),len(id$)+1))
         slot2=val(mid$(brain$(t),len(id$)+2))

if right$(brain$(t),1)="W" then inc aichoose(slot),2
         if right$(brain$(t),1)="L"
            dec aichoose(slot)
            inc aichoose(slot2),10
         endif
      endif
   next t

`disp results:
   `center text 150-remain*10,130,str$(aichoose(0)) : center text 350-remain*10,130,str$(aichoose(1)) : center text 550-remain*10,130,str$(aichoose(2))
   `center text 150-remain*10,330,str$(aichoose(3)) : center text 350-remain*10,330,str$(aichoose(4)) : center text 550-remain*10,330,str$(aichoose(5))
   `center text 150-remain*10,530,str$(aichoose(6)) : center text 350-remain*10,530,str$(aichoose(7)) : center text 550-remain*10,530,str$(aichoose(8))

`go with best option
   option=0
   for t=0 to 8
      if aichoose(t)>aichoose(option) then option=t
   next t

x=-1
   y=0
   for t=0 to option
      inc x
      if x=3
         x=0
         inc y
      endif
   next t

if grid(x,y)=2
      grid(x,y)=1
      square=option
   else
      `we have a problem
   endif
endfunction square

function mousebetween(x1,y1,x2,y2)
   if mousex()=>x1 and mousex()=<x2
      if mousey()=>y1 and mousey()<=y2
         res=1
      endif
   endif
endfunction res

+ Code Snippet

Rem Project: AI crosses
Rem Created: 15/02/2007 20:03:37

Rem ***** Main Source File *****
sync off
randomize timer()

global turn as boolean
global x
global y
global cond$
global id$
global remain

dim grid(2,2) as byte
dim brain$(-1)
dim aichoose(8)

if file exist("brainsize.dat")
   open to read 1,"brainsize.dat"
   read long 1,size
   close file 1
   for t=0 to size
      add to queue brain$(0)
   next t
   load array "brain.dat",brain$(0)
endif

set window on
set window size 600,650
set display mode 600,650,32
set window position 100,100
set window title "1's and 0's: Your turn."

start:
cls
print size
drawui()
remain=8
id$=""
turn = 0
for t = 0 to 8
   aichoose(t)=0
next t
for s=0 to 2
   for t=0 to 2
      grid(s,t)=2
   next t
next s

do
   updatewindowtitle()
   if turn=0
      repeat
         square=waitclick()
      until validmove(x,y)
   endif

   if turn=1
      square=aiclever()
      if square=10 then square=airandom(remain)
   endif

   aichoose(square)=-10000
   `update info
   grid(x,y)=turn
   center text x*200+100,y*200+100,str$(turn)
   id$=id$+str$(square)
   text 10,620,id$
   dec remain

   winner=checkwin()
   if winner<2
      print "winner: ";winner
      print cond$
      if winner=0 then id$=id$+"L" else id$=id$+"W"

      add=1
      for t = 0 to array count(brain$(0))
         if brain$(t)=id$ then add=0
      next t

      if add=1
         text 10,640,"adding"
         array insert at bottom brain$(0)
         brain$(array count(brain$(0)))=id$
         save array "brain.dat",brain$(0)

         delete file "brainsize.dat"
         open to write 1,"brainsize.dat"
         write long 1,array count(brain$(0))
         close file 1
      else
         text 10,640,"combo already done. not adding."
      endif

      wait key
      goto start
   endif

   inc turn : if turn=2 then turn=0
   if remain=-1
      print "draw"
      id$=id$+"D"
      array insert at bottom brain$(0)
      brain$(array count(brain$(0)))=id$
      save array "brain.dat",brain$(0)

      delete file "brainsize.dat"
      open to write 1,"brainsize.dat"
      write long 1,array count(brain$(0))
      close file 1

      wait key
      goto start
   endif
loop

function drawui()
   line 200,0,200,600
   line 400,0,400,600
   line 0,200,600,200
   line 0,400,600,400
endfunction

function updatewindowtitle()
   if turn = 0 then set window title "1's and 0's: Your turn."
   if turn = 1 then set window title "1's and 0's: AI's turn."
endfunction

function waitclick()
   square=10
   repeat
      if mouseclick()=1 and mousebetween(0,0,600,600)
         x=mousex()/200
         y=mousey()/200
         square=y*3+x
      endif
   until square<10
endfunction square

function validmove(x,y)
   if grid(x,y)=2
      valid=1
   else
      `print "invalid"
   endif
endfunction valid

function airandom(remain)
   x=-1
   y=0
   s=-1
   t=-1
   wait rnd(500) `simulate thinking time
   choose=rnd(remain)
   repeat
      inc t
      inc x
      if x=3 then x=0 : inc y
      if x<3 and y<3
         if grid(x,y)=2 then inc s
      endif
   until s=choose
   square=t
endfunction square

function checkwin()
   win=2
   for a=0 to 2
      if grid(a,0)=grid(a,1) and grid(a,1)=grid(a,2) then win=grid(a,0) : cond$="vert "+str$(a)
      if grid(0,a)=grid(1,a) and grid(1,a)=grid(2,a) then win=grid(0,a) : cond$="hori "+str$(a)
   next a
   if grid(0,0)=grid(1,1) and grid(1,1)=grid(2,2) then win=grid(0,0) : cond$="diag DR"
   if grid(2,0)=grid(1,1) and grid(1,1)=grid(0,2) then win=grid(2,0) : cond$="diag UL"
endfunction win

function aiclever()
   for t = 0 to 8
      if aichoose(t)>-10000 then aichoose(t)=0
   next t

   square=10
   for t=0 to array count(brain$(0))
      if left$(brain$(t),len(id$)-1)=left$(id$,len(id$)-1)
         `print brain$(t)
         slot=val(mid$(brain$(t),len(id$)+1))
         slot2=val(mid$(brain$(t),len(id$)+2))

         if right$(brain$(t),1)="W" then inc aichoose(slot),2
         if right$(brain$(t),1)="L"
            dec aichoose(slot)
            inc aichoose(slot2),10
         endif
      endif
   next t

   `disp results:
   `center text 150-remain*10,130,str$(aichoose(0)) : center text 350-remain*10,130,str$(aichoose(1)) : center text 550-remain*10,130,str$(aichoose(2))
   `center text 150-remain*10,330,str$(aichoose(3)) : center text 350-remain*10,330,str$(aichoose(4)) : center text 550-remain*10,330,str$(aichoose(5))
   `center text 150-remain*10,530,str$(aichoose(6)) : center text 350-remain*10,530,str$(aichoose(7)) : center text 550-remain*10,530,str$(aichoose(8))

   `go with best option
   option=0
   for t=0 to 8
      if aichoose(t)>aichoose(option) then option=t
   next t

   x=-1
   y=0
   for t=0 to option
      inc x
      if x=3
         x=0
         inc y
      endif
   next t

   if grid(x,y)=2
      grid(x,y)=1
      square=option
   else
      `we have a problem
   endif
endfunction square

function mousebetween(x1,y1,x2,y2)
   if mousex()=>x1 and mousex()=<x2
      if mousey()=>y1 and mousey()<=y2
         res=1
      endif
   endif
endfunction res

the AI now has the ability to predict the human's next move by looking at past games.

I've also stopped it from storing repeats of games, to both save memory and stop biasing its desisions unnessecerily (sp?).

Given the number of possible combinations for th egame to take place, the AI will be extremely dumb to begin with. I find it best to always go for the same win every time, for example I always aim for a diagonal down-left. after a just a few games, the AI will manage to make the game a draw 80% of the time, win about 5%, whenever I go for the said diagonal.

edit: just spotted a glitch when detecting wins. im on it now and then i think i'll declare this finished.

edit2: the win condition glitch seems to have been a one off. I noticed that the AI was no repeating moves that caused it to lose, i found that re-enabling the logging of repeated games helped solve this.

at line 83 change the block of text to this:

+ Code Snippet

      text 10,630,"adding"
      array insert at bottom brain$(0)
      brain$(array count(brain$(0)))=id$
      save array "brain.dat",brain$(0)

      delete file "brainsize.dat"
      open to write 1,"brainsize.dat"
      write long 1,array count(brain$(0))
      close file 1

hence, final code: