Thanks for looking at this Phaelax. I was sure both didn't work, but it turns out the second one, which I wasn't using, worked after all after I retested it. I know I didn't just replace it for no reason... maybe it was a bug somewhere else that messed it up when I tested the second one, or something...
In any case, thanks
The first one on the other hand doesn't. Here's the test I made that tests against the mouse, with yours being function number 3. They appear to work at first until you change the order of the vertexes twice (by hitting space).
sync on
sync rate 0
ox=300
oy=300
x1=100
y1=100
x2=-100
y2=100
x3=100
y3=-100
do
cls 0
line x1+ox,y1+oy,x2+ox,y2+oy
line x1+ox,y1+oy,x3+ox,y3+oy
line x3+ox,y3+oy,x2+ox,y2+oy
dot ox,oy,0xFFFF0000
dec ox,(rightkey()-leftkey())
dec oy,(downkey()-upkey())
if spacekey()=0 and ps=1 `Hit space to rotate the vertexes
cx=x1
cy=y1
x1=x2
y1=y2
x2=x3
y2=y3
x3=cx
y3=cy
endif
ps=spacekey()
text 0,0, "function 1: "+str$(pointintriangle1(mousex()-ox,mousey()-oy,x1,y1,x2,y2,x3,y3))
text 0,10, "function 2: "+str$(pointintriangle2(mousex()-ox,mousey()-oy,x1,y1,x2,y2,x3,y3))
text 0,20, "function 3: "+str$(pointintriangle3(mousex()-ox,mousey()-oy,x1,y1,x2,y2,x3,y3))
sync
loop
function PointinTriangle1(x#, y#, x1#, y1#, x2#, y2#, x3#, y3#)
`by Sven B at http://forum.thegamecreators.com/?m=forum_view&t=122406&b=7
`Initialize some variables
dec x2#, x1# : dec y2#, y1#
dec x3#, x1# : dec y3#, y1#
dec x#, x1# : dec y#, y1#
`Get L
L# = ((x2# * y#) - (x# * y2#)) / ((x2# * y3#) - (y2# * x3#));
`Exlude 0
if L# < 0 then exitfunction 0
`Get newX
newX# = (x# - (x3# * L#)) / x2#
if newX# < 0.0 then exitfunction 0
`Final check
if newX# > 1 - L# then exitfunction 0
`Point is in triangle
endfunction 1
function PointinTriangle2(px as float, py as float, x1 as float,y1 as float,x2 as float,y2 as float,x3 as float,y3 as float)
dABx = x2-x1
dABy = y2-y1
dBCx = x3-x2
dBCy = y3-y2
`messagebox("px:"+str$(px,2)+"py:"+str$(py,2),"x1:"+str$(x1,2)+" y1:"+str$(y1,2)+" x2:"+str$(x2,2)+" y2:"+str$(y2,2)+" x3:"+str$(x3,2)+" y3:"+str$(y3,2),0)
rem if verts are clockwise
if (dABx*dBCy - dABy*dBCx) < 0
if dABx*(py-y1) >= dABy*(px-x1) then exitfunction 0
if dBCx*(py-y2) >= dBCy*(px-x2) then exitfunction 0
if (x1-x3)*(py-y3) >= (y1-y3)*(px-x3) then exitfunction 0
rem verts are ccw
else
if dABx*(py-y1) < dABy*(px-x1) then exitfunction 0
if dBCx*(py-y2) < dBCy*(px-x2) then exitfunction 0
if (x1-x3)*(py-y3) < (y1-y3)*(px-x3) then exitfunction 0
endif
rem point is inside triangle
endfunction 1
function pointInTriangle3(px, py, ax, ay, bx, by, cx, cy)
v = Bx - Ax
q = By - Ay
r = Cx - Bx
t = Cy - By
rem if vertices are clockwise
if (v*t - q*r) < 0
if v*(Py-Ay) >= q*(Px-Ax) then exitfunction 0
if r*(Py-By) >= t*(Px-Bx) then exitfunction 0
if (Ax-Cx)*(Py-Cy) >= (Ay-Cy)*(Px-Cx) then exitfunction 0
rem vertices are counter-clockwise
else
if v*(Py-Ay) < q*(Px-Ax) then exitfunction 0
if r*(Py-By) < t*(Px-Bx) then exitfunction 0
if (Ax-Cx)*(Py-Cy) < (Ay-Cy)*(Px-Cx) then exitfunction 0
endif
endfunction 1
EDIT:
It turns out it did have another problem when something was exactly on the corner. I added code to compare the point against the three vertexes though, and now it works fine. Such an annoying function though, heh.
