division error - GameCreators Forum

Author

Message

Phaelax

DBPro Master

21

Years of Service

User Offline

Joined: 16th Apr 2003

Location: Metropia

Posted: 12th Jun 2010 00:46

Link

I've checked my math multiple times, and the final answer I should be getting is 187, however, after the division in the equations below, I'm getting over 4 billion, greater than the original number I divided into. My comments on each line show what should be the answer, except 'a' is not coming out as expected

+ Code Snippet


base as dword
a as dword
temp as dword

sync on

do
	cls

	set cursor 0,0

	base = 0xBB7AB3FD 		: `187,122,179,253
	r = rgbr(base)
	g = rgbg(base)
	b = rgbb(base)
	
	a = r*65536 + g*256 + b : ` =     8,041,469
	a = base - a 			: ` = 3,137,339,392
	temp = 16777216	: ` =    16,777,216
	
	a = a / temp			: ` =           187
	
	print a				: ` ??? = 4,294,967,227

	
sync
loop

"Any sufficiently advanced technology is indistinguishable from magic" ~ Arthur C. Clarke

Back to top

Profile PM Email Website

Kevin Picone

21

Years of Service

User Offline

Joined: 27th Aug 2002

Location: Australia

Posted: 12th Jun 2010 05:21

Link

Looks like it's using signed division. ANDing A with $ff will remove the top bits. But if you just want to grab A from an ARGB , then shifting is it down by 24 bits a lot less hassle. A= MyColour >> 24