Sorry your browser is not supported!

You are using an outdated browser that does not support modern web technologies, in order to use this site please update to a new browser.

Browsers supported include Chrome, FireFox, Safari, Opera, Internet Explorer 10+ or Microsoft Edge.

Newcomers DBPro Corner / DBP gives too many decimal points which are incorrect?

Author
Message
Dr UnCognizant
15
Years of Service
User Offline
Joined: 10th Aug 2009
Location:
Posted: 6th Sep 2009 11:37
Link
simple problem here, if using this code



but when x gets to 3 the program dosent wait?. Surley there should only be 1 decimal point?. there loads of decimal points. Please help
Back to top
Profile PM
BatVink
Moderator
21
Years of Service
User Offline
Joined: 4th Apr 2003
Location: Gods own County, UK
Posted: 6th Sep 2009 13:11
Link
unfortunately, x will never be 3.0. The closest it gets is 2.99999928474. This is because floating point numbers are not always (in fact, more often than not) exactly what you set them to be. Look up floating point numbers on Wiki if you want the details.

replace your

print x

with

print str$(x)

and you'll see the real, non-rounded value.

The workaround to your issue would be



Back to top
Profile PM Email
Back to top

Login to post a reply

Server time is: 2024-11-27 19:14:19
Your offset time is: 2024-11-27 19:14:19