Sorry your browser is not supported!

You are using an outdated browser that does not support modern web technologies, in order to use this site please update to a new browser.

Browsers supported include Chrome, FireFox, Safari, Opera, Internet Explorer 10+ or Microsoft Edge.

DarkBASIC Discussion / What are the largest numbers that variables can hold?

Author
Message
Robert The Robot
17
Years of Service
User Offline
Joined: 8th Jan 2007
Location: Fireball XL5
Posted: 22nd Feb 2008 10:12
According to the help files for memblocks, a float can range from -2147483648 to 2147483648 and a Dword can range from 0 to 2147483648. Are these the limits of integer/float variables within DB itself, or do these only apply to memblocks?

I only ask, because I'm working on an object/light positioning system and I'm getting some very strange things happenning as I hit these limits. When floats hit one million, they change to powers of ten (at really, really large values it changes to 1.#INF). If I key in an intger variable around 99999999, it immediately jumps to 100000000 - and when I enter 2147483648 it actually becomes -2147483648.

I'm wondering what are the maximum safe values I can apply? I'd like to retain 100% positioning accuracy if I can, but I don't mind if it's a little off - after all, I doubt there are going to be many levels that are 2 billion units across!

Did you hear about the two criminals who crashed their getaway car into a cement mixer? Police are looking for two hardened criminals...
jason p sage
16
Years of Service
User Offline
Joined: 10th Jun 2007
Location: Ellington, CT USA
Posted: 22nd Feb 2008 17:11
Floats are not really accurate like you are implying. Value wise - they are decent - but the very way in which they are stored - and how doing claculations on them chages this storage until your final result - is displayed - which can even be modified before its "seen" via a print - means you need to deal with NOT exact values.

Someone yanked a datatype list out of help before - I don't use Classic much - but I think you are somewhat limited with data types - which is usally not really a bummer - 1: The engine uses these 32bit floats for objects etc - DBPro has Double Precision - (Classic might) but I'm pretty sure you can only tell an object a regular float value - maybe not....

you know - you will need to experiment - but you might get by quickly id DOUBLE precision works for ya!

Latch
17
Years of Service
User Offline
Joined: 23rd Jul 2006
Location:
Posted: 22nd Feb 2008 23:21
@Robert
Well, not talking about memblocks (yet) you can figure out the maximum value of an integer. Since DBC is 32 bit, the first 31 bits (bits 0 to 30) all added up is the maximum positive value:



If you add 1 to result, it will turn negative and show you the value of 2^31 (the last bit). That is because the last bit (bit 31) is used to determine the sign. If it's set, the sign is negative. So the range is -2147483648 to 2147483647.

When writing a BYTE, WORD, or DWORD to a memblock, the memblock will not accept any negative values outright, and the positive limit (for a DWORD) is a 4 byte value (32 bit). Remember, DBC won't accept negative values for DWORD so that means the last bit cannot be set which means the maximum value is 2147483647.

Using 2's compliment you can figure out the equivalent negative value in terms of bytes and write the individual bytes to the memblock. You can then read back a negative integer as a BYTE, WORD, or DWORD. If I want to write a negative value as a DWORD, I can input the 4 bytes that make up the negative number and then read it back as a complete DWORD. All I have to do is make sure I set bit 31. The 4 bytes of a DWORD go from the lowest byte value to the highest:


A float value in DBC follows the same basic rules but it is always signed: as a float variable or as a FLOAT data type in a memblock. A float is always 4 bytes so that means it's values should range the same -2147483648 to 2147483647. However, the reason it's called a float value is because it has a "floating" decimal point. JPS mentioned DBPro having double precision - DBC only has single precision (32 bit signed). It seems you can have up to 45 decimal places but the number returned is rounded to 6 digits and displayed in scientific notation. The byte calculation is different than an integer (or DWORD) in a memblock so you can't just use 2's compliment to figure out a series of bytes to post (say if you wanted a 16 bit floating WORD). So, to make things easy on yourself, use a FLOAT for a memblock when you want negative values and/or a decimal. To make this rather long story shorter:

BYTE 0 to 255
WORD 0 to (2^16)-1
DWORD 0 to (2^31)-1
FLOAT -(2^31) to (2^31)-1

Integer -(2^31) to (2^31)-1

Enjoy your day.
Robert The Robot
17
Years of Service
User Offline
Joined: 8th Jan 2007
Location: Fireball XL5
Posted: 23rd Feb 2008 16:02
Thanks Latch, that's very interesting. However, I'm starting to wonder if I've actually been asking the right question. My program took the keyboard input and converted it to a number - when 2147483648 jumped to -2147483648, I thought it was a problem with storing integers.

But even after taking this into account as you suggested, I still get innaccuracies when dealing with integers:



I$, the original number, is different to what is effectively Str$(Val(I$)). Is this just a problem with the Val() and Str$() functions for numbers larger than about 7 digits? And if it is, is there a way round it?

I'm considering trying to code a DLL in Delphi 5 (using it's IntToStr functin, which appears to work perfectly with all strings) but I can't get my head round how functions are created - when I was taught the basics of it they only looked at the procedures...

Did you hear about the two criminals who crashed their getaway car into a cement mixer? Police are looking for two hardened criminals...
Latch
17
Years of Service
User Offline
Joined: 23rd Jul 2006
Location:
Posted: 23rd Feb 2008 17:49
Quote: "when 2147483648 jumped to -2147483648, I thought it was a problem with storing integers."

That has to do with the 31st bit being set to indicate negatve. (just reiterating)

Quote: "But even after taking this into account as you suggested, I still get innaccuracies when dealing with integers"

I think it has to do with VAL(). If I'm not mistaken, VAL() always interprets a numeric value as a float. Even if you put an integer in there. Look at the result of text 50,100,str$(val(Istring$)) in the following example. It outputs a rounded float value.



When you convert the string first with VAL() then assign that to an integer, the inaccuracies of a FLOAT value that Jason P Sage mentioned start to show up. The VAL() function is converting the string to a float, then it's rounding the value, converting it to scientific notation, then assigning that to an integer. By the time it's converted to an integer, the value has been altered multiple times.

Is there any reason you have to store your integers as strings? Why aren't they just integers from the getgo?

Instead of an external DLL, why not write your own string to int function in DBC? Take a string input, get the length of it, go through each position until you hit a non-numeric character. Drop the decimal and anything after it if there is one. Check for a '-' and if it exists, when you are about to output your integer, subtract it from 0.

pseudo code


Enjoy your day.
jason p sage
16
Years of Service
User Offline
Joined: 10th Jun 2007
Location: Ellington, CT USA
Posted: 23rd Feb 2008 17:56
Good Techniques - I've seen similiar used in mission critical financial apps on legacy systems where they insisted on STRING STORAGE of numbers.

Creating the number a digit at a time (to get your integer) and then doing the oppoiste to get it back to a string your self (I call this man hnalding cuz you're forcing the desired result - and not leaving it to a function you didn't write)

Good Form! This will by your command "DEMAND" accuracy when and how you need it!

Brilliant Pseudo Latch!

Latch
17
Years of Service
User Offline
Joined: 23rd Jul 2006
Location:
Posted: 23rd Feb 2008 18:43
Quote: "Brilliant Pseudo Latch!"

Wow, thank you!

I do have to admit I got a bit of a shudder when I read:
Quote: "mission critical financial apps on legacy systems "

An old life coming back to haunt...

Enjoy your day.
Robert The Robot
17
Years of Service
User Offline
Joined: 8th Jan 2007
Location: Fireball XL5
Posted: 24th Feb 2008 11:02 Edited at: 24th Feb 2008 11:02
@jason p sage and Latch
Thank you!! The pseudo code looks great, I'll try working it up into a proper function later today, and let you know how I go on. As JPS says,
Quote: "Brilliant Pseudo Latch!"



Quote: "Is there any reason you have to store your integers as strings? "

My program lets you use the keyboard to enter a number as a string of text. When you press enter, it has to convert the number to an integer (or float, depending on the function) and stores it as a number in an array.

I'll also try experimenting with a "string to float" function, do you think something similar would work for converting them?

Did you hear about the two criminals who crashed their getaway car into a cement mixer? Police are looking for two hardened criminals...
jason p sage
16
Years of Service
User Offline
Joined: 10th Jun 2007
Location: Ellington, CT USA
Posted: 24th Feb 2008 14:33
You're Welcome... Latch is great.... Um was thinking reading this that YOU CAN'T be the first person with this as critical to you and a small problem.

I Wonder if VanB, IanM or Benjamin ... Someone like that who has made Plugins doesn't have something just for this!

I THINK it was CattleRustler talking and he said something like - "I've got that in one of my plugins...I Don't Remember" - Not the SAME issue - but the kinda makes ya think he's so many he can't keep track! LOL

I Guess I'm just saying don't forget to Google. Good Luck.

Latch
17
Years of Service
User Offline
Joined: 23rd Jul 2006
Location:
Posted: 24th Feb 2008 17:30
Quote: "I'll also try experimenting with a "string to float" function, do you think something similar would work for converting them?"

Yup. It's a little trickier though. But for a float, i suggest using the VAL() function because you're gonna end up with a DBC float one way or the other (you'll be subject to how DBC handles floats as you build it in the function). The integer situation was different because when you used VAL() it converted the string to a float before converting it to an integer so part of the value changed in the translation.

If you're worried about the accuracy of a float using VAL(), then maybe use an outside DLL. JPS is right, there's been a need for this somewhere before! But make sure it doesn't return a double, or the value will be wrong. Also, there's a bug with using float arrays with a dll call, so avoid that as well. Though, once the float is in DBC it will be subject to how DBC handles floats! You just can't get around it - so in my opinion, VAL() is the best option for converting a string to a float. But I could be way off base.

I do encourage you to try and make your own float function. You may come up with something great and much more accurate than val().

Enjoy your day.
Robert The Robot
17
Years of Service
User Offline
Joined: 8th Jan 2007
Location: Fireball XL5
Posted: 25th Feb 2008 14:02
Quote: ""I'll also try experimenting with a "string to float" function, do you think something similar would work for converting them?"
Yup. It's a little trickier though. "


It was, but I've done it



with number validation:



and for an integer:



and again, with validation (to make sure it is a pure number):


Quote: "DBC only has single precision (32 bit signed). It seems you can have up to 45 decimal places but the number returned is rounded to 6 digits and displayed in scientific notation."


Does that mean that floats - when they exist as floats - are formatted to 6 digits, or are they merely rounded to 6 digits when you convert it to a string for display on screen? And if that is the case, would it be possible to access the individual bits making up the float? I can read off each byte of the float by writing it to a memblock and reading the integer component, but I'd need to know what each bit actually signified.

Or am I just being picky, and none of this really matters at all ?

Did you hear about the two criminals who crashed their getaway car into a cement mixer? Police are looking for two hardened criminals...
Latch
17
Years of Service
User Offline
Joined: 23rd Jul 2006
Location:
Posted: 25th Feb 2008 17:51
Starting to look pretty good. I think you'll have to manage non numeric characters a little differently if you want these functions to be more general. See what the following returns:

Str_To_Float("k3kgo6gfd5")

And for the float function, make sure it can accept a number that doesn't contain a decimal. Look at the result of
Str_To_Float("16")

Also for both, make sure you manage the maximum and minimum value. See what the following returns for your integer function:

str_to_int("5214743647")

And for the integer function, if there is a decimal point, drop it and everything after it. Your integer function behaves a little odd:
str_to_int("14.6")

You almost got it. You just have to put a few checks and balances in the functions.

Quote: "Does that mean that floats - when they exist as floats - are formatted to 6 digits, or are they merely rounded to 6 digits when you convert it to a string for display on screen? And if that is the case, would it be possible to access the individual bits making up the float? I can read off each byte of the float by writing it to a memblock and reading the integer component, but I'd need to know what each bit actually signified.
"


A couple of things. You can easily view which bits are set using the BIN$() command. So let's kill 2 birds with one stone. We'll look at how a float is rounding and we'll also look at the bits so we can see what is being set:



Even without a decimal, the float rounds up because it converts the number to 6 digits in scientific notation. You can see the bits set are different because float# ends up 1000000000.

Quote: "would it be possible to access the individual bits making up the float?"


If you mean set the individual bits on or off, yes. You can use DBC's binary & to test/set a bit and you can use | to set a bit(s). ! is the not operator and is useful in this also.

It's easy for an integer and a little tougher for a float because DBC won't let you use | & on floats directly. For a float you'd have to check each byte from a memblock, set the bit for the specific byte then write the byte back to the memblock, then retrieve the float out of the memblock.

Here's an integer example:


Enjoy your day.
jason p sage
16
Years of Service
User Offline
Joined: 10th Jun 2007
Location: Ellington, CT USA
Posted: 25th Feb 2008 19:28
Quote: "And for the integer function, if there is a decimal point, drop it and everything after it. Your integer function behaves a little odd:
str_to_int("14.6")"


(Unless you want to add in rounding to it!
0.1 = 0
0.5 = 1

Either way is fine

Robert The Robot
17
Years of Service
User Offline
Joined: 8th Jan 2007
Location: Fireball XL5
Posted: 27th Feb 2008 09:40 Edited at: 28th Feb 2008 21:41
Oh dear. I see what you mean with the last 3 problems on my functions, but I disagree with the first - I created two functions for each conversion, one which just converts the string and another that performs a validation check on the string before making it into a float. I think you must have been using the one which doesn't check for non-number characters. (No matter, it was probably a mistake me making two functions for the same task anyway )

Anyway, here are some revised versions. First, here's the one for floats:



And here's one for integers, complete with validation and rounding up of decimal places >=5 (thanks for the idea JPS!)




Quote: "You can easily view which bits are set using the BIN$() command"
When I run your code snippet, I get a % symbol at the start of each string of binary - is that just there to let you know that it's a binary string, or does it have a meaning? I'm also struggling a bit understanding how the binary operators work. Is it that Variable | 2^Bit_Being_Tested turns on the bit responsible for that value (i.e. 2^6 = 64 so turns on the bit for 64)? Will the ! operator then mean that Variable ! 2^Bit_Being_Tested becomes the inverse of what the bit currently is, and does the & symbol just read the value of Variable & 2^Bit_Being_Tested without affecting it?

Also, I found this on Wikipedia:

http://en.wikipedia.org/wiki/IEEE_754-1985

and while it tries to explain how single precision 32-bit floats are encoded, its waaaaay over my head. I can read binary integers, but I don't understand how the exponent biasing works or how the fraction part ofthe binary string is read. I'm really starting to think you were right, and that I should just stick with Val()... but I'm still curious as to how all this works!

Did you hear about the two criminals who crashed their getaway car into a cement mixer? Police are looking for two hardened criminals...
Latch
17
Years of Service
User Offline
Joined: 23rd Jul 2006
Location:
Posted: 27th Feb 2008 16:18
Quote: "I get a % symbol at the start of each string of binary - is that just there to let you know that it's a binary string"

In DBC, to let you know it's a binary string.

Quote: "Is it that Variable | 2^Bit_Being_Tested turns on the bit responsible for that value (i.e. 2^6 = 64 so turns on the bit for 64)? "

Yes. This is an OR operator. If any of the bits of two numbers that are ORed are on, then the result of those two numbers will have that(those) bits truned on. So the binary values:

10101 , 11000 ORed together using the | operator will result in

11101

Don't confuse it with addition because you'll notice bit 2 is 1 in both values. If it was addition, the number would be larger because of carry. In decimal the | values look like 21 | 24 = 29 while addition would be 21 + 24 = 45 a very different number.

Quote: "Will the ! operator then mean that Variable ! 2^Bit_Being_Tested becomes the inverse of what the bit currently is"

the ! operator will compare 2 values and return 1 if the values don't match and 0 if the values do match. The & operator will compare the bits of a number and the result will only be those bits that are ON in BOTH number being compared.
This blurb from my example:


tests if num & value have the same bits turned on. In this case, value = 2^bit_being_tested so if num has that bit turned on, the result will be value because all of the other bits in value are off.

value is then NOTed against the result of num*value. If they match the result is 0 (because of the !). If they don't match, the result is 1. I then subtract that result from 1. So, if Temp=1 then I know the bit I'm testing is on. I then subtract value*temp from num which effectively turns off whatever bit I was testing if it is on. If it wasn't on, then temp=0 so temp*value=0.

For the float bit stuff - you're on your own!

Enjoy your day.
Robert The Robot
17
Years of Service
User Offline
Joined: 8th Jan 2007
Location: Fireball XL5
Posted: 28th Feb 2008 21:53
Fascinating! I'll bear all that in mind, it may come in useful sometime!

Quote: "For the float bit stuff - you're on your own!"

Well, maybe val() isn't too bad for floats...

I'll have a bit more of a google round, and see what I come up with, but for now I think I'll just stick with the way DB normally handles floats.

Oh, by the way - I just edited the integer routine on my last post. I got some funny results out of it when I called the function in a Do : Sync : Loop setup - it turned out I wasn't initialising the variables. Strange, really, I thought that all function variables were created from scratch each time you called the function, but apparently not. The still seem to linger in the system and affect the function next time it's called.

Did you hear about the escapologist's funeral? He was buried on June 25th, 26th, 27th, 28th...
TDK
Retired Moderator
21
Years of Service
User Offline
Joined: 19th Nov 2002
Location: UK
Posted: 28th Feb 2008 23:19
Still on the subject of binary, something which DBC only supports as an afterthought with Bin$() which converts a decimal to binary in the form of a string, there's no way to convert it back to a decimal.

So, here's a function which takes the binary string after you've messed with it and returns a decimal:



And an example of using it:



TDK_Man

Login to post a reply

Server time is: 2024-03-29 15:51:05
Your offset time is: 2024-03-29 15:51:05