Jump to content

Script 9 digit int to float bug


dylbill

Recommended Posts

I'm encountering a bug where, if I try to cast an integer that's more than 9 digits as a float, the float returns a different value than the integer.

 

Example:

 

Int TestInt = 123456789

Float TestFloat = TestInt as Float

 

The float will return as 123456792.0000

 

Has anyone else encountered this problem and found a solution?

 

I've tried a number of things already.

 

I've tried casting the int as a string first, then the string as a float.

I've tried setting a global variable value to the int

I've tried doing Float TestFloat = TestInt * 1

 

The result is always the same. Any help would be greatly appreciated!

 

Link to comment
Share on other sites

Float values can't hold precise integer values that large. They get rounded off as you've seen. It's part of the nature of floats and ints. The price you pay for being able to store larger values and fractional values in the float type is that you get less precision than an int.

Link to comment
Share on other sites

Thanks for the info. Is there unsigned floats in Skyrim? I haven't heard of that. Specifically what I'm trying to do is after using the function GetFormID(), converting the form ID from decimal to hex. The function I wrote is working, but the problem is the float isn't the same as the int ID. As a work around, I'm now casting the int ID as a string, and puting a decimal place in the middle, so that the value is smaller, then doing the same thing and moving the decimal place over again.

Link to comment
Share on other sites

  • Recently Browsing   0 members

    • No registered users viewing this page.
×
×
  • Create New...