HP OpenVMS Systems

ask the wizard
Content starts here

Floating Point Handling?

» close window

The Question is:

This may seem simple but when I run this BASIC code,
32767   END
I get the folowing output:
Why is this number 1 less than it should be?  There was no subtraction
 involved.  I came accross this while I was doing a type cast between double
 and long in one of my payment processing programs. I ran this on two ALPHA
 systems so far and the same outpu
t came up for both. For some reason I could not get another number to do the
 same (e.g. 43.50) other than 39.80, any guesses?

The Answer is :

    VAX and Alpha systems, like just about every modern computer, represent
    real numbers in a binary floating-point format.  Floating point refers
    to numbers being represented internally with the radix point adjusted
    so that the number's fraction is always between .05 and 1.  This is
    similar in concept to 'scientific notation' of very large or small
    numbers being represented in a notation such as  "6.02 * 10^23" where
    "10^23" represents an exponent indicating a large power of 10.
    Internally, a floating point value is stored as a combination of three
            - A base-2 fraction of a certain number of digits
            - An exponent (in powers of 2)
            - A 1 bit sign
    Some common floating point formats on VAX and Alpha are as follows
    (note that not all formats are 'native' to both architectures):
            Data Type               Bits    Fraction Bits   Exponent Bits
            ---------------         ----    -------------   -------------
            F Floating              32      24              8
            D Floating              64      56              8
            G Floating              64      53              11
            H Floating              128     112             16
            IEEE S Floating         32      23              9
            IEEE T Floating         64      52              12
    For the F Floating format, the fraction is 24 binary digits (bits),
    and the exponent is 8 bits.  The exponent is the power of 2 which,
    when multiplied by the fraction, gives the value.  In addition, things
    are manipulated so that the fraction's leftmost digit is always 1 -
    this is called "normalization" - and the exponent adjusted
    accordingly.  Since that bit is always 1, there is no need to store
    it, so it is assumed.  So the fraction "f" is always in the range (0.5
    <= f < 1).  Note that the fraction is 24 bits long, but only 23 bits
    are stored.  A sign bit is included as well, so there is 1 sign bit, 8
    exponent and 23 fraction bits actually store in memory for F Floating
    The exponent for F Floating can range from -127 to +127, and is stored
    by adding 128 to the exponent value - this is called "biasing".  A
    stored exponent of zero is reserved - if the sign is positive, then
    the value is zero, regardless of the fraction.  If the sign is
    negative, this is called a "reserved operand", and generates an
    exception if it is used.
    Let's take a simple read-world example - the number 1.  Remembering
    that the fraction is between 0.5 and 1 (but less than 1), we have to
    represent this as a fraction of 0.5 and an exponent of 1 (0.5 times
    2**1). 0.5 can be exactly expressed as a binary fraction, so there's
    no problem with this.  The bits would work out this way:
            Sign:     0 (positive), goes in bit 15
            Exponent: 1, biased with 128 gives 129, bits 14:7
            Fraction: 0.5, or in binary, 0.100000000000000000000000
                      bits 6:0 and 31:16 (23 actual bits plus hidden bit)
    Putting all the bits together we get:
            3              111      00     0
            1              654      76     0
    or in hex:
            0   0   0   0   4   0   8   0
    D Floating format is the same as F Floating except that it has another
    32 fraction bits available (all zero in this case).
    The first thing you can see is that since we only have 24 fraction
    bits, we are limited in the accuracy to which we can store values.  24
    binary fraction digits translates roughly to 6 decimal digits, so if
    we have a value with more than 6 significant decimal digits, it's
    unlikely it can be represented accurately in F Floating.  We'll choose
    the closest representation we can in 24 bits.
    It is important to realize that "nice, clean" decimal fractions such
    as 0.1 and 0.05 don't translate to "nice, clean" binary fractions.  In
    fact, they end up as repeating fractions, where you can keep adding
    bits forever and you'll never get it exactly right.  The binary
    fraction for .05 looks like:
                0.110011001100110011001100110011001100... ad infinitum
            The 24th fraction bit is here|
    And since the next bit is 1, we'll round up, and thus the F Floating
    value will be slightly higher than .05.  How "slightly"?  Well, the F
    Floating value of CCCD3E4C turns out to be in decimal:
    What would we have gotten if we didn't round, and left the 24th bit
    zero?  The hex would be CCCC3E4C and in decimal:
    which is much further away from .05 than the first value.
    Now take this F Floating value and convert it to D Floating.  This is
    done by tacking on 32 extra fraction bits of zero.  But since the
    original F value is only correct to 24 bits, the D value isn't going
    to be any better.  We'll end up with hex 00000000CCCD3E4C which is
    exactly the same decimal value as above.
    If we had started out by converting 0.05 to D Floating, adding 32 bits
    of precision, we'd STILL get a repeating fraction, but the rounding
    error would be much further out.  In hex we'll get:
    Certainly different than the F-converted-to-D value above. This is
    good to at least 16 decimal digits, but again isn't EXACTLY .05 but
    slightly higher.  You could go to H Floating and get a whopping 113
    fraction bits for about 33 decimal digits of accuracy, but you'd STILL
    not have exactly the right answer.
    So when dealing with floating point, remember that you've only got an
    approximation of the value you want.  Sometimes it's exactly right,
    when the fraction can be exactly expressed, but often it isn't,
    especially when dealing with decimal fractions.
    And the other thing to remember is that simply converting a value from
    single-precision to double-precision doesn't magically conjure up
    those fraction bits that got chopped off in the first place.  Choose
    your initial precision wisely, and don't necessarily believe that
    those last decimal digits you print out are meaningful.
    When arithmetic is performed on these approximations of decimal
    values, the error is compounded to propagated to the final result.  So
    tiny differences in conversion can result in much larger errors later
    on.  Obviously, multiplication or division can magnify the differences
    even further.
    Because in many cases an exact decimal number (for example, .05 as
    shown previously) does not accurately convert to a binary number, it
    is important to remember that these numbers are approximate when
    stored in binary floating point format.  This accounts for the common
    advice that financial data and calculations representing dollars and
    cents should not use floating point numbers.
    In case you are tempted to "check" your binary computer's floating
    point arithmetic operations by using your "pocket" calculator, be
    aware that the results rarely agree.  Calculators almost always use
    BCD (Binary Coded Decimal) numeric representation, so their results
    tend to be more nearly "exact".  Unfortunately, BCD calculations tend
    to be quite slow, that's why computers tend to use floating point
    arithmetic natively.  Software is usually used to handle BCD
    operations (though the VAX architecture does describe optional support
    for Decimal-string instructions).
    The OpenVMS Wizard would also encourage you to review together the
    article "What Every Computer Scientist Should Know About Floating-Point
    Arithmetic: by David Goldberg of the Xerox Palo Alto Research Center
    (available on the internet in several places including
    All that nonsense out of the way, this program appears to run just as
    would be expected.  OpenVMS Alpha V7.3-1 (all patches installed), with
    BASIC V1.4-000.  Never, never, never use floating point format for
    financial data, as floating point is, will be, and always has been
    an approximation -- most accountants will prefer integer values for
    monetary data, whether stored in a longword or a quadword.
    $ type x.bas
            DECLARE LONG    TMP.LONG
            PRINT TMP.LONG
    32767   END
    $ basic x
    $ link x
    run x
    And the same on OpenVMS VAX V7.3 (all patches installed), BASIC
    $ basic
    VAX BASIC V3.9-000
            DECLARE LONG    TMP.LONG
            PRINT TMP.LONG
    32767   END
    NONAME    19-JUN-2003 19:04
  You could also resolve the current and undesired (but entirely correct
  and valid) result with the following change to the code:
  Put another way, you cannot represent 39.8 in a floating point value,
  and assigning it to an integer will truncate it.
  You will also want to read the following information in the BASIC
  HELP library:

answer written or last revised on ( 1-JUL-2003 )

» close window