This document discusses the way DriveWorks performs calculations, explains why sometimes you may see unexpected results, and provides tips on how they can be avoided.
DriveWorks was designed in accordance to the IEEE Standard for Binary Floating-Point Arithmetic (IEEE 754). The standard defines how floating-point numbers are stored and calculated. The IEEE 754 standard is widely used because it allows-floating point numbers to be stored in a reasonable amount of space and calculations can occur relatively quickly.
| Variable Name | Value |
|---|---|
| LargeNumberA | 1.2E+50 |
| LargeNumberB | 1.2E+100 |
| LargeNumberAplusLargeNumberB | 1.2E+100 |
The resulting value of LargeNumberAplusLargeNumberB is 1.2E+100, the same value as LargeNumberB. This is because DriveWorks stores 15 digits of precision. At least 100 digits of precision would be required to calculate the formula above.
| Variable Name | Value |
|---|---|
| SmallNumberA | 0.000123456789012345 |
| SmallNumberB | 1 |
| SmallNumberAplusSmallNumberB | 1.00012345678901 |
The resulting value in cell A1 is 1.00012345678901 instead of 1.000123456789012345. This is because DriveWorks stores 15 digits of precision. At least 19 digits of precision would be required to calculate the formula above.
Many combinations of arithmetic operations on floating-point numbers may produce results that appear to be incorrect by very small amounts.
For example, the equation:
=1*(.5-.4-.1)may be evaluated to the quantity (-2.78E-17), or -0.0000000000000000278 instead of 0.
This is due to the fact that the IEEE 754 standard requires numbers to be stored in binary format. Not all decimal numbers can be converted exactly to binary, as in the case of 0.1. The conversion caused the loss of precision.
When working with numbers that calculate to more than 15 digits of precision it is necessary to obtain the required precision from the outset.
DriveWorks provides the following functions that will round numbers to a specified number of digits:
| Knowledge Base Article Ref: | KB13103003 |
|---|