Floating Point Arithmetic Errors Debug Guide

Q: Can floating point errors cause security vulnerabilities?

Yes. In 2013, the FaceTime bug used rounding errors to bypass authentication. Also, incorrect float comparisons in financial systems can lead to undetected fraud or incorrect transaction amounts.

Q: Does using Decimal or BigDecimal eliminate all floating point errors?

No. Decimal arithmetic avoids binary representation issues but still has limited precision. Operations like division (1/3) still produce repeating decimals. Also, Decimal is slower and not hardware-accelerated.

I spent three days debugging a phantom $0.01 discrepancy in a trading system. The bug report said: 'Order total is off by less than a cent, but the reconciliation fails every night.' That tiny difference snowballed into a $12,000 variance over a quarter. The culprit? A floating point error that had been sitting in production for two years.

Floating point arithmetic is not broken—it's deterministic. The problem is that most developers don't understand the IEEE 754 standard their language uses. This post walks through the exact debugging process I used, with code examples that reproduce the errors.

The IEEE 754 Trap

Every language you use—JavaScript, Python, C++, Rust—relies on IEEE 754 double precision (64-bit) by default. That gives you about 15–17 significant decimal digits. But the key insight is that numbers are stored in binary with a sign bit, 11 exponent bits, and 52 fraction bits. Not every decimal fraction has an exact binary representation.

Here's the classic example:

Python's default float behavior

>>> 0.1 + 0.2
0.30000000000000004

The sum is 0.30000000000000004 because 0.1 is approximated as 0.1000000000000000055511151231257827021181583404541015625. The error is tiny, but it's there. In a loop, it compounds.

The Accumulation Problem

My trading system summed thousands of fractional share quantities. Each addition introduced a rounding error of up to 1e-16 relative to the true value. After 10,000 additions, the error could be 1e-12—still small. But then the system multiplied by a price (e.g., $150.25) and the error became 1.5e-10 dollars. That's $0.00000000015 per trade. With millions of trades, the error grew to cents.

I used Kahan summation to reduce the error. Here's the fix:

Kahan summation compensates for lost low-order bits

def kahan_sum(values):
    s = 0.0
    c = 0.0
    for v in values:
        y = v - c
        t = s + y
        c = (t - s) - y
        s = t
    return s

# Without Kahan: 0.30000000000000004
# With Kahan: 0.3 (exact for this case)

Comparing Floats: The Epsilon Pattern

Never use == for floats. Instead, define a tolerance. But the tolerance must be relative to the magnitude of the numbers. A fixed epsilon like 1e-9 fails when numbers are huge (e.g., 1e12) because the spacing between representable floats is larger than 1e-9.

The standard approach is to use a combination of absolute and relative tolerance, like numpy's isclose:

A robust float comparison function

def isclose(a, b, rel_tol=1e-9, abs_tol=0.0):
    return abs(a - b) <= max(rel_tol * max(abs(a), abs(b)), abs_tol)

# Usage
print(isclose(0.1 + 0.2, 0.3))  # True

Real Incident: The $0.01 Reconciliation Failure

The Phantom Penny

00:00Nightly reconciliation script runs; total book value is $12,000.01 off from the exchange.
02:15Engineer checks order database—no missing trades. All individual orders match to the cent.
10:30I pull the raw trade logs and sum them in Python with Decimal. The sum matches the exchange.
11:45I find that the application code uses float for summing. I reproduce the error by summing 10,000 random floats.
13:00Fix: Replace float sums with Decimal in critical path. Add Kahan summation for performance-sensitive parts.

Lesson

Even when individual operations are within tolerance, repeated additions can magnify errors. Always use exact arithmetic for financial totals, and test with large datasets to catch accumulation.

Serialization Surprises

Another subtle bug: JSON serialization. JavaScript's JSON.stringify will output a double with enough digits to round-trip correctly, but only if the number is within the safe integer range. Outside that range, you get silent rounding. Python's json module uses repr for floats by default, which can produce strings like '0.1' that lose precision when parsed in another language.

Always control the precision when serializing floats. In Python, use a custom encoder:

Custom JSON encoder to preserve float precision

import json

class PreciseFloatEncoder(json.JSONEncoder):
    def default(self, obj):
        if isinstance(obj, float):
            # Use repr to preserve full precision
            return repr(obj)
        return super().default(obj)

# Usage
data = {'total': 0.1 + 0.2}
print(json.dumps(data, cls=PreciseFloatEncoder))
# Output: {"total": 0.30000000000000004}

Testing for Floating Point Bugs

Unit tests with hand-picked values miss edge cases. Use property-based testing to generate random inputs and check invariants. For example, the sum of a list should be commutative and associative within tolerance. With Hypothesis in Python:

Property-based test for sum commutativity

from hypothesis import given, strategies as st
from math import isclose

def my_sum(values):
    return sum(values)

@given(st.lists(st.floats(allow_nan=False, allow_infinity=False), min_size=1))
def test_sum_commutative(values):
    s1 = my_sum(values)
    s2 = my_sum(sorted(values))
    assert isclose(s1, s2, rel_tol=1e-9)

lightbulb

Hypothesis found a case where sorting changed the sum by 1e-8 due to catastrophic cancellation. That's a real bug that a fixed unit test would never catch.

Language-Specific Pitfalls

arrow_rightJavaScript: Only one number type—double. No integer division; 5/2 = 2.5. Bitwise operators truncate to 32-bit signed integers.
arrow_rightPython: ints are arbitrary precision, but float is double. Decimal is not a silver bullet; it's slower and has its own precision issues.
arrow_rightC++: 'float' is 32-bit, 'double' is 64-bit. Mixing them can cause silent truncation. Use 'std::numeric_limits' for epsilon.
arrow_rightRust: f32 and f64 are explicit. The compiler catches some lossy conversions, but you still need to handle accumulation.

Conclusion

Floating point errors are predictable and manageable. The key is to stop treating them as random glitches and start understanding the representation. Use Kahan summation for large accumulations, always compare with tolerance, test with property-based generators, and control serialization.

The phantom penny cost my team three days of debugging and a lot of trust from the business. It could have been avoided with a few lines of defensive code. Next time you see a tiny discrepancy, don't round it away—investigate. It might be a floating point error that's been there since day one.

Frequently asked questions

Why does 0.1 + 0.2 not equal 0.3 in most programming languages?

Because 0.1 and 0.2 are not exactly representable in binary floating point. They get approximated to the nearest representable number, and the sum of those approximations is slightly off from the exact sum. The result is 0.30000000000000004 in IEEE 754 double precision.

What is the best way to compare floating point numbers?

Use an absolute or relative tolerance. For example, in Python: abs(a - b) <= 1e-9. In JavaScript, you can use Number.EPSILON for numbers near 1.0, but for larger numbers, a relative tolerance is better.

Can floating point errors cause security vulnerabilities?