Our network

floating-point errors

Floating-point errors, explained

November 5, 2009 by David Smith
with 50 views
0

Back in March, we looked at the reasons behind apparent arithmetic "mistakes" that can occur in software programs like R that use floating-point arithmetic, like this:    > 1.0 - 0.9 - 0.1[1] -2.775558e-17Of course, the answer should be zero, but it's not, not exactly anyway. That example (which I wish I'd... [read more]