adjust rounding decimals to expected values
This may have been the cause for some tinderboxes choking on this test on Windows and Mac if the forecast results slightly differed due to the initialization with random values and the calculation taking place with a different internal floating point precision. So let's try. Blindly throwing ROUND(...,12) at all values is wrong anyway because that doesn't take the magnitude of the expected value into account. What we actually want is an accuracy of 12 digits, not an accuracy of 12 decimals. 0.123456789012 1.23456789012 12.3456789012 123.456789012 ... Change-Id: If88d75277c3d17cfd72f36d0f88d093b63da8add
Showing
This source diff could not be displayed because it is too large.
You can
view the blob
instead.
Please
register
or
sign in
to comment