round() for floats/doubles is returning incorrect results for some edge cases, like round(cast(1.55 as float), 1) gives 1.6, but it should be 1.5, since the result after casting to float comes to 1.5499999523162842, due to inaccurate representation of floating point numbers in memory. Removing an intermediate explicit cast to float statement for a double value, which is used in subsequent computations, minimises the error introduced due to the incorrect representation.
Reporter: Sagnik Chakraborty / @sgnkc
Assignee: Sagnik Chakraborty / @sgnkc
PRs and other links:
Note: This issue was originally created as ARROW-10234. Please see the migration documentation for further details.
round() for floats/doubles is returning incorrect results for some edge cases, like round(cast(1.55 as float), 1) gives 1.6, but it should be 1.5, since the result after casting to float comes to 1.5499999523162842, due to inaccurate representation of floating point numbers in memory. Removing an intermediate explicit cast to float statement for a double value, which is used in subsequent computations, minimises the error introduced due to the incorrect representation.
Reporter: Sagnik Chakraborty / @sgnkc
Assignee: Sagnik Chakraborty / @sgnkc
PRs and other links:
Note: This issue was originally created as ARROW-10234. Please see the migration documentation for further details.