Hi,

To display a number rounded down to a user selected number of decimal places, I wrote some code that gave a surprising answer.

Attempt 1

local x = 0.20629048
local dp = 4
local x_round = round(`x', 10^-`dp')
display "this is x `x' this is x_round `x_round'"

The final line results in the following unexpected output - this is x .20629048 this is x_round .2063000000000001


From the following you would expect x_round above to be calculated as .2063

local x = 0.20629048
local dp = 4
display 10^-`dp'
display round(`x', .0001)


However, I can get around the problem with creating a new macro that evaluates the number of decimal places

Attempt 2

local x = 0.20629048
local dp = 4
local dp_value = 10^-`dp'
local x_round = round(`x', `dp_value')
display "this is x `x' this is x_round `x_round'"

The final line results in the following correct output - this is x .20629048 this is x_round .2063

Why does Attempt 1 fail?

Thanks for taking the time to consider this,

Don Vicendese