Columns++ calculations

M Andre Z Eckenrode

I have several issues with Columns++ calculations. When adding the following:

0313127385110486788
2360843530033342354

…Columns++ presents the sum as:

2.6739709151438295e+18

According to several available calculator apps, including the built-in Windows one, the actual answer (as an integer) is:

2673970915143829142

…so I’m trying to understand how the last three digits 142 get rounded to 5?

My other issue is that I’d actually LIKE to have the answer as an integer, or as a decimal number if that’s what it calculates to, not scientific notation. I don’t see anything in the help or options regarding getting calculated values as scientific notation or not, though casual experimentation seems to indicate that scientific notation only happens with values whose integer value is longer than some long number of digits. Is there any way to control or change that? If not currently, I’d like to request it. In the meantime, maybe the help file should provide some clarification on when scientific notation is or is not used.

Coises

@M-Andre-Z-Eckenrode said in Columns++ calculations:

…so I’m trying to understand how the last three digits 142 get rounded to 5?

There is some explanation in the help, though I can imagine it’s easily missed; from the Formulas section:

Numeric values are represented internally as double precision floating point numbers. Any number up to 9,007,199,254,740,992 without a fraction or decimal, positive or negative, is represented exactly. Most fractions and decimals cannot be represented exactly, but in ordinary use, rounding to a reasonable number of decimal places (so that the total number of digits before and after the decimal is under 15) will make discrepancies irrelevant.

Columns++ simply does not do integer arithmetic accurately with numbers larger than 9,007,199,254,740,992 (what can be represented in a double precision floating point). I’m sorry to say that I really don’t see any way to change that without immensely complicating the plugin, since it must also deal with non-integers. I can only suggest that some other software will be required to do accurate integer arithmetic with numbers that large.

My other issue is that I’d actually LIKE to have the answer as an integer, or as a decimal number if that’s what it calculates to, not scientific notation. I don’t see anything in the help or options regarding getting calculated values as scientific notation or not, though casual experimentation seems to indicate that scientific notation only happens with values whose integer value is longer than some long number of digits. Is there any way to control or change that? If not currently, I’d like to request it. In the meantime, maybe the help file should provide some clarification on when scientific notation is or is not used.

That same number (9,007,199,254,740,992) is the limit above which scientific notation is used, since past that point it is certain that integer values are not precise (unless by chance). You are correct, I did not make that clear in the help, and I will add a note to that effect the next time I update the help.

I’m sorry I don’t have an answer you would be happier to hear.

M Andre Z Eckenrode

@Coises said in Columns++ calculations:

Columns++ simply does not do integer arithmetic accurately with numbers larger than 9,007,199,254,740,992

That same number (9,007,199,254,740,992) is the limit above which scientific notation is used

Ah, well. Thanks for the explanation, anyway.

Mark Olson

Anything you can do with Columns++ can (in principle) be done with PythonScript, and Python allows arbitrarily large integers.