I have a few scientists for clients and they have some problems with how toPrecision is rounding in JavaScript. They want everything rounded to a MAX of 3 sig figs which works most of the time but let me give a few examples of what they want:

Lab value to rounded value

123.5 to 124

1234 to 1230

12.0 to 12.0

0.003 to 0.003

So in other words round things with more than 3 sig figs down to 3. If something has 1 or 2 sig figs DONT append a zero (as that implies the lab was more accurate then they really were) but also in the case of 12.0 DONT remove the zero (as that implies the lab is less accurate then they really were)

I have a few scientists for clients and they have some problems with how toPrecision is rounding in JavaScript. They want everything rounded to a MAX of 3 sig figs which works most of the time but let me give a few examples of what they want:

Lab value to rounded value

123.5 to 124

1234 to 1230

12.0 to 12.0

0.003 to 0.003

So in other words round things with more than 3 sig figs down to 3. If something has 1 or 2 sig figs DONT append a zero (as that implies the lab was more accurate then they really were) but also in the case of 12.0 DONT remove the zero (as that implies the lab is less accurate then they really were)

• javascript

• So, you want to round values with three or more significant figures down to three, ignoring any other values. – Jack Bashford Commented Jun 27, 2019 at 1:46
• @JackBashford yes exactly! Round numbers with more than three sig figs down to three, and don’t touch those with three or less (24.0 being three sig figs for those, like me, rusty with high school science) – ikenread Commented Jun 27, 2019 at 1:51

2 Answers 2

Using toPrecision works for all examples given except for the 12.0 to 12.0 example:

var nums = [123.5, 1234, 12.0, 0.003]
var out = nums.map(num => parseFloat(num.toPrecision(3)))
// => [124, 1230, 12, 0.003]

It rounds numbers with more than 3 sig figs to 3, but if you use a number with a .0 or .00 on the end it fails. The reason for this is that the JavaScript engine equates 1.00 to 1, and 12.0 to 12, so the problem is actually not toPrecision, but rather JavaScript itself.

To work around this, what you can do is input numbers as strings, and use toPrecision if there isn't a decimal zero, otherwise operate on the string itself:

var nums = ['123.5', '1234', '12.0', '0.003', '1.000', '1236.00'];
var out = nums.map(str => {
    if (/\.0+$/.test(str)) { // test if it ends with .0 or .00, etc.
        // use alternative string method:
        var zeros = str.match(/[0]+$/)[0].length; // count the no. of trailing zeros
        var sigfigs = parseFloat(str).toString().length; // no. of other sig figs
        var zerosNeeded = 3 - sigfigs;
        if (zerosNeeded < 0) {
            return parseFloat(parseFloat(str).toPrecision(3)).toFixed();
        } else {
            return str.substring(0, sigfigs + 1 + zerosNeeded); // +1 for the decimal point
        }
    } else {console.log(str)
        return parseFloat(parseFloat(str).toPrecision(3)).toString()
    }

});
// => ["124", "1230", "12.0", "0.003", "1.00", "1240"]

This works, however as the result must be in a string format if you need to work with floating point numbers and similar, I'd remend using a different language such as Python. Anyway, I hope this helps!

All you need to do is to write a custom parser.

See this example:

const data = [123.5, 1234, 12.0, 0.003, 100.0, 1.0];

data.forEach(n => {
  const result = customToPrecision(n, 3);
  console.log(`${n} -> ${result}`);
});

function customToPrecision(number, precision){
  let result = number.toPrecision(precision);

  // Check if original number is a float
  if (number % 1 !== 0){
    result = result
      .replace(/0+$/, '')     // Remove trailing zeros
      .replace(/\.$/, '.0');  // Add one zero to inplete decimal
  }

  return result;
}