Page 1 of 1
Significant figures in a calibration table
Posted: Wed Jan 26, 2022 5:25 pm
by LabProARW
If I had a standard material mass of 0.2055g and a Certificate of Analysis of the material of 99%, should I only have 2 significant figures in the calibration table, i.e. 0.20g?
I have seen several labs use the full assay corrected value of 0.203445g in the calibration table. Should I be using all that are certain 0.20 plus one digit in doubt 0.203 in the calibration table?
I should add that my reported results are two places to the right of the decimal.
Re: Significant figures in a calibration table
Posted: Wed Jan 26, 2022 6:55 pm
by Steve Reimer
An often cited rule is to carry all digits until the end and then round. Your rules (or SOP) may vary.
Re: Significant figures in a calibration table
Posted: Thu Jan 27, 2022 6:56 pm
by LabProARW
An often cited rule is to carry all digits until the end and then round. Your rules (or SOP) may vary.
Do I understand you to mean carry all digits through a calculation until I get to the report results, and then only use the 2 digits to the right of the decimal as in my initial example for the least number of significant figures - unless my SOP rules expect all significant figures plus one which is in doubt?
Re: Significant figures in a calibration table
Posted: Sat Jan 29, 2022 12:56 pm
by sbashkyrtsev
From statistics point of view if you round your intermediate results you increase the uncertainty of the final result. Rounding adds additional error since you're basically adding a random value with some variance to your measurement. And the variance of sum (of the measurement and the random value) is the sum of variances (your measurement variance plus the variance of the random value).
So when doing any type of measurement or calculations the common practice is to never round the intermediate results.
Re: Significant figures in a calibration table
Posted: Sun Jan 30, 2022 8:39 pm
by LabProARW
by sbashkyrtsev ยป Sat Jan 29, 2022 12:56 pm
From statistics point of view if you round your intermediate results you increase the uncertainty of the final result. Rounding adds additional error since you're basically adding a random value with some variance to your measurement. And the variance of sum (of the measurement and the random value) is the sum of variances (your measurement variance plus the variance of the random value).
So when doing any type of measurement or calculations the common practice is to never round the intermediate results.
Thank you for your answer! I do appreciate it.