rounding

[mpepler]

some discussion in our lab recently regarding rounding of HPLC peak areas. Software used gives peak areas to 4 decimal places, and one group rounds these to 2 or 0 dps before calculations (averaging etc) another group uses all 4dps. I would have thought if sample or std wt was recorded to x sig figures eg 50.01mg = 4 sig figures then, even if PA was say 12345789.3333 only first 4 or 5 figures would be significant, thus PA = 1.2346 x 10 exp 7. What do others think?

[David Blais]

I don't believe there is any regulation requiring a certain methodology to be followed. My group uses PA reported to whole numbers only (so 12345789 based on your example). The only time we extend into decimal places is when the software averages the PAs of the standards, and then it's only to 1 dp. Ultimately, you need to follow what is in your internal SOPs. If nothing is specifically stated, then it sounds like you need to get your different groups to agree on one set procedure.

[Mark Tracy]

As a general rule, you should carry all the digits through to the final calculation and do the rounding as the last step. This prevents the accumulation of truncation errors. You may want to report areas as rounded, but you should use the unrounded values for computation. Any sane spreadsheet program or chromatography data system works this way internally.

Try the following experiment with Excel:
A1 = 1.234546 ; B1 = A1*A1
A2 = A1 ; B2 = A2*A2 ; but format A2 to 2 decimals
A3 = round(A1,2) ; B3 = A3*A3 ; see what happens when you truncate an intermediate value?

[mpepler]

Thank you both for your replies. We do have SOPs that indicate internal prcedure and, no there are no regulations that I know of either. I was more interested from a scientific point of view ie how many figures of a peak area value are actually signficant. To take Marks example, B3 would actually be the correct result if the original values were only 3 significant figures. Suppose your software gave you PAs to 10 DPs -would you still use them all?
(Personally I favour whole numbers)

[Mark Tracy]

A pretty good rule of thumb for significant figures is to express the absolute standard deviation for the measurement, in this case peak area, to one significant figure and align it to the measurement. The last significant figure is the first figure of the standard deviation. Nobody except a statistics pedant would complain if you reported one more, just don't try to assign any "significance" to the extra digit.

A typical chromatography experiment with one measurement is usually 3 significant figures; with averaging, you can do 4.

You should report the values rounded to the appropriate number of figures. You should compute using more figures. Why? The computational uncertainties accumulate independently of the measurement uncertainties. You can't do anything about the measurements, but you don't need to round early. And don't round the intermediate values, that makes it even worse. For the kind of calculations you are doing, carrying one or two extra figures is sufficient protection against accumulated rounding error. (It is a different story if you are doing numerical integrations, simulations, etc.)

Besides, you should not be doing your calculations by retyping the data into a spreadsheet or a hand calculator in the first place! The risk of blunder exceeds the relatively trivial loss of precision that we have been discussing so far. Insist that your computer people set up your data system to eliminate all manual transcriptions, and then you master that system. Arithmetic is a rather tedious chore, don't you think?

[mpepler]

Thank you again Mark for your reply, the point you make regarding the distinction between computational and measurement uncertainty clarifies the issue for me. Clearly eliminating manual transcriptions is the best solution, not likely to happen anytime soon however. Any existing proprietary data system you recommend?

[unmgvar]

hello mpepler,

in one pharmaceutical company that worked with they did have a SOP regarding rounding rules.

it was a very simple one. according to the value needed to be reported you could round for calculations only to one digit more.

for example your specs require you to report your assay results as XX.X%, then your calculations can be performed with aprecision of XX.XXor more, but never less.
only at the end would you perform a final rounding to your needed specification.

also it is very important not to perform double rounding by mistake.

if you have a number like 99.9949. rounding it to XX.XX correctly will give you 99.99.
if you double round it then first you get 99.995 then you get 100.00

we also add problems of rounded result between different softwares.
we would calculate uniformity of content with excell sheets. the rounding given by the sheet would have about 7 digit precision. that would give a certain average result.
that result would be many times off to the result of average given by the lims software by +- 0.1% because the results of the 10 uniformity of content there would be feeded manually with a precision of XX.X.

hope this helps also

[bert]

I agree with Mark, to compute with non-rounded figures. The rounding of the final results depends on the calculated standard deviation:

Ther is an ISO paper, dealing with rounding

http://www.iso.org/iso/en/prods-service ... ASUREMENTS

you can also look at this site:

http://ts.nist.gov/ts/htdocs/230/235/ir ... r_2003.pdf

Regards Bert

[Mark Tracy]

mpepler,

Since you ask, I recommend Chromeleon by Dionex. Of course, it is my job to do so. One nice feature is that it has its own embedded spreadsheet based on an older version of Excel. It is compatible with a wide variety of instruments. It is a bit complex, but no more so than Agilent or Waters, just complex in a different way. You can get all the propaganda at www.dionex.com

As has been discussed in this forum before, know and understand your needs before investing in any chromatography data system.