Calculator versus Empower- different results

[EmpowersBane]

I have to validate a custom field which divides the area of a peak in one injection by the average area of the same peak in the 6 standard injections and multiply by 100. This custom field is set to a precision of 3.Problem is, Empower gives me a result which isn't EXACTLY the same as the calculator albeit only off by a decimal place eg Empower gives a result of 99.76 but calculator 99.79, or 100.85 from Empower and 100.90 for Calculator.

Its probably down to each of the 6 areas of the standard injections being displayed at a precision which is slightly off eg an area of 100000.0976 would display as 100000.098 at a precision of 3 but then where do you stop?? If I went up to 4 it would still round off the 5th number etc. Our SOP only states that custom fields "should work correctly" not very helpful I know.

Is there any logical way to explain to our QA department why the figures, despite not matching exactly, are still sound and robust?

[Peter Apps]

You have ten significant figures in the peak areas - and so the difference in results cannot be anything to do with rounding errors. Are the results from the calculator repeatable ?

Peter

[EmpowersBane]

Thanks for reply. Yes, I have tried it on the calculator several times and I still get the same answer. I think the issue is that Empower will report the CF to a precision of 3 so a CF value of 99.308 might actually be a stored value of 99.30754666 etc from Empower.

I set the area to precision of 3 as well so its these figures that I input into the calculator, do you think the differences in precision is leading to this error?

[HPLC chemist]

Yes. We had in our SOP that you had to have the same digits as the specification, which meant rounding was our last arithmetic function. In addition the numbering had to comply with the USP (general notices, page 4, I think?) and the CDS software was compliant with 21 CFR part 11.

Microsoft Excel and Access alone DO not comply, unless a macro is made.

[Peter Apps]

Thanks for reply. Yes, I have tried it on the calculator several times and I still get the same answer. I think the issue is that Empower will report the CF to a precision of 3 so a CF value of 99.308 might actually be a stored value of 99.30754666 etc from Empower.

I set the area to precision of 3 as well so its these figures that I input into the calculator, do you think the differences in precision is leading to this error?

No, the precision (actually relative resolution) of any number depends on the number of significant figures, not only the figures after the decimal. To refer to the number of decimal places as "precision" is sloppy terminology, to put it politely. You cannot account for the discrepancy between calculator and Empower by rounding errors or "precision" in the sense that you are using it.

Peter

[DR]

Just because you tell Empower to use a specific precision does not mean it does not include the full length number in its calculations, same as w/ Excel.

If you *really* want to have Empower use fewer digits, there will be extensive use of CFs that force rounding of numbers at intermediate stages of calculations. Frankly, I'm inclined to simply recognize that when people use a hand calculator, they will also use only the digits they see when plucking intermediate values from various places and that *this* is the source of most errors. I consider it poor practice to try to force Empower to use "pre-rounded" numbers to produce final values that match those coming from a calculator. It is also a TON of extra work.

That said, I'm finding that I am frequently in the same predicament. Someone with a calculator and no inclination to use a PC will come up with a different hundredths or thousandths digit and declare Empower to be wrong.

[EmpowersBane]

Thanks DR.

I understand that Empower will use the full stored result for any and all calculations. The difference as you say is when you have a custom field that involves two custom fields eg Weight Adjust A - Weight Adjust B, you can get two values that appear rounded. A could be 109776.5099 but displayed as 109766.510 when precision is set to 3. Or...you could get a calculator value of 100766 but an Empower value of 100765.509477!

I checked my SOP a little deeper, there does seem to be a small caveat for this case when using a calculator to check CFs. It says that due to some fields being reported as Integer (like Area) there may be a difference of 1 decimal place eg Assay of 99.8 on calculator but 99.9 on Empower, but that any further differences need justification.

Since I report my CFs to its default of precision I have had instances of it being out by one digit.

[lmh]

not really an empower user here (been a few years since I last saw it) but some general observations:
(1) If you want your calculator to give you the same results as empower, you have to start with the same numbers. If empower is doing its job properly, it will keep maximum precision through the whole process and round at the final number it presents to you. Does it also round the areas that it reports? Because if you start your calculator-calculations with rounded areas, but it started with the true area, then of course your value will have lost precision.
(2) Not relevant in this case, but for general information: numerical calculation isn't totally trivial. There are very good ways to destroy precision completely. Your calculator will certainly work to plenty of decimal places - it will have one or two spare that it doesn't display. Excel has 15 by default, I think. Empower will have loads too. But say you subtract the average of 6 replicate injections (1000.1234) from one of the individual measurements (1000.0234) the answer is now -0.1000. Note that you have just lost 4 decimal places of precision by looking at the small difference between two large numbers. Do this a couple of times in a calculation and suddenly those 15 significant figures have dwindled to 5 or less.
(3) Validation: You cannot prove that a calculation is correct by running it on test-values, although this is a good thing to do, to screen for obvious errors. Proof in arithmetic, like any branch of maths, is by being right, rather than producing the right answer X times and therefore, by extrapolation, always.
(4) Rounding in other packages: there are multiple conventions for rounding. Excel and Access use reputable ways to round, but behave differently. Excel uses arithmetic rounding (0.5 is upward), vba and Access use banker's rounding (0.5 moves in different directions depending on whether the integer part is odd or even). Rounding, again, isn't totally trivial, but if your final decision based on an analysis depends on which way you carry out your rounding, you have rounded far too far, and there is something seriously wrong with the methodology!