Snedecor's F test for linearity

[Kwet]

Does anyone realise a Snedecor's F test to validate the calibration curve of a (liquid) chromatographic method?

What do you think about this statistical test?
What are the advantages – disadvantages of the test?
Do you think it's a powerful, interesting, practical tool or not?

I often see calibration curves passed the more classical criterias of validation.
For example: Standards are prepared in triplicate (3 dilution series) for 5 levels + the solvent in triplicate (so 6 levels). Each solution (18) injected one time.
R² > 0,999, trueness (residue) of each point between 98 – 102 % (calculated value / "true" value x 100) and RSD of each point < 2%
So that a lot of analysts will consider it as "validated".
Even thought the same data will failed the Snedecor's F test!
Statisticians usually said that it's because variability inside a level (RSD of the 3 replicates) is smaller than the variability between response factors of the 5 levels. I am not an expert in statistics so I believe them. Do you think it's the right explanation?
Does anyone have already met this problem?
What could be the solution to avoid this problem in the method validation ?

Thanks in advance for your comments.

[Noser222]

Are you familiar with weighted least squares?

[Kwet]

I effectively know Weighted Least Squares.
Thanks to Tom Jupille for his explaination on http://www.lcresources.com/resources/ex ... ghting.pdf

I totally agree that wheighed calibration curves can solve problems when points in the lower of the curve don't fit with a linear regression.
But let’s see below an example of results I obtained:

Conc. Area Calculated Trueness R.S.D.
Conc.

0,0000 0 0,7991 #DIV/0!
0,0000 0 0,7991
0,0000 0 0,7991
60,21 2280056 60,1946 100,0% 0,272
60,21 2292267 60,5127 100,5%
60,21 2283432 60,2826 100,1%
120,4 4561743 119,6326 99,4% 0,093
120,4 4569891 119,8449 99,5%
120,4 4567964 119,7947 99,5%
180,6 6886100 180,1822 99,8% 0,182
180,6 6861530 179,5421 99,4%
180,6 6869434 179,7480 99,5%
240,8 9156725 239,3320 99,4% 0,180
240,8 9189261 240,1796 99,7%
240,8 9178662 239,9035 99,6%
301,0 11537569 301,3531 100,1% 0,338
301,0 11599714 302,9719 100,6%
301,0 11610322 303,2483 100,7%


F observed is 15.9 so higher than the theoretical F of 3.26 (probability 0.95, k1 = 4 and k2= 12). So the linearity test failed!

But take a look at the results, which analyst can honestly say that there is a problem with this linear fitting?

I already know that is not correct to put 0 as area for the solvent and take it into account for the curve calculation and in the F test. But I must follow my SOP so I do it. If I remove all data for the level at 0, the F test become worse so it’s not the problem.

I think it comes from variances that are too small and not equal for the 5 levels (equal variance test failed) so in theory the test cannot apply but I red somewhere that the test is robust again difference in variance....so what to do with such result?

[Kwet]

Oh I'm sorry the data are not well presented, I hope you will understand:
first row = theoretical concentration
2sd row = area
3rd row = concentration calulated with the regression equation
4th row = calculated concentration / theoretical concentration x 100
5th row = RSD

I forgot to give regression equation and r²:
y = 394539x - 989,3
r² = 0,99997
I tryed to add the graphical vue of the curve but failed!
I hope someone can explain me the problem with these statistical test!

[tom jupille]

Since that earlier posts, I have learned quite a bit about the topic from an excellent series of articles in American Laboratory magazine. The direct url is several lines long, but here's a shorter link:

http://tinyurl.com/4g5z4z

The examples used are all chromatography-based. I'm still going through the articles (slowly and in detail) to learn the material more thoroughly.

[zokitano]

Since that earlier posts, I have learned quite a bit about the topic from an excellent series of articles in American Laboratory magazine. The direct url is several lines long, but here's a shorter link:

http://tinyurl.com/4g5z4z

The examples used are all chromatography-based. I'm still going through the articles (slowly and in detail) to learn the material more thoroughly.

Thank you Tom for the provided link. It's very helpful!

Regard

[Noser222]

This article helped me:

http://www.lcgceurope.com/lcgceurope/da ... rticle.pdf

[aceto_81]

In fact your Snedecor's test looks for systematic deviations in your area.
So if you look at your curve (I dont used your 0 points), you will see that the areas of conc 60.21 will be all above the calibration line, the next 9 all below, and the last 3 above the calibration line.
This indicates that the deviance of the calibration line is not due to variance (as all the 3 points are above, or below, and not randomly above and below distributed).

Did you make 3 injections of each concentration, or did you make 3 different preparations?
If you only make 3 preps, you incorporate the error of your sample prep in your test, and average it out. In case of 3 injections, you average your injection error, but not your preparation error, and your F value will be higher (or too high).

Ace

[Noser222]

I think you also have to ask yourself what is gained by a higher order fit of your data, if that is your intention.
If not perfectly linear, you are still >99% accurate.

[aceto_81]

I think you also have to ask yourself what is gained by a higher order fit of your data, if that is your intention.
If not perfectly linear, you are still >99% accurate.

I agree, that >99% is good enough for most purposes, but when talking about validating linearity, you aren't talking about 99% accuracy, but with finding the right fit with the minimum of terms and proof that this is correct.

With R^2 you can't prove linearity, only with a few "specialised" tests, as there is an F-test, chi-square test, ...
But sometimes your variance is so low that you will see a systematic deviance of your calibration line for some concentration, and then it's up to the analyst to find out if is really a non linear fit, or a systematic error.

Ace

[Kwet]

Tom, Nosser222,
Thanks for the links. It seems to have a lot of information of interest. I see I will need a lot of time to read and understand all these statistics!

Aceto_81,

I make 3 different preparations but it's only a dilution of the analytical standard stock solution. So standards for the calibration curve are not prepared as the real samples.
I agree with a problem of a systematic error. But how explain that? Should it comes from glass pipettes used for dilution: the same pipette is used for the three dilutions of the same level while a different one is use for each level!

My intention is not to move to a higher curve. I just want to understand why the F-test failed while accuracy and repeatability results seem enough to me to validate the linear curve. I'm also wondering about the need and the benefit to realize such a test which is time and energy consuming and brings problems to validate a curve when I think it should be accepted! Is it really enhancing the quality of our work?

[aceto_81]

I make 3 different preparations but it's only a dilution of the analytical standard stock solution. So standards for the calibration curve are not prepared as the real samples.
I agree with a problem of a systematic error. But how explain that? Should it comes from glass pipettes used for dilution: the same pipette is used for the three dilutions of the same level while a different one is use for each level!

Whenever possible, we always weigh all the references instead of diluting a stock solution. I would certainly use different pipettes instead of the same pipette for the 3 dilutions (concentrations also differs so how about carryover/contaminations?). We try to mimic the real sample/reference prep as much as possible, and error also belongs to sample/referenc preparation. So instead of avoiding as much error as possible, we try to get equal error to a normal preparation.

My intention is not to move to a higher curve. I just want to understand why the F-test failed while accuracy and repeatability results seem enough to me to validate the linear curve. I'm also wondering about the need and the benefit to realize such a test which is time and energy consuming and brings problems to validate a curve when I think it should be accepted! Is it really enhancing the quality of our work?

I'm not saying that it enhance the quality of your work, but it's the same as accuracy: if you consistently get values between 99.7 and 99.9%, it's possible that your confidence interval goes from 99.65-99.95%. This raises the question that 100% isn't in your confidence interval, although 99.8% recovery isn't that bad at all. So would you search for a way to find a recovery of 100% or go further with this results?
The same story for your linearity: from an analysts viewpoint it's more than acceptable, but if you look at the statistics it's not.

In both cases the statistics doens't add more quality to your work, but if it's in your SOP, you have to follow this.

Ace