Method development and calculating of precision and batch

[dresdentl]

During method development the precision of the method should be determined. Supposed, that according to the validation guideline, the coffecient of variation should not exceed 15% and at least 5 replicates of three different concentration are measured.


If you have determined a very large serie of measurements for each concentration, for instance 100 measurements per concetration, how do you calculate the coffecient of variation for this method?

If you include all 100 measurements for calculation, in may happen that CV is > 15 %. Is it possible to exclude some of the measurements in order to get a better CV, because normaly 5 measurements per concentration are sufficent?


On the other hand, according to my experience, the accuracy of this method is very good,dispite the high CV (for instance 20%), because the values of the measurements are symmetrically ditributed around the average. The batch run conditions are also met, because the included standards differ not much from the theoretical value (rel. error < 6%).

There is a calibration curve which consits of such 50-100 replicates for 8 concentration and you use it for calculation. The results (accuracy) are very reliable although the precicion of some replicates are not so good. Maybe greater deviations only happen sometimes/seldom for an unknown reason.
Would you accept this method as long as the batch run conditions are met? (6 calibration standars, included in each run deviate less than 15% from theoretical value)

[Kwet80]

First step is to measure the repeatability RSD. Repeatability means that replicates are analysed under the same conditions with the minimum variation between the measurements: short period of time (same day), same batch of analyses (5 replicates prepared and analysed at the same time or one just after one), same analyst, same apparatus, same concentration. Each replicate follows the entire procedure from A to Z (from sampling to final measurement). If your procedure required 3 concentrations, make 3 series (one for each concentration) of at least 5 replicates and calcule a RSD for each serie. statistically, calculate a RSD for the 15 measurements has no sense.
This step is maybe enought in early stage of method development.

Second step: intermediate precision. Continue to realize series of 5 replicates (for exemples, series of 3 replicates are allowed) with a small variation between them. Exemple: a second analysist, a second day or a second apparatus. 3 differents series is usually considered as the minimum in a method full validation. During the method development, I don't think there are usual rules. It's just to give you confidence (or not) that the method will function in routine.
Calcul of intermediate precision RSD is more complicated.

calculate the mean and the variance of each serie.
Intermediate precision variance = variance between series + variance within series
variance within series = mean of the variances
variance between series = variance of the means - (mean of the variances / n)
where n is the number of replicates by serie.

[michaelbarnes42]

ok, so one person is talking about %CV the other is talking about %RSD are they not the same thing?

[HPLCaddict]

If you have determined a very large serie of measurements for each concentration, for instance 100 measurements per concetration, how do you calculate the coffecient of variation for this method?
If you include all 100 measurements for calculation, in may happen that CV is > 15 %. Is it possible to exclude some of the measurements in order to get a better CV, because normaly 5 measurements per concentration are sufficent?

Clear answer: NO! You want to drop the data that doesn't suit your needs and only use the "good values"? What do you think an auditor or any regulatory authority would say to this procedure?
BTW, if there are only the usual experimental errors, the CV should rather go down the more measurements you have. How does the distribution of your values look like? is it just very wide or are there single outliers that boost up your CV?
And another BTW, for precision you're supposed to use INDEPENDENT preparations. Did you really weigh/dilute and go through sample preparation for 100 INDEPENDENT samples???

[Kwet80]

CV (coeficient of variation) and RSD (relative standard deviation) are the same = standard deviation / mean *100

[dresdentl]

-It should be possible to exclude outliers. The question is, what method schould be applied to determin outliers?

-There should also be a difference if you calculate %RDS/CV from larger set of measurements in contrast with a smaller set. The larger set should be more convincing than a smaller set of measurements.

[DR]

Acceptance criteria and dealing with outliers are both functions of the compendium and type of method. Eliminating outliers is, at best, tricky business. I tend not to do so without clearly assignable cause(s) for the strange values. Typically, elimination of outliers leaves you short a few injections as specified by your protocol, so you will have to repeat some of it anyway. If there are no clear causes for outliers, take your best guess at the causes of outliers and address them via changes to the protocol and/or method itself. If your guesses are correct, you will have no more outliers.

[dresdentl]

The samples which are measured are generated by SPE. The SPE cardriges which have been used for about 2 years are not completely identical. Then over this time some work technique has changed. This are the reasons for greater deviations and outliers.

I think it would be more reasonable to use a set of more recently generated measurements (for instance the measurements of the last three months). Maybe things change over time.


But despite greater deviations, I think the calibration curve generated from all data is useful.

If I exclude outlier or measurments with strong deviation, I can improve the %CV or %RSD, but the calbration curve itself doesn't cange much. The accuracy is always good.

[HPLCaddict]

The samples which are measured are generated by SPE. The SPE cardriges which have been used for about 2 years are not completely identical. Then over this time some work technique has changed. This are the reasons for greater deviations and outliers.
I think it would be more reasonable to use a set of more recently generated measurements (for instance the measurements of the last three months). Maybe things change over time.
But despite greater deviations, I think the calibration curve generated from all data is useful.
If I exclude outlier or measurments with strong deviation, I can improve the %CV or %RSD, but the calbration curve itself doesn't cange much. The accuracy is always good.

Hmm. Did I understand correctly that
1) you want to use actual batch analysis data generated over the years to assess method precision?
2) you changed sample preparation procedure?

Concerning 1) You want to determine the precision of the METHOD, not of the PRODUCT, right? The methods precision must be determined using ONE homogeneous sample. Kwet80 already outlined the way to go (determining reapeatability and intermediate precision). After you've determined the method precision, you may assess the precision of the product over several batches.
I do not really understand your comment about the calibration curve. You want to use all the data to generate a calibration? That's a bit strange. A fresh calibration should be done with each sequence.
Concerning 2) An analytical method is not only HPLC. The method consists of sample preparation, HPLC analytics and data evaluation. You've changed sample preapration? -> You've changed the method! It might be a minor change without much impact on the results, but in your case, as you already found out, the changes actually had an impact on the results. So (a bit provocative ): You're trying to compare data generated by different methods!

[bujjikanchi]

See the below link for outlier tests.

http://www.uspbpep.com/usp32/pub/data/v ... s0_c1010t3

Regards,
Bujji Kanchi.