How bad does a peak have to be to manually integrate?

[anionman]

The last QC manager looked at another lab's SOP and it said if it makes a 3% difference in the result it should be manually integrated. I am assuming they got that number by control charting the RPD of duplicate analyses. We don't control chart the duplicate RPD and just used a RPD must be < 10%. So I have been using a 10% criteria for manual integration. I am only having to do it on fluorides and chlorides that are close to the lower reporting limit. A 0.11 mg/L fluoride result might have some coeluting acetate that the program didn't skim off the peak and the manual integrated result is 0.10 mg/L. That's not much difference. I have refined the quantitative method so that these manual integrations are rarely needed but it can never be perfect.

Could I get away with requiring a 20% difference so I never have to manually integrate? I don't actually have a problem doing it myself. It's the other people at the lab who never look at the peaks to see if they need manual integration. I check all their data and once a month or so I have to show people what to do. But then the next time it happens they miss it again.

[Chromavore]

Looking around at various guidelines and sources it seems the use of manual integration is frowned upon but should never be banned, and that its more important that you have a procedure for tracking the use of manual integration and an SOP on how/when to perform integration manually.

"Both the European Pharmacopoeia (chapter 2.2.46) (17) and the United States Pharmacopoeia <621> (18) discuss criteria for good chromatography; there are no criteria or chromatographic integration or a similar discussion on data integrity. However, in the interest of scientific soundness: can you justify and defend your actions on the basis of good chromatographic science?"

source https://rx-360.org/wp-content/uploads/2 ... l-2015.pdf

some of these sources are a bit old but I would say you can set the rules for your labs to whatever you need as long as you can scientifically justify it.

Regards, Chromavore

[anionman]

Thanks for replying. Years ago there was s thread on whether you should be allowed to manually interrogate. You should.
https://www.chromforum.org/viewtopic.php?t=10304

I like these 5 rules;

Rule 1: The main function of a CDS is not to correct your poor chromatography.
This places greater emphasis on the development of robust chromatographic procedures so that the factors involved in the separation are known and controlled adequately. Whenever possible, separations should be developed such that automatic integration is the norm not the exception. Management need to understand that adequate time must be given to method development and validation. This is especially true for pharmacopoeial methods that never work as written.

Rule 2: Never use default integration parameters, always configure specific integration for each method.
Without exception, peak integration and result processing must be defined and validated for each method so that all peak windows and names are defined and if necessary any system peaks are identified. Using a default or generic method results in excessive need for manual integration to name and calculate peaks.

Rule 3: Always use automatic integration as a first option and control manual integration practices.
Remember that the use of manual integration is a regulatory concern and use needs to be scientifically sound. Also be aware that, as discussed earlier, manual integration slows down a process, so see Rule 1 to get the right method depending on the sample matrix and peaks of interest.

Rule 4: Understand how the CDS works and how the numbers are generated.
This requires basic training in the principles of peak integration and how a CDS works. The problem is that with mergers, acquisitions, and encouraging experienced analysts to retire and employ younger workers, skills are being eroded and a CDS can be looked at as a black box that always gives the right answers.

Rule 5: Use your brain-think.
This rule is sometimes difficult to follow but follows on from Rule 4. You can have what appears to be a perfect separation and peak integration, but look at peak start and end placement-do they look right? Use the zoom and overlay functions of the CDS to see if standards and samples have the right peak shape. The analyst is responsible for executing applicable procedures correctly, which includes correct peak integration. The reviewer, however, also has a role to ensure that all integration (whether automated, optimized, or manually placed) follows the method guidance for placing baselines as the SOP describes, especially when the representative area for unresolved peaks are being estimated. Significant peak area manipulation should be easily noticed by an experienced reviewer.

https://www.chromatographyonline.com/vi ... -draw-line

4 and 5 weren't listed t in the article you attached. # 4 is a big problem. People want to treat the CDS as a big pH meter and just accept the answer that pops out without thinking - see rule #5. The lab missed a proficiency several years ago when a baseline problem interfered with a Nitrite peak. After that I got some things written into the SOP about magnifying the chromatogram and looking closely at the baseline. But years later the woman who made the mistake because a supervisor and wanted to undo that to simplify things. She and the other analysts didn't understand how to program the instrument's quantitative method for consistent integration either. I'm still the only one in the lab who seems to know how to do it. One woman just wanted to raise the reporting limits for things, especially fluoride, so the small baseline problems and co-elections don't need to be dealt with.

I attended a TNI training a couple of years ago and it said to maintain the original data that wasn't manually integrated. The rule I am telling people to go by and am trying to get written into the SOP is to make a copy of a file and manually integrate in the copy. Then save the copied file with an M post-fix on it so it is easy to identify as a batch with a manual integration. It's happening 2-3 times a month. Unless someone has a better idea I am going by a 10% rule. I don't know if an auditor will even bother to look at this. They have never looked at our manual integrations in the 13 years I have been at the lab - that's 7 audits. I remember one time the head auditor asked if we ever had to manually integrate and I said yes, but not often. He never looked at any. I am pretty meticulous about it. I print magnified images of the calibrator peaks and the samples peaks before and after manual integration showing how they match how the calibrators are integrated. I also note that the problem cannot be fixed by changing the quantitative method.

My lab has an SOP for manual integration but it was originally written for the organic section because until 10 years ago they are the only ones who performed chromatography. Then we starched using an IC for anions. I have complained about the integration SOP and asked that it be updated, but nothing happens ...So I am pushing for more specifics in the SOP for anions that I deal with.

[Chromavore]

Solidly agree with all your points, out of interest what CDS are you using and can it be configured to flag manual integrations? I know Chromeleon can flag manual integration across injections and even tag single peaks as manipulated or not and can be configured to require a comment if manual integration is used. The report generator can also be configured to show full chromatograms and baseline zooms of regions. Similarly the waters Empower set up of sample sets result sets and report IDs can be used to track changes made to integration, you could have, for example, one set of results with automatic integration then another as manually integrated and a reporting form that said "results reported from result set X manually integrated Yes/No" and delete as appropriate.

I would build in a prompt somewhere in your write ups to make people comment on manual integration use, this will satisfy your and any auditors concerns about tracking its use and also force your analysts to consider its use and review their baselines to check if it was/is needed before submitting results for reporting.

[anionman]

I use chromeleon and it flags chromatograms that are manually integrated, but not specific peaks. If there is a way to do that I haven't figured it out. It will not in any way flag a batch with a manual integration either. It seems to be an easy to cheat so I'm surprised auditors don't look more closely at the manual integrations. I wrote into the SOP for reviewers to check on the chromatograms in all of the calibrators for every batch also. Because you can change a peak in a calibrator to get a QC to pass, especially a reporting limit standard - the rules since 2016 require a passing reporting limit standard.

There is a form we have to fill out for all manual integrations. The problem can be in making the procedure too complicated though. It's hard to find qualified people. The last guy we hired only took 3 chemistry classes in college. A few years ago we had one woman quit because she got frustrated with all the documentation we have to do.

[vmu]

I use chromeleon and it flags chromatograms that are manually integrated, but not specific peaks. If there is a way to do that I haven't figured it out.

It is indicated in the "Type" column of the peak table in the data processing window or in the report.
E.g., see page 9 in the link below.
https://tools.thermofisher.com/content/ ... 455-EN.pdf
The asterisk "*" in "M*" means that the peak was manually integrated. If you use formula "peak.manipulated" in some column of the peak table, there will be indications "TRUE" and "FALSE" for manipulated and automatically integrated peaks, respectively. Similarly, these indicators can be added to the peak labels on the chromatogram (howewer, they look bad when there are many peaks on the chromatogram). You can find more explanations about this in the software help files.

[anionman]

OK I see the * under peak type, but that is not ideal. For some reason when I manually integrate a fluoride it changes the following Cl peak from BMB to BMb and labels it manually integrated also - even though it did not change the area at all. I'm not considering it manually integrated if the area doesn't change.

But I don't know how much time I want to devote to this. I cannot get any other people to look at the chromatograms that closely anyway. They are not even updating the retention times regularly. One run I was given to review had a lot of small Nitrite peaks in it and I thought that was odd. I looked at it and the Nitrate peaks were not Nitrite at all but some interferent that elutes just after Nitrite. I made the woman reprocess the entire run even though none of the peaks was above the reporting limit. If she doesn't learn to look at the chromatograms and update the quant method she will eventually be misreporting results.

[vmu]

I'm not considering it manually integrated if the area doesn't change.

There is almost infinite number of ways to draw a baseline to get the same peak area. Only few of them are sound. Anyway you have to look at the baseline and decide whether it is appropriate for you.

[Chromavore]

Looking back over the thread it seems you don't have a CDS or a manual integration problem but a culture and training problem and honestly I don't have a lot of advice on this one other than an SOP tweak or damage limitation strategies like amending on review isn't going to solve your problem.
A lot easier for me from the outside to say than it is for you to do but your analysts need some quality intervention before results are misreported.

A couple suggestions you can take up with management so this post is productive,
Host a training morning, you obviously know your stuff and care so speak to management and suggest blocking out a couple of hours for refresher training. Other data reviewers or quality managers should care about this or will be annoyed with some other quality aspect if this is such a common problem your lab.

Try and get rid of your generic processing method and make people start thinking about data reporting and setting up their own processing method to assess if they are happy with the integration and peak assignment etc.

You have mentioned correcting analysts work and I obviously don't know you but how do these interactions go in person? Are you approaching corrections from a technical standpoint or a quality perspective or even the legal angle of reporting responsibilities. I care about being technically correct from a personal pride viewpoint but it's the legal aspect of reporting data that keeps me double checking everything and quality comes in from a pure productivity aspect because I don't want to be writing deviations or rewriting reports. Try and find the aspect that folks in your lab care about and drive that home when you bring this up again.

Good luck with some hard conversations,
Regards,
Chromavore.

[anionman]

I'm not considering it manually integrated if the area doesn't change.

There is almost infinite number of ways to draw a baseline to get the same peak area. Only few of them are sound. Anyway you have to look at the baseline and decide whether it is appropriate for you.

I understand that. What I meant to say is that it doesn't change ANYTHING about the Cl peak that follows F except to label it Bmb and put the asterisk next to it that signifies a manual integration. This is in the old Chromeleon 6.8 software. The lab has a newer Integrion with Chromeleon 7.2 where that doesn't happen. The ICS-2100 with 6.8 is still our main instrument though. Looks like a software flaw.

Looking back over the thread it seems you don't have a CDS or a manual integration problem but a culture and training problem and honestly I don't have a lot of advice on this one other than an SOP tweak or damage limitation strategies like amending on review isn't going to solve your problem.
A lot easier for me from the outside to say than it is for you to do but your analysts need some quality intervention before results are misreported.

A couple suggestions you can take up with management so this post is productive,
Host a training morning, you obviously know your stuff and care so speak to management and suggest blocking out a couple of hours for refresher training. Other data reviewers or quality managers should care about this or will be annoyed with some other quality aspect if this is such a common problem your lab.

Try and get rid of your generic processing method and make people start thinking about data reporting and setting up their own processing method to assess if they are happy with the integration and peak assignment etc.

You have mentioned correcting analysts work and I obviously don't know you but how do these interactions go in person? Are you approaching corrections from a technical standpoint or a quality perspective or even the legal angle of reporting responsibilities. I care about being technically correct from a personal pride viewpoint but it's the legal aspect of reporting data that keeps me double checking everything and quality comes in from a pure productivity aspect because I don't want to be writing deviations or rewriting reports. Try and find the aspect that folks in your lab care about and drive that home when you bring this up again.

Good luck with some hard conversations,
Regards,
Chromavore.

You have no idea how hard the conversations have been before this. I have a long thread about people ignoring coelutions in the IC section. We have a policy that all work must be reviewed by another analyst and if I don't do the analysis I review it.

Before trying to get everyone is trained I want to get opinions on when manual integration is needed. I don't have the authority to change our lab's Peak integration SOP because that is mainly used by the organic section. I have made suggestions that it needs changes, but I don't think it will happen. That's why I want to get things written into the IC SOP.

I didn't think it would be such a complicated answer. I also just want to make sure I'm not nitpicking over the data quality. So far these manual integrations have only been needed in peaks that are within 10% of the reporting limit. In the old days you only had to check a mid level CCV and a blank, but now TNI rules say labs have to check at the reporting limit. One women who retired last year just wanted to raise the reporting limit. If the reporting limit for fluoride was 0.2 mg/L manual integration might only be needed once a year.

[wss]

The answer depends on your quality system really. The general approach is, if you can justify it technically and it maintains the quality of the end results, do it. This does put a lot of the onus on the individual analyst however. We use MassHunter which can flag manual integrations and is useful if data needs to be audited. We've never been pinged on a lack of manual integration guidelines whenever we've been audited. Then again, our scope of work may be quite different (contract environmental testing).

[anionman]

In our lab, we don't have specific documented guidelines regarding manual integration. The general approach is, if you can justify it technically and it maintains the quality of the end results, do it. This does put a lot of the onus on the individual analyst however. We use MassHunter which can flag manual integrations and is useful if data needs to be audited. We've never been pinged on a lack of manual integration guidelines whenever we've been audited. Then again, our scope of work may be quite different (contract environmental testing).

Thanks. We have never been called out on manual integrations either, but I want to stay ahead of the curve and address the issue. We analyze drinking water samples for the health department and TNI comes by every 2 years to audit. They have never looked at a manual integration and I don't know if they even look at chromatograms. Last time one woman from EPA looked at ONE chromatogram. I don't see any guidance in all the TNI regulations about when to manually integrate. And they have so many regulations!

[lmh]

I'm going to be a bit harsh about this one. If you can't get consistent peak areas from an integrator, the inconsistency almost certainly isn't a problem with the integrator, it's a fundamental problem with the peak. And because it's a fundamental problem, manual integration can hide it, but won't make it go away. I can get consistent areas out of a peak by manually integrating it until I get the answer that I "know" is right, but the only real evidence that it's right is my opinion. This is define-your-own-error-bars territory.
The fact that manual integration is necessary is a massive, massive red flag.

But I'm in agreement (or near-agreement) with all the points in that list, including using a human brain. The human can accept/reject the computer's assessment. It's just that my view is the human should be defining the overall high-level stuff: what level of inconsistency can I tolerate, how am I going to set a threshold below which I think the peak cannot be handled in a way that gives me that consistency? The aim is for the human to set up the CDS to carry out an automated analysis that's fit for purpose and leaves most samples measured, and the remaining rejects rejected. Not to try to salvage rejects by human optimism.

[anionman]

I'm going to be a bit harsh about this one. If you can't get consistent peak areas from an integrator, the inconsistency almost certainly isn't a problem with the integrator, it's a fundamental problem with the peak. And because it's a fundamental problem, manual integration can hide it, but won't make it go away. I can get consistent areas out of a peak by manually integrating it until I get the answer that I "know" is right, but the only real evidence that it's right is my opinion. This is define-your-own-error-bars territory.
The fact that manual integration is necessary is a massive, massive red flag..

Not really a red flag. It just shows that I am being very precise. As I said, TNI is now telling us we have to be able to see at the reporting limit.
The only peaks that have been a problem are fluoride and it is written in the method that " Known coelution is caused by carbonate and other small organic anions. At concentrations of fluoride above 1.5 mg/L, this interference may not be significant, however, it is the responsibility of the user to generate precision and accuracy information in each sample matrix."

Acetate is the problem when quantifying fluoride at low levels. I set a low shoulder threshold, but sometimes the instrument will not skim it no matter how low I go on the shoulder threshold or how high I go on sensitivity. The instrument has been skimming the acetate in all the standards up to 4 mg/L so for consistency it should skim it is all the samples. If the fluoride level is below our reporting limit it often doesn't skim acetate but it doesn't matter because the result reported is the same. On most samples it makes less than a 10% difference. It's only in the low level right around the 0.1 mg/L reporting limit where it makes that much difference. Last week a sample peak was read by the IC at exactly 0.1000 mg/L, but the acetate was not skimmed. With it skimmed the result was 0.0924 mg/L. That's less than a 10% difference, but I manually integrated it because it should have reported as < reporting limit.

This is not an image from my IC but it illustrates that problem. In the right hand image that would be enough of a shoulder for the instrument to see, but the left image shoulder might be too small.