How does Empower round this number?

Discussions about chromatography data systems, LIMS, controllers, computer issues and related topics.

4 posts Page 1 of 1
I am working on a custom field (Boolean) that calculates a result of Pass or Fail but my spec is NMT 2.0%. What is the limit that you can put in the CConst field of the proc method that will return a Pass? If I have a result of 1.5 does Empower round it to 2.0 or does it depend on reported precision eg is 1.499999 still less than 2.0?
I'm having trouble trying to decide what to put when I'm trying to decide what would exceed a spec of for example NLT 98%, NMT 1.0%. Any tips?
Hi,

Not sure , Im guessing but depending on dp shown empower rounds up

ie 1.49 shown = 1.5 (to 1dp)

Maybe worth having custom field with ROUND or truncation formula , though may need to check /follow internal SOP's ie
combination of both (probably not needed here but good example):

ROUND(Amount+0.005,-2)-0.01
When comparing two numbers, ALWAYS use ROUND function to round your result to the same precision as your specification limit before making the comparison. Never compare two number with different precision. Never assume the program has the same rounding rule as what you are expecting.
1 good example above...
This has been covered at great length in Waters Inform conferences over the years - Google that for custom field information.
Thanks,
DR
Image
4 posts Page 1 of 1

Who is online

In total there is 1 user online :: 0 registered, 0 hidden and 1 guest (based on users active over the past 5 minutes)
Most users ever online was 1117 on Mon Jan 31, 2022 2:50 pm

Users browsing this forum: No registered users and 1 guest

Latest Blog Posts from Separation Science

Separation Science offers free learning from the experts covering methods, applications, webinars, eSeminars, videos, tutorials for users of liquid chromatography, gas chromatography, mass spectrometry, sample preparation and related analytical techniques.

Subscribe to our eNewsletter with daily, weekly or monthly updates: Food & Beverage, Environmental, (Bio)Pharmaceutical, Bioclinical, Liquid Chromatography, Gas Chromatography and Mass Spectrometry.

Liquid Chromatography

Gas Chromatography

Mass Spectrometry