How does Empower round this number?

Discussions about chromatography data systems, LIMS, controllers, computer issues and related topics.

4 posts Page 1 of 1
I am working on a custom field (Boolean) that calculates a result of Pass or Fail but my spec is NMT 2.0%. What is the limit that you can put in the CConst field of the proc method that will return a Pass? If I have a result of 1.5 does Empower round it to 2.0 or does it depend on reported precision eg is 1.499999 still less than 2.0?
I'm having trouble trying to decide what to put when I'm trying to decide what would exceed a spec of for example NLT 98%, NMT 1.0%. Any tips?

Not sure , Im guessing but depending on dp shown empower rounds up

ie 1.49 shown = 1.5 (to 1dp)

Maybe worth having custom field with ROUND or truncation formula , though may need to check /follow internal SOP's ie
combination of both (probably not needed here but good example):

When comparing two numbers, ALWAYS use ROUND function to round your result to the same precision as your specification limit before making the comparison. Never compare two number with different precision. Never assume the program has the same rounding rule as what you are expecting.
1 good example above...
This has been covered at great length in Waters Inform conferences over the years - Google that for custom field information.
4 posts Page 1 of 1

Who is online

In total there are 7 users online :: 0 registered, 0 hidden and 7 guests (based on users active over the past 5 minutes)
Most users ever online was 234 on Tue Feb 06, 2018 7:33 am

Users browsing this forum: No registered users and 7 guests