Reporting with decision rules ISO 17025

Do you report results as pass or fail on test reports? As normal, high or low? Within spec, outside spec, complying or not complying? Would your clients like you to report with a statement of conformity? Then you need to know about decision rules in the new ISO 17025.

While you have the utmost confidence in your laboratory’s test results, no test result is exact: there is always a band of uncertainty around it. If the result is close to the limit do you really know if it passes or fails when you consider the band of uncertainty? Is this a decision or risk you are willing to take?

Uncertainty of Acceptance

The drawing shows

The tolerance limits: allowable results, normal range, standard specification, etc. Any result within this range is acceptable to the customer.
Four possible laboratory results, with 95% confidence intervals shown.

We need a way of deciding if we will report the results as pass, fail, or even indeterminate.

The old approach

Using a standard decision rule that the results will be reported as pass or fail based on the numerical result, with 95% certainty:

A can be reported as a pass with a 95% confidence interval.

B can be reported as a fail, or above spec with a 95% confidence interval.

C can be reported as a pass, with a 95% confidence interval.

D can be reported as a fail, or below spec with a 95% confidence interval.

The new approach

The revised ISO/IEC 17025 uses the term “decision rule”and emphasises the need to communicate clearly with your clients about the strengths and limitations of the testing (clause 7.1.3) and to formally document decision rules (clause 7.8.6).

It might be more relevant to explain to your clients that:

A is a pass with a 95 or even 99% confidence interval.

B there is only 60% certainty that B is a fail, 20% of products that get this result could pass.

C There is a 2.5% chance that C is a fail.

D can be reported as a fail, or below spec with a 95 or 99% confidence interval.

Then you can discuss setting “grey zones” where results may be reported with additional comments to highlight that they fall close to the cut-off.

How you handle this depends on a range of factors, including the industry your clients are in, the uncertainty inherent in your method, the regulatory environment, the cost of false passes or fails to you and your clients and perceptions of risk.

Do we need a decision rule?

Before you get into the details, there is an easy and safe way out: Don’t report pass or fail and let the client make the decision. You could report the tolerance and specification alongside the results to help the client make the decision, and reduce your risk in taking that final decision.

Developing a decision rule

Go to your client/s armed with some facts and thoughts on the following:

How big is the band of uncertainty? (i.e. what is the uncertainty of measurement). How does this compare to the tolerance? Is that overlap in the band of uncertainty and tolerance likely to occur regularly?
Is it more important for the property to not exceed the limit or to fall below it? Or does it really need to hit the mark?
When you talk in terms of probability of a false positive/pass/accept versus probability of a false negative/fail/reject, how does you client see it?
Can they place a cost or risk assessment on the value of false passes or false fails?

You’ll get some other ideas by searching the internet with the following search phrases:

Accuracy Method

Consumer’s Risk

Shared Risk

Global Risk

Producer’s Risk

Guard Bands

Reporting compliance statements

If you are reporting compliance or non-compliance, make sure the reports are clear about exactly what you are passing or failing. The report also needs to clearly state:

Which results the statement of conformity applies to
Which specification/standard is met / not met
The decision rule applied (unless it is part of the specification/standard)

Examples of Decision Rules

The PME Services Decision Rule

The IKEA Test Laboratory Decision Rule

WADA Decision Rules

While these examples make interesting reading, you must get agreement from your customers for the decision rule/s you select. If you have significant customers, I suggest conferring with them and getting their agreement before publishing a blanket rule on your website.