Creeping criteria: this wonderful term was coined by my old boss at NATA (Helen Liddy) for those times we put a “condition” in an assessment report that was almost a requirement (but really an opinion on how to address a requirement), or when we had taken a criterion from one part of the accreditation documents and applied it to a completely different situation. It also referred to when we heard something from a technical assessor and applied it without looking up the standard.
Whether you work for NATA or as an internal auditor, you will almost certainly fall prey to creeping criteria from time to time. Luckily most internal auditors have their work checked by the quality manager before the audit report is issued. The same cannot be said for NATA auditors, many of whom have approval to issue assessment reports without review by another staff member.
What you need to know to successfully respond to audit and NATA reports
To avoid wasting time on unnecessary tasks, you need to be fully across what really constitutes a requirement and what does not. You cannot rely on auditors to know every detail of how the requirements apply to your lab.
But aren’t the auditors and NATA the experts, I hear you ask? Don’t they know?
The truth is that auditors have a huge amount of information to deal with and process, particularly when auditing in labs they’re unfamiliar with, so sometimes they get it a little bit wrong, or even a lot wrong. It can really pay to look up the documents that they reference in the assessment/audit report to get a better idea of what they are referring to and what the requirements actually are before responding to the report.
Here are some examples of how creeping criteria happens and the best way to respond:
Clause creep
Taking a requirement from one part of the standard and applying it to another part, or from one piece of equipment and applying it to other pieces of equipment. This real scenario happened many years ago during a NATA assessment:
Condition: Whiteout must not be used in controlled documents. Methods in the Serum Rhubarb section contain multiple examples of use of whiteout.
Clause referenced: Medical Testing FAD section 4.13.
Investigation: Clause 4.13 is about test records. It is talking about making sure that all original test data is preserved in a way that is permanent and legible. We then spoke to the lab manager concerned, who explained that she can’t stand having typos in the bench copies of test methods. She just has to fix them & doesn’t want to print them out again.
Response: Gently pointed out the reference was to test records, not controlled documents.
Corrective action: None required. However we did update our document control procedure to define how hand-written and white-out changes to documented procedures could be accommodated.
Take-away: Always look up the requirements if a non-conformance does not make good solid sense. If we had followed through with this for controlled documents, we would have prohibited a practice that is in fact allowed.
Context creep
This happens when an auditor hears something during an audit of one laboratory and applies that to another laboratory. This can be a battle you have with staff who are technical assessors and come back from assessments and apply these criteria during internal audits. A simple example that happened to me once:
Condition: Each line in the training records must be individually dated and initialled – you can’t draw a bracket around a group of training topics and mark them with one date and initials.
Clause referenced: ISO 17025: 6.2.6 & 8.4.1
Response: We don’t see in those clauses that it says we can’t record training this way.
Auditors response: But the NATA lead auditor at the lab assessment I did last week said that they couldn’t do this. I thought it was “a thing”.
Quality manager’s response: You’re asking them to do busy work. What is the benefit for them? There’s nothing specific in the requirements to prohibit this unless you are saying it’s not clear when or what training was done because of the way they’ve recorded it?
Take-away: This may be a requirement for the other lab because of the way their system is designed, or the way they have approached their training. All non-conformances, whether raised by NATA or others, are highly influenced by context and even the questions that we asked at the time. The NATA auditor may also express thoughts during the audit that do not end up in the assessment report.
The never-ending story
Something along the lines of this scenario happens to one of our clients every now and again:
Condition: “Staff member Orange at lab G conducted testing to method One; however, records to indicate whether the staff member had been assessed for competence or authorised to conduct the testing were not available.”
Response: Sent in the training records for the whole lab (as an extract from the database, the easiest way to do it).
Auditor’s response: Thank you but there do not appear to be records to show that staff member Brown or Green have been approved to do test methods Two, Four or Seven.
Response a: Staff member Brown does not do methods Two, Four or Seven as he is mainly out on field work. Staff member Green was trained at one of the other labs – her records are in their system. Attach copies of training records for Green.
or
Response b: We have supplied the training records for staff member Orange and are unsure why you are asking for other training records which were available at the assessment. (This response is for those who are feeling assertive)
Corrective action: non required
Root cause analysis: the lab manager didn’t know where to find the training records for staff she hadn’t trained herself.
Take-away: Audits and assessments are finished when the audit team leaves the building (whether that be literally or figuratively). You do not have to keep answering additional questions after the assessment.
Implied corrective action
Sometimes an auditor tells you what to do instead of what is wrong. In this case the message came through from a combination of what was discussed at the assessment and what ended up in the assessment report.
Condition: “The lab has held two meetings in the past 7 months, about 4 months apart.” (No indication of what was wrong with that.)
Clause referenced: ISO 15189 4.1.2.6 Communication (closest clause in ISO 17025 is 5.7a)
“Laboratory management shall have an effective means for communicating with staff. Records shall be kept of items discussed in communications and meetings.
Laboratory management shall ensure that appropriate communication processes are established between the laboratory and its stakeholders and that communication takes place regarding the effectiveness of the laboratory’s pre-examination, examination and post-examination processes and quality management system.”
The lab was planning to reply that they would hold lab meetings every month. This seemed to be what the auditor was asking for. The quality officer investigated:
Investigation:
- A small lab with 4 staff members who all work day shift and infrequent changes to
methods or processes. They see and speak to each other 5 days a week. There was
no problem with the meeting frequency given these factors, the scope and stability of the lab. - The quality manual did not mention lab meetings, nor how often they should be.
- We looked at the clause mentioned, and saw that it is about communication, not meetings
per see. - We discussed what communication needs the lab has other than the things they normally
cover in staff meetings and how these can be recorded. Some examples include:- Notices
- Read receipts for document updates
Response to NATA: Reported on our investigation and advised that current meeting frequency is sufficient for the needs of the lab. Refrained from pointing out that the condition reads like an observation. To be a condition there must be a specific example of something that is wrong.
Corrective action: Maintain informal approach to scheduling meetings as required. Keep copies of notices and read receipts in a dedicated folder.
Take-away: Always look up the original requirements if a non-conformance does not make sense for your laboratory. Don’t jump too quickly to adopt whatever the auditor seems to be suggesting!
Giving extra backbone to the requirements
Condition: The laboratory must document a procedure for addressing risks and opportunities
Clauses referenced: ISO 15189 4.14.6 or ISO 17025 8.5.
Investigation:
- A small lab with very limited capacity to carry out additional administrative processes.
- Part of a larger organisation with risk management processes that can be accessed if needed. Required to consider risk when deciding on corrective action.
- Risks have been identified and controls put in place – these are detailed in the regular data review processes, management review, internal audit program, and extensive QC checks.
- We read through all of the clauses mentioned by the assessor and found no reference to formal risk management systems. You have to DO risk management, not write about how to do it.
Response: we can’t find any requirement for a formal risk management system in the documents referenced.
Auditor’s response: nil
Take-away: Standards say we have to use risk based thinking and evaluate risks but they don’t say we have to have a formal system or procedure for doing this.
There are many similar examples, especially arising from more recent ISO 17025 assessments, which is much less prescriptive of procedures. You have DO your processes, not so much write about them. If your processes are well designed, they will be visible and easy to follow without having to write procedures for them.