The EQA Blind Spot: A Good Score, A Bad Result

A quality manager I respect keeps a green EQA report pinned above her desk. It is a good report. Every analyte in range, every return on time, a tidy row of acceptable flags stretching back through the year. She points at it when auditors visit and she is right to. It is honest, hard-won evidence that her people run their controls, read the instructions, and take part in the wider community of laboratories that agree to be measured. I would trust her service further than most.

And yet I have watched services with reports exactly like hers hand out results that were quietly, systematically wrong. Not because anyone was lazy or dishonest. Because the report answered a narrower question than the one everybody assumed it answered. A passing external quality assessment score tells you that, on a handful of shared samples, at a few moments in the year, your method sat near its peers. It does not tell you that the finger-prick result you gave a worried patient at three in the morning was right.

That gap is the subject of this piece. My thesis is simple and, I hope, uncomfortable: a good EQA score is reassuring, necessary, and not proof that your patient results are correct. EQA is one instrument on the dashboard. Treat it as the verdict on your service and you will, sooner or later, get caught out.

What EQA actually measures, and what it quietly does not

External quality assessment, run by schemes such as UK NEQAS and WEQAS, works by sending the same material to many laboratories, collecting the results, and telling each participant how far they sat from the consensus or a reference target. It is genuinely powerful. It surfaces gross error, it holds methods to account against a wider population, and ISO 15189:2022 rightly requires participation in interlaboratory comparison. If you are not in a scheme, you are flying without one of the few external references available to you. Nothing that follows should be read as an argument against EQA. It is an argument against mistaking it for something larger than it is.

The trouble is that the picture EQA paints is small and sharply framed. It sees your analytical trueness, on that specific material, at that specific distribution, against whatever peer group the scheme could assemble. Everything outside that frame, and there is a great deal, is invisible to it.

Let me take the blind spots one at a time, because each is a different way for a green report to sit on top of a bad result.

Commutability: the sample that behaves like nothing you will ever test

Start with the one most people have heard of and fewest have felt in their bones. Commutability is the property of an EQA sample behaving, across different measurement methods, the way a real patient sample would. It is not automatic. EQA material is often pooled, processed, stabilised, sometimes spiked to hit target concentrations, and that manufacturing can change how the sample interacts with a given reagent or analyser. ISO 15189:2022 added a formal definition of commutability precisely because it matters this much.

When material is non-commutable, the EQA result and the patient result can part company. A device may sit beautifully on the survey and carry a real bias on patients. Or, just as awkwardly, it may look like an outlier on the survey while measuring patients perfectly well, because the artificial material provokes a matrix effect that living plasma never would. The green tick and the truth have simply stopped being the same thing.

A non-commutable sample can make a biased device look accurate and an accurate device look biased. The report tells you about the sample, not always about the patient.

This is why the standard is careful about peer groups: an interlaboratory comparison is only meaningful when you are compared against the same method and equipment, on material that behaves like a real specimen for that method. Read your EQA reports with commutability in mind and you start to read them as a professional rather than a spectator.

The snapshot problem: drift lives in the gaps

EQA is periodic. Distributions arrive at intervals, monthly or quarterly depending on the scheme and analyte. Your patients arrive every day. Between two green distributions there is a long, unlit corridor, and analysers drift down corridors like that all the time: a reagent lot changes, a calibration ages, a probe fouls, a fridge warms overnight. None of it announces itself to a scheme that will not sample you again for weeks.

So a method can pass in March, drift through April, and pass again in May because the drift happened to be small at the two moments the scheme looked. The report is not lying. It simply never saw the corridor. This is the deepest reason EQA cannot stand alone: it is the wrong tool for continuity. Internal quality control, run every day between distributions, is what watches the corridor. EQA checks where you are against the world a few times a year; IQC checks whether you have moved since yesterday. You need both, and they answer different questions.

Small and mixed peer groups: when "acceptable" is soft

For core chemistry on large analysers, peer groups can run to hundreds of laboratories and the consensus is robust. For point-of-care methods, the ground is far less firm. A given POCT device, in a given scheme, for a given analyte, may have a peer group of a handful of participants. Sometimes the group is not only small but mixed, several related methods pooled together to reach a workable number, which blurs the very comparison you are relying on.

When the peer group is thin or heterogeneous, the acceptance limits widen and the consensus wobbles. Your "acceptable" flag becomes a soft judgement drawn from very few neighbours, and an entire method's shared bias can hide inside it, because if everyone using the device drifts the same way, the consensus drifts with them and no one looks like an outlier. A green flag against three peers is not the same evidence as a green flag against three hundred. It is worth knowing which one you actually hold before you lean your service on it. This is one of the genuinely hard parts of POCT quality, and it is a recurring theme in our training: the flag is only as strong as the group behind it.

The white-glove sample: EQA never meets your real world

Here is the blind spot I find most persuasive at the bench, and the one most invisible on paper. EQA samples are handled with unusual, atypical care. They arrive labelled, they are logged, the coordinator runs them personally or hands them to the most careful operator, and everyone knows they are being watched. The sample is treated like an exam. Your patients are not.

Real point-of-care testing happens in the actual world. A cold finger, milked too hard, giving a diluted or haemolysed drop. A cartridge from a box that sat in a warm store cupboard above a radiator all summer. A locum who was shown the device once, months ago, working a night shift with a queue building. A strip pulled from a pot left open on a windowsill. None of this reaches the EQA scheme, because the EQA sample never experiences your untidy Tuesday. It experiences your best behaviour on a good day.

EQA tests your best operator on a good day. Your patients meet your tiredest operator on your worst one. The gap between those two is where harm lives.

So a device can pass every survey and still be handled, day to day, in ways that generate wrong results, because the survey and the daily reality are different events performed by different people under different pressure. This is exactly why operator competence, technique and environment matter as much as the box itself, and why we treat them as first-class quality issues rather than afterthoughts. The fundamentals of POCT live here.

Pre-analytical error: the mistakes EQA structurally cannot see

Widen the lens once more. In laboratory medicine as a whole, the largest share of error is pre-analytical: wrong patient, wrong sample, wrong tube, wrong timing, wrong handling, results entered against the wrong record. These are the mistakes that reach patients most often, and EQA is structurally blind to almost all of them. The scheme hands you a known sample with a known identity and asks only whether your analyser measured it near its peers. Identification, collection, timing and transcription, the whole pre-analytical arc where real harm concentrates, never enters the frame.

You could run a service with immaculate EQA and still give a diabetic the wrong glucose because the strip belonged to a different patient, or a wrong INR because the sample sat too long before testing. The analytical phase EQA guards can be perfect while the phases it cannot see fail. If you audit your own incidents honestly, most of them will have started before the analyser ever ran. Our resources and templates are deliberately weighted towards this arc, because it is where the numbers say the damage is done.

The answer is triangulation, not a bigger EQA

None of this means EQA is weak. It means EQA is one instrument, and no single instrument tells you your service is safe. The services that stay safe triangulate, they insist on several independent lines of evidence that have to agree before anyone relaxes:

• EQA for periodic, external, peer-referenced trueness. Keep participating, and read the reports critically: check your true peer group size, watch for commutability caveats, and treat a soft "acceptable" against few peers as weaker evidence than a firm one against many.
• Well-designed IQC for daily, internal continuity. This is what watches the corridor between distributions. Choose rules that catch real shifts without drowning you in false alarms, and act on trends, not just single breaches.
• Split-sample comparison with your reference laboratory. Send the same patient specimen to both the POCT device and the main lab and compare. This is one of the few checks that sees your real matrix and your real bias together, on a genuine patient sample, which is exactly what EQA cannot promise.
• Your own patient-result patterns. Watch the distribution of the results you actually issue. A slow drift in your mean, a change in how often you flag abnormals, a shift after a lot change, these are early warnings that no periodic survey will give you.

Put those four together and each one covers another's blind spot. EQA sees the world you cannot; IQC sees the days EQA misses; split-sample comparison sees your real patients in your real matrix; your own result patterns see the drift before the next distribution ever arrives. That is a dashboard. A single green report is one dial with the rest of the panel dark. If you want a hand building this into something your team will actually maintain, that is precisely the kind of work our consultancy exists to do, and it is why we spend so much time on how a device behaves in real hands rather than only on the numbers it prints. The analyte reference is there for the method-specific detail.

What to actually do on Monday

If you take one thing from this, make it a habit of reading EQA as a professional, not a passenger. Concretely:

• Read the small print on every report. Note your actual peer group size and method group. A green flag against a handful of peers is weaker evidence than a green flag against hundreds. Write the number down.
• Never let EQA stand in for IQC. If your IQC is thin, fix that first. Continuity between distributions is where most drift is caught, and no survey will catch it for you.
• Run split-sample comparisons on a schedule. Pick real patient specimens, send them to both the POCT device and the reference lab, and log the differences over time. This is your best line of sight onto real-world, real-matrix bias.
• Watch your own issued results. Track means and abnormal-flag rates by device and by lot. Investigate step-changes even when the last EQA was green.
• Audit the pre-analytical arc. Most of your real errors start before the analyser. Identification, technique, storage, timing and transcription deserve as much attention as the survey, arguably more.
• Treat operator competence and environment as quality data. Who ran it, how, in what conditions. The white-glove sample never captures this, so you must.

The services that get caught out are almost never the ones with bad EQA. They are the ones that treated a green EQA report as permission to stop thinking. A good score earns you the right to keep looking, not to look away. Pin the report above your desk, by all means. Then turn round and check the corridor.