Automated grading concordance in prostate biopsy cohorts

This is an OncoWiz educational overview of a research area, written for clinicians. It summarises the shape of the evidence and the questions worth asking of it. It is not a summary of any single study, and it reports no individual trial’s results.

Prostate biopsy grading is an attractive target for computational pathology because the grading system is explicit, the clinical consequences of the grade are large, and inter-observer disagreement among expert pathologists is well documented. That last point is also what makes the evidence hard to read.

Concordance with what?

A model’s grading concordance is only as meaningful as the reference it is measured against. Agreement with a single reporting pathologist, with a panel consensus, or with a specialist uropathology review are three different questions with three different answers, and the strongest reference is the one hardest to assemble.

Because expert pathologists disagree with each other, a model measured against one reader is being measured against a moving target. Reported superiority to an individual may say more about that individual than about the model.

Where disagreement concentrates

Disagreement is not spread evenly across grade groups. It clusters at the decision boundaries that carry the most clinical weight — particularly the distinction that separates active surveillance candidates from those directed to treatment. A model with strong overall agreement can still be unreliable at precisely the boundary the report exists to inform.

• Boundary cases between adjacent grade groups drive most of the observed discordance.
• Small tumour volumes and limited cancer on a core give the model less signal, and give the pathologist less too.
• Variant morphology and treatment effect are under-represented in training data relative to their clinical importance.

The laboratory-transfer problem

Staining protocol, scanner model and slide preparation differ between laboratories, and a model trained in one laboratory frequently degrades in another. Stain normalisation and colour augmentation reduce the gap but do not close it. Any adoption decision should rest on performance measured on slides prepared and scanned the way your laboratory prepares and scans them.

Questions worth asking

• What reference standard defined the ground truth, and how many pathologists contributed?
• Is performance reported per grade group, or only in aggregate?
• Was the model validated on slides from external laboratories with different staining and scanning?
• How does the system behave on cases it should decline — poor-quality slides, unusual morphology?
• Is the intended use assistive, and what is the pathologist expected to do with the output?