Reproducibility of radiomic features across acquisition protocols

This is an OncoWiz educational overview of a research area, written for clinicians. It summarises the shape of the evidence and the questions worth asking of it. It is not a summary of any single study, and it reports no individual trial’s results.

Radiomics extracts large numbers of quantitative descriptors — texture, shape, intensity statistics — from routine images and treats them as candidate biomarkers. The field’s central methodological problem is not finding signal. It is establishing that the signal describes the tumour rather than the scanner.

Why reproducibility is the first question, not a later one

A radiomic feature is a number computed from voxel values. Anything that changes those voxel values changes the feature, whether or not the underlying biology changed at all. Slice thickness, reconstruction kernel, field of view, dose, scanner vendor, contrast timing and the segmentation itself all move features, and many texture features move a great deal.

This means a signature can achieve genuine, reproducible predictive performance within one institution and carry none of it to the next, because what it learned was partly the local imaging protocol. A model that is not tested for feature stability has not been shown to be a biomarker.

The main sources of instability

• Acquisition and reconstruction. The largest effect, and the one most often unreported in published work.
• Segmentation variability. Shape and boundary-sensitive features move with inter-observer contour differences; automated segmentation reduces variance but introduces its own systematic bias.
• Preprocessing. Voxel resampling, grey-level discretisation and intensity normalisation are analysis choices, and different choices produce different features from identical images.
• Feature-definition drift. Two implementations of “the same” feature can compute different values unless both follow a standardised definition.

What good practice looks like

Test-retest and multi-scanner phantom work identify which features are stable enough to be worth modelling, and should precede model fitting rather than follow it. Harmonisation methods can reduce batch effects statistically, but they correct for measured site differences and cannot rescue a feature that is inherently unstable.

Standardised feature definitions and fully reported preprocessing are what make a published signature reproducible by anyone else. Without them, an external validation cannot be attempted, let alone fail.

Reading a radiomics paper

• Was feature stability assessed, and were unstable features excluded before modelling?
• How many features were screened against how many patients? High-dimensional screening on a small cohort produces significant-looking findings by construction.
• Was the model validated on data from other institutions and other scanners?
• Are preprocessing choices reported in enough detail to reproduce the pipeline?
• Does the signature add anything beyond established clinical and pathological predictors?