Risk-model integration into multidisciplinary workflow

This is an OncoWiz educational overview of a research area, written for clinicians. It summarises the shape of the evidence and the questions worth asking of it. It is not a summary of any single study, and it reports no individual trial’s results.

A prediction model that performs well and is never used has changed nothing. The literature on integrating risk models into multidisciplinary team working is consistently less flattering than the literature on building them, and the reasons are organisational rather than statistical.

The gap between accurate and adopted

A tumour board is a structured negotiation between specialties under time pressure. A model output entering that setting competes for attention with imaging, pathology, performance status and patient preference, and it arrives without the professional standing those inputs carry. Accuracy is necessary and nowhere near sufficient.

The recurring barriers are practical:

• Workflow placement. A number that requires leaving the working system to obtain is a number that will not be obtained.
• Timing. Output arriving after the discussion has converged changes nothing.
• Interpretability. Clinicians asked to act on a figure reasonably want to know what drove it, and a plausible explanation is not the same as an account of the model’s reasoning.
• Accountability. Where a recommendation is followed and the outcome is poor, the responsibility remains with the clinician, which is a strong and rational reason for caution.

Automation bias runs the other way

The mirror risk is equally real. Once a model is trusted, its outputs stop being scrutinised, and a system that is right most of the time can still cause net harm through the cases where it is wrong and unchallenged. This is why the meaningful evaluation measures the clinician-and-model system prospectively, not the model in isolation.

What separates implementations that hold

• The output appears inside the system clinicians already use, at the moment the decision is made.
• Calibration is demonstrated in the local population, not just discrimination in the development one.
• The model states its uncertainty, and declines cases outside its competence rather than answering anyway.
• A named owner monitors performance after go-live, because data drift degrades a model silently.
• The team agreed in advance what the output does and does not license.

The question behind the question

Before asking whether a risk model is accurate enough to adopt, it is worth asking which decision it is meant to change, and whether that decision is currently being made badly. A model that predicts well but informs no decision that was in doubt adds work without adding value.