- Published
- Updated
Changing a decision threshold can alter user actions even when the underlying model remains the same. More positive outputs or fewer alerts do not automatically mean better clinical performance. Evaluate the threshold within the intended decision context.
Define the assessment
Explain what output the threshold produces and who uses it for which action. Separate technical accuracy from consequences of false-positive and false-negative results. Selecting a threshold and presenting definitive validation on the same data can produce an optimistic assessment.
Worked example
For example, screening software is adjusted to generate more alerts. Retrospective examples show fewer missed cases but a higher review workload. The team reports more than the improved sensitivity: it explains the threshold-selection data, independent assessment scope and effects of the additional workload on the workflow.
Practical checks
- Compare outcome changes at old and new thresholds on traceable examples.
- Document separation of threshold-development data from independent evaluation data.
- Assess changes to presentation, user training and subsequent action.
Decision and limitations
Acceptance should explain the rationale and uncertainty for the intended decision. Do not extend improvement in one metric to every clinical benefit. Authorised reviewers assess effects on regulatory and investigation arrangements; an adjustable setting does not grant unlimited authority to change research conditions.
Frequently asked questions
- Is a threshold change minor if the model is unchanged?
- Not necessarily. Changed outputs and user actions may alter clinical interpretation.
- Is maximum accuracy the only objective?
- No. Consider error types, intended use and the possible consequences of the decision together.