.jpg){kind=link}
- Published
- Updated
Automated triage may perform well on initial examples, yet the same result should not be assumed when incoming records change. A new language, longer text or different site practice can alter error patterns. Oversight should examine the inputs being used as well as the system version.
Scope of the decision
Input change does not necessarily mean the model itself changed. Situations absent from the original evaluation may constitute a different use scope. Connect relevant input characteristics and the human reviews triggered by their change to the intended decision.
Worked review
For example, a triage system was evaluated using short English notes. New sites begin providing long bilingual records. Overall accuracy appears unchanged, but some important issues receive low priority. The team examines that subgroup separately and defines an appropriate use boundary and additional control.
Practical checks
- Compare new inputs with characteristics represented in the earlier evaluation.
- Examine important errors by language, length or source group where relevant.
- Keep appropriate human routing visible for inputs outside the supported scope.
Acceptance and limits
Acceptance requires adequate evidence for the current input group and explicit limitations. Successful historical testing does not provide unlimited assurance for different records. Define in advance responsibility and authority to restrict or stop automated decision support when monitoring identifies a problem.
Frequently asked questions
- Is reassessment needed if the system has not changed?
- It may be. Input characteristics and the setting of use can also affect reliability.
- Is an overall accuracy percentage sufficient?
- Not alone. Important misprioritisation can remain hidden within one subgroup.