.jpg){kind=link}
- Published
- Updated
Even a correct device-algorithm output may be misinterpreted or trusted inappropriately. Interaction between people and software is a different research question from independent algorithm performance. Define the intended decision role clearly.
Define the assessment
Specify whether software recommends, prioritises or makes a decision independently. Access to additional information and the ability to reject outputs affect the assessment. A decision-support label does not automatically eliminate overreliance or disregard in actual use.
Worked example
For example, image readers change their initial decisions after viewing software markings. Some changes help and others introduce errors. Rather than reporting only final accuracy, the team examines transitions between initial judgement, software output and final judgement. It investigates the data and user conditions associated with disagreements.
Practical checks
- Distinguish decisions made before and after output exposure through an appropriate design.
- Assess responses to incorrect or uncertain outputs under safe research conditions.
- Consider effects of experience, training and presentation on changes in judgement.
Decision and limitations
The report should explain performance of the user working with the device. Do not describe standalone algorithm performance as the outcome of the human workflow. Separate exploratory observations from established benefit claims, and connect important misunderstandings to design, training or further evidence needs.
Frequently asked questions
- Is high algorithm accuracy sufficient?
- Human interaction also matters when intended use depends on a person’s decision. Assess how correct outputs are understood.
- Is rejecting a recommendation a failure?
- Not automatically. Assess whether rejection is appropriate given available information and the intended decision role.