- Published
- Updated
A note that a participant failed does not explain what happened during a device-use test. Record behaviour, assistance and task outcome separately to distinguish design problems, unclear instructions and effects introduced by the test arrangement.
Define the assessment
Define the task start, expected actions, completion and failure situations in a coding dictionary. Do not infer a user’s intention from an observation alone; a separate post-task interview may help investigate it. Illustrate what behaviours justify labels such as struggled, and test agreement between assessors.
Worked example
For example, a user attempts measurement without switching the device on. One observer records noncompliance, while another records difficulty finding the switch. Video and interview review show that the user interpreted a light as the power indicator. The code retains the observed sequence and separates the assessment of possible causes.
Practical checks
- Use distinct observation codes for assistance, repeated attempts and spontaneous correction.
- Give two assessors the same example recordings and discuss disagreements in their classifications.
- Assess comparability and the need for rereview when the coding dictionary changes.
Decision and limitations
The classification is useful when another reviewer can trace it back to the same behaviour. Do not present the number of use errors as the sole measure of device safety. The affected task and potential consequences are different dimensions from frequency and deserve separate interpretation.
Frequently asked questions
- Is every hesitation a use error?
- No. Assess the defined task, observed action and potential impact together; hesitation can still be an informative observation.
- Can interviews replace observation?
- A user’s explanation adds context, but the independent record of observed steps should be retained.