- Published
- Updated
A questionnaire scoring algorithm can change while participant answers remain unchanged. If only the numeric score is retained, reviewers may be unable to explain the rule used or discrepancies between reports generated at different times.
Preserve the meaning of data
Link source responses, missing-item handling, reverse-coded items and the scoring version. Software teams can implement a calculation correctly while scientific suitability remains a separate expert judgement. Explicitly decide whether historical scores will be recalculated after an update. Avoid combining results generated under different definitions as though their meaning were automatically identical.
Worked example
For example, an algorithm changes its handling of one missing item. Records previously producing no score now receive numeric results. The team compares examples with missingness in different positions against independently expected calculations. It documents where revised scores enter reports and whether the rule applies retrospectively. A functioning application screen is not sufficient acceptance evidence.
Review checks
- Use small examples targeting reverse coding and missing items.
- Trace each score to source responses and algorithm version.
- Distinguish recalculated results from those produced under earlier rules.
Acceptance and boundaries
Acceptance means the score definition and its inputs can be reconstructed. Technical correctness alone does not establish clinical validity of a measure. Assess the suitability and conditions of use of authoritative scoring materials separately. Calling a version change a bug fix does not remove the need to consider scientific and analysis consequences for affected study outputs.
Frequently asked questions
- Should identical answers produce an identical score?
- Under the same accepted rule, yes; if the rule changes, explain the difference and its intended use.
- Can an earlier score be overwritten?
- Preserve the recalculation rationale and version relationship rather than allowing the previous result to disappear without explanation.