- Published
- Updated
When simulation results inform a design decision, the computational process also needs review. A performance table in a presentation is insufficient if reviewers cannot identify which code and assumptions produced the numbers supporting the proposed trial.
Define the question and assumptions
Separate scenario definitions from program logic and record software versions. Connect random-number control, repetition counts and failed runs to reporting. Validation against simple known situations and independent review can reveal defects in complex adaptation logic. Fast execution and plausible-looking summary statistics do not establish computational correctness.
Hypothetical study example
In a hypothetical design, a program counts remaining participants incorrectly after one interim decision. The overall power estimate still looks plausible. The team finds the defect through small manually traceable scenarios and boundary tests rather than treating a large number of simulation repetitions as proof of accuracy.
Design review checks
- Identify the code version, scenario inputs and corresponding output files together.
- Test boundary cases such as zero events and values exactly on thresholds.
- Report failed and nonconvergent repetitions rather than silently removing them.
Record the decision and its limits
Acceptance lets an independent person rerun a selected scenario under the recorded conditions. Explain numerical uncertainty and result tolerances. After a code correction, reassess affected tables and decisions while retaining the earlier output trail. Selecting only a version favourable to the preferred design is not scientific version control or adequate computational validation.
Frequently asked questions
- Is an identical random seed sufficient?
- Not alone. Code, software and scenario versions also matter.
- Do many repetitions correct a programming error?
- No. Repeating faulty logic can still produce an incorrect estimate.