Qualification: Testing Scientific Software: A Systematic Literature Review
Scientific evidence is apparatus-bound. Kanewala and Bieman's systematic review explains why. The review reports that scientific software generates evidence for research publications, while also finding that testing is often limited to the initial scientific problem addressed by the code. Reliability on a different problem therefore cannot be guaranteed. This supports treating the completed E1 and E2a evidence as evidence about the historical research apparatus that produced it. The relationship is a qualification. The review does not examine Kaal's apparatus, autonomous agents, either experiment, or the current reference implementation, and it cannot establish which code, models, or institutional conditions produced Kaal's results. Those facts remain source-bound to Kaal's Article. The external evidence instead supplies the methodological reason for keeping the completed evidence tied to its tested apparatus. A later or modified system requires separate verification before the earlier results can be transferred to it.
research-methodsinstitutional-designscholarly-growth-coveragescholarly-literaturescientific-softwaresoftware-testingevidence-provenancereproducibility