The good, the bad, and the spaghetti code: Software put to the test

Computer scientists at HITS and KIT Karlsruhe have developed “SoftWipe”, a largely automated open source tool and benchmark to assess the quality of scientific software in terms of compliance with coding standards. They provide a coding standards´ adherence ranking of 48 computational tools from diverse scientific fields.

Digital tools are essential in almost all scientific disciplines. Reliable, carefully developed software is particularly important when large amounts of research data accumulate and have to be processed quickly in order to analyze the data and interpret it correctly – for example in biology, physics or currently also in epidemiology. However, the development and maintenance of scientific software are often influenced by financial uncertainty and time pressure, which can lead to badly written code . In order to evaluate the software quality automatically, computer scientists from the Heidelberg Institute for Theoretical Studies (HITS) and the Karlsruhe Institute of Technology (KIT) have designed the “SoftWipe” tool. Their results have now been published in the Nature Scientific Reports Journal .

SoftWipe: Fast, reliable, and cost-effective

“In scientific software, compliance with coding standards is rarely taken into account, although this can even lead to incorrect scientific results,” says Alexandros Stamatakis, head of the Computational Molecular Evolution (CME) research group at HITS and professor at the Institute for Theoretical Computer Science (ITI) of the KIT. The open source tool “SoftWipe” offers a fast, reliable and cost-effective solution by automatically evaluating the adherence to software development standards. In addition to the tool itself, the researchers deployed their tool to rank 48 scientific software programs from various research areas to assess their quality.

“SoftWipe can also be used in the review process for scientific software papers and support this  process,” adds Adrian Zapletal. The master’s student and his fellow student Dimitri Höhler made a significant contribution to the development of “SoftWipe”. The evaluation criteria were based on existing standards used in safety-critical areas, such as at NASA and CERN.

“We found substantial differences in software quality,” says co-author Carsten Sinz from the ITI at KIT. Some software models had a low score and performed poorly in the ranking. In their paper, the authors recommend that tools such as SoftWipe should be used as standard in the selection and assessment process for scientific software.

SoftWipe – tool and benchmark for assessing coding standards´ adherence

SoftWipe is a pipeline, written in Python3, that predominantly uses freely available static and dynamic code analyzers to assess the code quality of software written in C/C++. SoftWipe initially compiles the software and subsequently executes the software to detect potential errors. Using the output of the static code analyzers, SoftWipe computes a score for each static analyzer and then outputs an overall score.

Original publication:

Zapletal, A., Höhler, D., Sinz, C. et al.: The SoftWipe tool and benchmark for assessing coding standards adherence of scientific software. Sci Rep 11, 10015 (2021).
https://doi.org/10.1038/s41598-021-89495-8

Press contact:

HITS
Dr. Peter Saueressig
Head of Communications
HITS Heidelberg Institute for Theoretical Studies
Tel: +49-6221-533-245
peter.saueressig@h-its.org

KIT
Dr. Felix Mescoli
Strategische Entwicklung und Kommunikation (SEK)
Karlsruhe Institute of Technology (KIT)
Tel.: +49 721 608-41171
E-Mail: felix.mescoli@kit.edu

Scientific Contact:
Prof. Dr. Alexandros Stamatakis 
Group Leader
Computational Molecular Evolution (CME)
HITS Heidelberg Institute for Theoretical Studies
alexandros.stamatakis@h-its.org