There is a chasm between a computational method, an algorithm, its implementation and software. While producing a computational method, furnishing its algorithm, and coding up an implementation is sufficient for the purposes of research, it is insufficient for two reasons in our opinion.

• First, research code is unmaintainable even within the same research group. There are many research groups that develop code for the purposes of completing a single research project, and then placing the line of research on hold. When the time comes to resume the line of research, these groups find that the code they developed is opaque or no longer works in new environments because of insufficient documentation, code style, design and testing. As a result, these groups expend valuable resources redeveloping research code that will face the same problem down the line.
• Second, research code often cannot be used by other research groups and industry partners. Consequently, the research code cannot be built upon by others, nor can its underlying methodology add value to potential applications.

The consequence is that research code, and their underlying methodologies, become a resource sink rather than a multiplier of research effort.

What are we doing about this?

In our group, we aim to go from research code to research software. We view research software as well-documented, well-tested, well-designed code that enables other researchers and industry professionals to meaningfully use and contribute to implementations of the methods that we and others have created. Basically, we implement mature, open-source software engineering practices. We have a few examples that you can explore in the above list. in the list on the left.

Why are you listing experiments?

Numerical experiments are examples of research code and suffer from the same pitfalls as research code that implements a particular algorithm or method. This means that numerical experiments should be well-documented, well-tested, well-designed and enable others to build on them; that is, numerical experiments must follow mature, open-source software engineering practices. Unlike typical software, numerical experiments have additional considerations: they must realize experimental designs and make them reproducible. We think carefully about this and we are constantly evolving our perspective to understand how to design numerical experiments and realize them with mature software engineering practices. We encourage you to explore some of our experiments in the above list. in the list on the left.