Why Release Your Source Code?

(and an example)

Science is all about reproducibility.  Twenty to thirty years ago, this usually meant reproducing the experimental results of another group, or verifying a series of calculations from a paper.  With the rapid expansion of computing power, and the push to ever more sophisticated models, these days it is extremely rare to find a paper that does not include some form of computer calculation/simulation.  Yet at the same time, the ability to reproduce simulation results is hampered by the fact that almost no one releases the computer code for their papers. When looking at a simulation plot from one of these papers, you cannot be sure whether the answer presented is in fact valid.  Perhaps they missed a factor of 2, maybe worse? Moreover, in some fields such as cosmology, the output of computer programs provides the only incite into physical processes that are out of reach to experimentalist, thus making the validation of the underlying computer codes a top priority.  However, regardless of the field, the release of numerical codes, along with analytical results, is needed for complete transparency and validation of published research.  Recently, this issue has started to receive some much needed attention.  The latest article came out at the beginning of this year:

"The case for open computer programs", Nature 482, 485 (2012)

highlighting the importance of transparent code, and giving some interesting counter examples that resulted in a number of errors.  Some other related articles can be found here:

"Shining Light into Black Boxes", Science 336, 159 (2012)

"Reproducible Research in Computational Science", Science 334, 1226 (2011)

In 2010 we started QuTiP with the goal of being a completely open-source, numerical simulation package for quantum mechanics research and education.  Being based on the Python programming language and the SciPy and NumPy packages means that the user has access to every single line of code used in the simulation. Contrast this to the quantum optics toolbox where, although the toolbox itself is open-source, the Matlab code on which it runs is surely not.  The propriety nature of the algorithms used in commercial software provides an additional barrier to scientific reproducibility (not to mention they cost some serious money).

Example: (updated 28/06/12)

We fully encourage QuTiP users to help further the reproducibility of computational science and release their source code in their publications, or possibly submit it to us as a demo.  Here we highlight one such example kindly given to us by Catherine Holloway at the IQC in Waterloo.  From the authors own mouth:

----

This code allows us to compare simulations of QKD to experimental
results, for example, we have done this in these papers:

http://arxiv.org/abs/1109.2519
http://arxiv.org/abs/1007.4495

In addition, this allows us to test how long QKD with entanglement can
go, for example, between the ground and a satellite, with different
configurations of detectors, and for telecommunications optical fiber
links.

We are currently using this simulation to develop a model for the
optimal squeezing parameter for any channel efficiency and background
noise, and the resulting secure key rate and two-fold coincidence
rates for this optimal squeezing parameter. In many real-world
implementations, such as a satellite link, the background count and
loss fluctuate randomly over the course of a link, and being able to
re-adjust the power level of the source (and thus the squeezing
parameter) in real-time based on the current values of efficiency and
background would be a great boon.

----

The QuTiP file is somewhat lengthy so we have attached both html and Python versions as separate links:

simulate_ent_qkd.html

simulate_ent_qkd.py

The output from this example, giving the key and bit error rates as a function of loss is given below.