[LMU-OSC News] Letter for requesting missing code & data supplements as a reviewer

[LMU Open Science Center News]

Hi all,

pasted below is a letter I've been sending to editors that ask me to do
blinded reviews of (statistics) papers without code & data supplements.
I was hoping that this might be useful for some of you as well or inspire
you to take similar action. I would also be grateful for feedback and
suggestions on how to make this more persuasive.

Best,
Fabian

--------------------------------------------

This submission does not include a code and data supplement that would
allow peers to replicate results or investigate details of the algorithm
that was used to generate them -- in my opinion, this is not acceptable for
a methodological contribution such as this one.
At least the code used for the simulation study should be made available to
reviewers as early as possible. Could you ask the authors to supply the
missing material on my behalf and forward it to me and the other reviewers?

I hope you do not feel unduly burdened by my request -- I am convinced that
our field has to take the issues surrounding replicability of results,
availability of code and data, and code quality much more seriously going
forward and I am trying to do my part to change scientific standards in
that direction, even if many journals have not yet made the decision to
require mandatory code and data supplements.

The document available at
https://staff.washington.edu/rjl/pubs/topten/topten.pdf summarizes my
viewpoint on this very well: Just as we would never publish theoretical
papers containing a theorem without a peer-reviewed proof, we should not
publish papers claiming methodological advances without a publicly
available, peer-reviewed implementation that was used to create the results
in the paper, in my opinion. In the absence of such a supplement, I am not
willing to review the content of this submission and my automatic
recommendation would be to revise & resubmit accompanied by suitable
supplementary material.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.lrz.de/pipermail/lmu-osc/attachments/20200729/0a06b2ea/attachment.html>