Hi all, pasted below is a letter I've been sending to editors that ask me to do blinded reviews of (statistics) papers without code & data supplements. I was hoping that this might be useful for some of you as well or inspire you to take similar action. I would also be grateful for feedback and suggestions on how to make this more persuasive. Best, Fabian -------------------------------------------- This submission does not include a code and data supplement that would allow peers to replicate results or investigate details of the algorithm that was used to generate them -- in my opinion, this is not acceptable for a methodological contribution such as this one. At least the code used for the simulation study should be made available to reviewers as early as possible. Could you ask the authors to supply the missing material on my behalf and forward it to me and the other reviewers? I hope you do not feel unduly burdened by my request -- I am convinced that our field has to take the issues surrounding replicability of results, availability of code and data, and code quality much more seriously going forward and I am trying to do my part to change scientific standards in that direction, even if many journals have not yet made the decision to require mandatory code and data supplements. The document available at https://staff.washington.edu/rjl/pubs/topten/topten.pdf summarizes my viewpoint on this very well: Just as we would never publish theoretical papers containing a theorem without a peer-reviewed proof, we should not publish papers claiming methodological advances without a publicly available, peer-reviewed implementation that was used to create the results in the paper, in my opinion. In the absence of such a supplement, I am not willing to review the content of this submission and my automatic recommendation would be to revise & resubmit accompanied by suitable supplementary material.