Frustration when authors dodge criticism

Markus Löcher writes:

I had to think of our discussions on the issue of (not) reproducible research which seems to be a real problem, when I carefully examined the code for this paper, “Stacked regressions and structured variance partitioning for interpretable brain maps,”
published by authors from the machine learning power house CMU. I quickly discovered a major problem when I noticed that instead of the claimed partial R2 the code computed the marginal R2, which leads to completely different conclusions. I communicated this to the authors last May and it took them until now finally contact the journal and suggest a small correction, which in my eyes is far from sufficient. I wrote up my main critique here.

I haven’t read the above paper or critique in detail, so I’m just sharing the story, not trying to make a judgment.

P.S. When following the above link to Löcher’s blog, I noticed this other post about “peeking in AB tests,” so I thought I’d to my post explaining why I think sequential data testing is ok, which is related to my view that type 1 error rate control is not a good thing.

P.P.S. Löcher informs us that the correction has finally been posted.