Okie-dokie, Casey Handmer edition

I’ve been prototyping an automated peer review project. With current models you get human level peer review for about $0.10 of tokens. For $5 you can rederive every result, check every statistical inference, and audit every paper cited in both directions. There are about 100m papers in the scientific corpus. Most are not important. Many are sloppy. For structural reasons it’s essentially impossible to change published work, even to correct a typo. For $500m today you can thoroughly audit every paper ever written. Every p-hacked result, every plagiarist, every fabrication, every citation club, everything. I don’t have $500m so instead I build a tree, focusing on the 1% of papers that carry 99% of the signal, doing a cheap first pass looking for obvious problems. I can find 90% of the nonsense for a few hundred dollars of tokens. Any guesses where the biggest surprises will be found?

Here is the link.

The post Okie-dokie, Casey Handmer edition appeared first on Marginal REVOLUTION.