(first posted at https://twitter.com/krzhang/status/1245288938664644608; which is why it has Twitter vibes and links to tweets. Slightly edited to flow better with medium, then some items are appended) tl;dr: Sharp writing. Callous style doesn't cover the fact the paper is written with real heart. I give << 10% that this becomes mainstream in 5 years. As homage to paper, I offer up to $18000 at 9:1 odds that such a system is created with >100 papers. (if not clear, this is a hedge where I *really want to lose* $18000). Other thoughts/criticisms: 1. certain studies take a lot of work to replicate; so those studies withstanding time is less of a proof of their quality. Idea: scale cost/reward with difficulty of work. 2. Chesterton's fence: the existing system actually protects against this regime (hard-to-replicate) where setting up the replication, counterfactuals, etc. is hard. When people spaz over methods instead of results, you actually protect against a partially-correlated set of problems: bad methods almost never create good results. 3. If we think of academia as a roof-of-work blockchain (I am somehow unironically using https://twitter.com/krzhang/status/1234891204585644032…) , the analogy of wasteful "work" is gatekeeping, language, method-checking etc. that comes from peer review. This paper urges moving from proof-of-work to proof-of-stake. 4. Idea: to cover regimes where the FPP can't be efficiently tested (see (1)), it may be good to have a *hybrid* PoW/PoS design. Concretely, you may want to set up a game where people can both challenge the results *OR* attack the methods (as a crowdsourced subgame). 5. Idea (not in original tweets): hard to decide sources of truth, counterfactuals, etc. This idea may need to co-evolve with blockchain + on-chain oracles, to automatically "Settle" things. Unlike blockchain, since this is "real life" we may have to occasionally defer to a "jury" of experts to settle the validity of a challenge which reverts a bit to peer review, however I think this is unescapable as the language needed to seriously put skin in the game is costly. (this is why IRL law is so convoluted)