DSHR's talk at Stanford "Designing for Authenticity" class

DSHR Blog，David Rosenthal 的博客，关于数字保存与分布式系统的深度思考。DSHR 博客。关注
DSHR's talk at Stanford "Designing for Authenticity" class 图片 1

Via John Markoff, I was invited to a conversation with Jonathan Dotan and the students of his EE292J course entitled Designing for Authenticity. Below the fold are my brief introductory remarks, and some notes for the discussion.

Thank you for inviting a relic from earlier days in the Valley. As with all my talks, the text of this brief introduction will go up at blog.dshr.org later this afternoon.

Sun GX version 1

When Curtis and Chris quit Sun and started hanging out in the now legendary Denny's with Jen-Hsun Huang, I also quit to become Nvidia's employee #4.

Curtis and I designed UDA (Unified Device Architecture), the way programs talk to Nvidia's chips. More than 30 years later, that is still the way they do it - the best engineering of my career. After 3 years, in the throes of Nvidia's first near-death experience, I had a big argument with Curtis and quit. It turned out that he was right and I was wrong. I immediately did another startup that also IPO-ed and ended up extremely burnt out.

My wife was part of the team at Stanford Library's HighWire Press that pioneered the transition of academic publishing from paper to the Web in 1995. One effect of the transition was that preservation of the academic record went from a side-effect of distribution to being at the whim of the publishers, which made librarians uneasy.

Our idea for fixing this was for libraries to crawl the journals to which they subscribed and keep a copy, as they did on paper. One problem was that the oligopoly publishers had consumed almost all of the libraries' budget. Whatever we did had to be very cheap, and thus not reliable. My idea for making the system reliable was a permissionless, peer-to-peer system in which libraries audited their copies and used inter-library copy to repair damage.

This was the LOCKSS system, for Lots Of Copies Keep Stuff Safe. In some ways it was a success; nearly 28 years later it is still going. In other ways it was a failure; it is now mostly a centralized system controlled by the publishers. The lesson we learn from this is that decentralization is extremely hard because it is an economic not a technical problem.

Twelve years ago in Economies of Scale in Peer-to-Peer Networks I explained the problem. The TL;DR is that the advantages of P2P networks arise from a diverse network of small, roughly equal resource contributors. Economies of scale mean the cost of participating will scale less than linearly. Unless the reward for participating decreases with scale faster than the cost the profit (reward minus cost) will increase with scale, and economics will drive centralization. No-one has found a way to make the reward decrease with scale, let alone faster than the costs.

Decentralized systems necessarily incur coordination costs that centralized systems don't. Here is an example from the BBC of coordination costs from 750 years ago:

Merton College Library

The requirement for three keys is like the requirement for a majority in Byzantine Fault Tolerance or Ethereum. But:

Just a few years after the Archbishop's decree, several books were stored outside the chest for the first time. They were chained to a table in the college, making them available at any time.

It is possible to make decentralized, permissionless systems as, or even more, reliable than centralized systems that use Byzantine Fault Tolerance. But doing so requires much higher levels of replication, and thus cost, and a large performance penalty. Thus in practice permissionless systems will either centralize, or be out-competed by centralized, peermissioned alternatives.

In effect both of these are what happened to LOCKSS:

  • Although the software was free open source, to get cash-strapped libraries to pay for support LOCKSS had to paywall the per-journal configuration files. Thus the system was in practice permissioned.
  • The oligopoly publishers both funded a permissioned version of it, and also supported a cheaper centralized alternative. Both out-competed the somewhat permissionless version of LOCKSS.

Now you have this background, lets have a discussion.

Thoughts that didn't fit

Why did the printed paper system work better?

  • Write-once, durable medium
  • Capex for replication high, per-replica opex low
  • Distribution forced dispersion of replicas
  • Preservation is a side effect of access
  • Inter-library loan & copy is a federated, not a decentralized system
  • Joining the federation is expensive, low churn rate

Are blockchains useful?
Stuart Haber and W. Scott Stornetta patented blockchains in 1991
They are a Merkle tree with only one branch, a great and useful idea

Haber & Stornetta's company Surety time-stamps documents by publishing the hash of the head of the chain of document hashes weekly in the New York Times classified ads. This is a centralized blockchain, and the root of trust is the New York Times and write-once, durable, dispersed media

But the crypto-bros didn't want to trust anyone, let alone the New York Times. Does the seductive idea of combining the concepts of a blockchain and decentralization deliver trustlessness?

SourceBetteridge's Law of Headlines because the answer to Are Blockchains Decentralized? is "No". We now have almost 18 years of experience on which to base this conclusion. One of the most important reasons is software monoculture.

The intensity of the crypto-bros' gaslighting about the virtues of decentralization is made necessary by the fact that among the 7 cryptocurrencies with market cap above $50B, only 3 even claim to be decentralized and they aren't really. The crypto-bros want people to assume that cryptocurrencies have the theoretical advantages of decentralization, while insiders can exploit the absence of these advantages.

Links from the discussion

Jonathan and the students asked good questions. Here are some links to topics that came up:

添加评论
点赞收藏
点踩分享查看原文
评论
?
参与讨论