@grantslatton

I wrote the S3 filesystem In practice 11 nines is an *under*estimate if your threat model is only normal bit rot, but an over estimate if your threat model includes asteroids and wars Erasure coding scales durability really unintuitively You can split a file into 30 shards — 20 identity shards and 10 parity shards — for an amplification factor of 1.5x and this means you have to lose 11 shards without repairing the file to lose the data But you can just as easily split the file into 60 shards with 40 identity shards and 20 parity shards. Same amplification factor, same amount of storage space. But you have to lose 21 shards now. Each additional shard gets you multiplicatively more durability. Double the parity shards, double the number of nines. So the erasure codes used by all the big cloud storage are actually way above 11 nines by the mathematics, but nobody updates their marketing numbers because it would be silly and pointless.

0 reply

0 recast

1 reaction