Systems that never stop (and Erlang) Joe Armstrong How can we get 10 nines reliability? SIX LAWS ONE ISOLATION ISOLATION 10 nines = 99.99999999% availability  P(fail) = 10-10  If P(fail | one computer) = 10-3 then  P(fail | four computers) = 10-12 Fixed  TWO CONCURRENCY Concurrency World is concurrent  Need at least TWO computers to make a non-stop  sytem TWO computer is concurrent and distributed  “My first message is that concurrency is best regarded as a program structuring principle” Structured concurrent programming – Tony Hoare Redmond, July 2001 THREE MUST DETECT FAILURES Failure detection If you can’t detect a failure you can’t fix it  Must work across machine boundaries  the entire machine might fail Implies distributed error handling,  no shared state, asynchronous messaging