Introducing Miranda: Prospero

OK, if a system with "9 9s of reliability" is unattainable with a sane budget, what is attainable?

The answer was Prospero.

Prospero began life as a "skunkworks" project with Erlang. While its initial goal may have been something else, it was used to make systems which only had 1 or 2 9s of reliability seem like they were up most of the time.

Clients would define a topic and then send HTTP POSTs to Prospero.  Other clients would "subscribe" to these topics and register a URL that Prospero would associate with them.  When a client POSTed Prospero would, in turn, POST to the URL that had been registered.

Prospero did its job pretty well, and Pearson later released it as an open source project with the name of "Subpub."

Chris Chew initially did most of the development.  I did a little bit in the way of XML processing.

Over the course of about 3 years of support, the following limitations were discovered:

• It was difficult to find people with Erlang experience
• Prospero was difficult to modify
• Prospero was limited to one data center
• Mnesia (the database that Prospero used) could become corrupted
• Prospero had difficulty with binary messages

There were other problems like one down subscriber could cripple the system, or that Prospero would stubbornly refuse to send any additional messages until all of the current messages were dealt with, but these were the issues that really stuck with us.