Events and Deliveries are controlled by remote policies, but creates/updates and deletes of User, Topics and Subscriptions go through a 2 phase commit.
The node that initiated the operation takes on the role of coordinator
During phase 1, nodes agree that a write is OK and prepare to go through with the operation (commit) or abort the operation. If any node signals that they are not ready then the coordinator should abort the operation.
During phase 2 the coordinator tells the rest of the node to either commit or abort. Assuming all the nodes responded that they were ready during phase 1, the coordinator signals that the nodes should commit.
As a concrete example, suppose an admin wants to add a new user to the system. The node that the admin contacts acts as the coordinator. The admin supplies the required information and the system announces it's intention to add the user to the rest of the nodes. In this case, the other nodes check to see if they already have a user with the same name. Assuming they don't they signal that they are ready to add the user. The coordinator issues a commit and the new user is added to the system.
Showing posts with label Architecture. Show all posts
Showing posts with label Architecture. Show all posts
Saturday, May 6, 2017
Sunday, April 30, 2017
To Divide or not to Divide?
I was thinking of making the part of Miranda that serves web pages separate from the rest of the system.
On the pro side this makes Miranda a bit more secure and reliable. The web side can go down without bringing down the rest of the system. If an attacker gains control of the web side the rest of the system is OK.
On the con side, Miranda will still have to host servlets, and they will probably be hosted by the same software as the web pages (Jetty).
Decisions, decisions.
I will divide Miranda into two parts because it offer more security and flexibility. The Miranda side can use something that is optimized for servlets while the web side can use something optimized for HTML.
On the pro side this makes Miranda a bit more secure and reliable. The web side can go down without bringing down the rest of the system. If an attacker gains control of the web side the rest of the system is OK.
On the con side, Miranda will still have to host servlets, and they will probably be hosted by the same software as the web pages (Jetty).
Decisions, decisions.
I will divide Miranda into two parts because it offer more security and flexibility. The Miranda side can use something that is optimized for servlets while the web side can use something optimized for HTML.
Saturday, March 4, 2017
State Classes: Where the Brains are in the Miranda System
I decided to use external states in the Miranda system. Because Miranda is a multi-threaded system this is what I call a Big Deal.
What this means, in practical terms, is that each thread reacts differently depending on what state it's in. For example, when a node is waiting for a "join" message, it reacts differently to a "join" message then when it is trying to connect.
What this boils down to is that, most of the behavior logic goes in the state classes and what I normally think of as the class itself does very little.
Time will tell if this was a good move or A Very Bad Idea.
What this means, in practical terms, is that each thread reacts differently depending on what state it's in. For example, when a node is waiting for a "join" message, it reacts differently to a "join" message then when it is trying to connect.
What this boils down to is that, most of the behavior logic goes in the state classes and what I normally think of as the class itself does very little.
Time will tell if this was a good move or A Very Bad Idea.
Friday, March 3, 2017
Exceptions and Panics
Miranda should never stop. Ever.
This means my blithe strategy of printing a stack trace and calling System.exit in response to an exception cannot hold. Instead, most threads must continue and instead throw a Panic when things get really bad.
A Panic is like an exception, except that the system treats a panic as a request to shut down instead of automatically printing a stack trace and exiting.
I have also created a panic method on the Miranda class. If the method returns at all (the default behavior is to call System.exit but this will change) it returns a boolean to indicate if the system as a whole is panicking, or if the caller should try and keep going. The system may attempt to shutdown instead of calling System.exit so correctly handling the response to a call to Miranda.panic is important.
This means my blithe strategy of printing a stack trace and calling System.exit in response to an exception cannot hold. Instead, most threads must continue and instead throw a Panic when things get really bad.
A Panic is like an exception, except that the system treats a panic as a request to shut down instead of automatically printing a stack trace and exiting.
I have also created a panic method on the Miranda class. If the method returns at all (the default behavior is to call System.exit but this will change) it returns a boolean to indicate if the system as a whole is panicking, or if the caller should try and keep going. The system may attempt to shutdown instead of calling System.exit so correctly handling the response to a call to Miranda.panic is important.
Thursday, March 2, 2017
Getting a Handle on Things
Given that
- netty TLS is giving me grief*
- Socket based SSl/TLS seems to work
I have decided the following:
- All network communication (send/receive) will go through the network object
- All network communications will use a handle: nothing will use the network directly
- For sockets, all receives will use their own thread
I'm no too crazy about the last point, and in particular I don't know if you can ask a socket to send a message while you already issued a receive. We shall see.
The good news is that this should make testing easier: just call a method and see what messages end up in the various queues.
* = After looking at the state of non-blocking, secure (i.e. SSL/TLS) I/O in Java, I changed my opinion of netty; but it still may have bugs.
Friday, February 24, 2017
Miranda and Encryption
Why does Miranda want to use a certificate authority? Why does Miranda use encryption at all?
Briefly, Miranda uses local certificate authorities to make it cheaper to use and easier to evaluate. Miranda uses encryption because events (messages) may contain things like personally identifiable information or other sensitive information.
The long answers are, well, longer.
First of all, if you don't have a requirement to encrypt things, then you can turn encryption off. Miranda was designed to be used on things like Amazon cloud, however, with traffic potentially going across the internet, so your events (messages) could be sent in the clear. If you are comfortable with that arrangement, then you can simply turn off encryption.
Miranda uses local certificate authorities because all nodes in the cluster are required to have certificates. It can quickly get expensive creating CERTs for every node in your cluster, not to mention inconvenient, with that approach. Instead, you create your own certificate authority and use the local CA to sign all your node keys.
Miranda uses encryption because I found myself in situations where I wished that its predecessor, Prospero, did. In particular, one of the obstacles to using Prospero in AWS was its lack of support for encryption. Another problem was crossing availability zones. If we had a node on the West coast, and another on the East coast, then they would probably talk across the internet.
As far as what to use, I thought SSL/TLS with their wide use, would be well supported, secure, cheap, and easy to use. While they are indeed well supported, secure and inexpensive I have not found SSL/TLS to be at all easy to use. I have run across a problem that has forced me to do all my development work "in the clear." I refer to the dreaded "Invalid signature" problem that I posted on Stack Overflow about.
At any rate, that is why Miranda uses local certificate authorities and encryption in general.
All hail netty!
Briefly, Miranda uses local certificate authorities to make it cheaper to use and easier to evaluate. Miranda uses encryption because events (messages) may contain things like personally identifiable information or other sensitive information.
The long answers are, well, longer.
First of all, if you don't have a requirement to encrypt things, then you can turn encryption off. Miranda was designed to be used on things like Amazon cloud, however, with traffic potentially going across the internet, so your events (messages) could be sent in the clear. If you are comfortable with that arrangement, then you can simply turn off encryption.
Miranda uses local certificate authorities because all nodes in the cluster are required to have certificates. It can quickly get expensive creating CERTs for every node in your cluster, not to mention inconvenient, with that approach. Instead, you create your own certificate authority and use the local CA to sign all your node keys.
Miranda uses encryption because I found myself in situations where I wished that its predecessor, Prospero, did. In particular, one of the obstacles to using Prospero in AWS was its lack of support for encryption. Another problem was crossing availability zones. If we had a node on the West coast, and another on the East coast, then they would probably talk across the internet.
As far as what to use, I thought SSL/TLS with their wide use, would be well supported, secure, cheap, and easy to use. While they are indeed well supported, secure and inexpensive I have not found SSL/TLS to be at all easy to use. I have run across a problem that has forced me to do all my development work "in the clear." I refer to the dreaded "Invalid signature" problem that I posted on Stack Overflow about.
At any rate, that is why Miranda uses local certificate authorities and encryption in general.
All hail netty!
Wednesday, February 22, 2017
Subsystems and Threads
Miranda uses threads that run state machines and which communicate through BlockingQueues.
Each subsystem has its own thread and runs a Consumer; which basically sits around, waiting for new messages from the thread's BlockingQueue. When the Consumer gets a new message, it hands it off to the Consumer's current state for processing.
There are two advantages to this arrangement. One, the module is inherently multi-threaded, and two, each state only has to worry about the messages relevant to it and not all the other messages that could occur.
When a state receives a message that it does not recognize, it asks the superclass to take a look at it and goes about normal processing. This allows you to factor out common behavior into super states (superclasses of states).
If I'm going to be honest about it, this is kind of the way Erlang does things, except in Erlang each object has its own thread and queue. Also Erlang is a functional language, that is, you can't do things like "i++" in Erlang; wheras Java is an imperative language.
One upside to this approach is that, in theory, testing should be very easy. You just send a message, wait a little, and then see what the subsystem sent in response. I say "in theory" because in practice instead you discover very embarrassing bugs.
I think this was a Good Idea, instead of A Very Bad Idea, but we shall see.
TLS is still not working.
All hail netty!
Each subsystem has its own thread and runs a Consumer; which basically sits around, waiting for new messages from the thread's BlockingQueue. When the Consumer gets a new message, it hands it off to the Consumer's current state for processing.
There are two advantages to this arrangement. One, the module is inherently multi-threaded, and two, each state only has to worry about the messages relevant to it and not all the other messages that could occur.
When a state receives a message that it does not recognize, it asks the superclass to take a look at it and goes about normal processing. This allows you to factor out common behavior into super states (superclasses of states).
If I'm going to be honest about it, this is kind of the way Erlang does things, except in Erlang each object has its own thread and queue. Also Erlang is a functional language, that is, you can't do things like "i++" in Erlang; wheras Java is an imperative language.
One upside to this approach is that, in theory, testing should be very easy. You just send a message, wait a little, and then see what the subsystem sent in response. I say "in theory" because in practice instead you discover very embarrassing bugs.
I think this was a Good Idea, instead of A Very Bad Idea, but we shall see.
TLS is still not working.
All hail netty!
Subscribe to:
Posts (Atom)