Friday, February 24, 2017

The Invalid Signature Problem

For some time now, I have been dealing with a problem where connections don't work when I try to use a local certificate authority with netty and transport layer security (TLS).  The code for this problem is available on gihub at

    https://github.com/ClarkHobbie/ssltest

When I try and connect, I get the following exception:

io.netty.handler.codec.DecoderException: javax.net.ssl.SSLKeyException: Invalid signature on ECDH server key exchange message.

The complete commands are:

    java -cp target\ssl-test-1.0-SNAPSHOT.jar;netty-all-4.1.6.Final.jar Server

and

    java -cp target\ssl-test-1.0-SNAPSHOT.jar;netty-all-4.1.6.Final.jar Client

I have modified the program to work with "remote CAs" like google, running the program this way, doesn't work (google isn't setup to send messages), but I don't get the invalid signature exception either.

The complete command to run against google is:

    java -cp target\ssl-test-1.0-SNAPSHOT.jar;netty-all-4.1.6.Final.jar Client remote google.com 443

Turning off encryption entirely works, it can be done with the following commands:

    java -cp target\ssl-test-1.0-SNAPSHOT.jar;netty-all-4.1.6.Final.jar Server nossl

and

    java -cp target\ssl-test-1.0-SNAPSHOT.jar;netty-all-4.1.6.Final.jar Client nossl

I have tried a variety of things, none of which work.  If anyone knows of a solution, I'm all ears.  Till then I've posted a question on Stack Overflow at:

    http://stackoverflow.com/questions/42445115/invalid-signature-on-ecdh-server-key-exchange-message

All hail netty!

The complete exception is:

io.netty.handler.codec.DecoderException: javax.net.ssl.SSLKeyException: Invalid signature on ECDH server key exchange message
        at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:442)
        at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:248)
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:373)
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:359)
        at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:351)
        at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1334)
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:373)
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:359)
        at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:926)
        at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:129)
        at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:651)
        at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:574)
        at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:488)
        at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:450)
        at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:873)
        at io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:144)
        at java.lang.Thread.run(Unknown Source)
Caused by: javax.net.ssl.SSLKeyException: Invalid signature on ECDH server key exchange message
        at sun.security.ssl.Handshaker.checkThrown(Unknown Source)
        at sun.security.ssl.SSLEngineImpl.checkTaskThrown(Unknown Source)
        at sun.security.ssl.SSLEngineImpl.readNetRecord(Unknown Source)
        at sun.security.ssl.SSLEngineImpl.unwrap(Unknown Source)
        at javax.net.ssl.SSLEngine.unwrap(Unknown Source)
        at io.netty.handler.ssl.SslHandler.unwrap(SslHandler.java:1097)
        at io.netty.handler.ssl.SslHandler.unwrap(SslHandler.java:968)
        at io.netty.handler.ssl.SslHandler.decode(SslHandler.java:902)
        at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:411)
        ... 16 more
Caused by: javax.net.ssl.SSLKeyException: Invalid signature on ECDH server key exchange message
        at sun.security.ssl.HandshakeMessage$ECDH_ServerKeyExchange.(Unknown Source)
        at sun.security.ssl.ClientHandshaker.processMessage(Unknown Source)
        at sun.security.ssl.Handshaker.processLoop(Unknown Source)
        at sun.security.ssl.Handshaker$1.run(Unknown Source)
        at sun.security.ssl.Handshaker$1.run(Unknown Source)
        at java.security.AccessController.doPrivileged(Native Method)
        at sun.security.ssl.Handshaker$DelegatedTask.run(Unknown Source)
        at io.netty.handler.ssl.SslHandler.runDelegatedTasks(SslHandler.java:1123)
        at io.netty.handler.ssl.SslHandler.unwrap(SslHandler.java:1008)
        ... 18 more

Miranda and Encryption

Why does Miranda want to use a certificate authority?  Why does Miranda use encryption at all?

Briefly, Miranda uses local certificate authorities to make it cheaper to use and easier to evaluate. Miranda uses encryption because events (messages) may contain things like personally identifiable information or other sensitive information.

The long answers are, well, longer.

First of all, if you don't have a requirement to encrypt things, then you can turn encryption off. Miranda was designed to be used on things like Amazon cloud, however, with traffic potentially going across the internet, so your events (messages) could be sent in the clear.  If you are comfortable with that arrangement, then you can simply turn off encryption.

Miranda uses local certificate authorities because all nodes in the cluster are required to have certificates.  It can quickly get expensive creating CERTs for every node in your cluster, not to mention inconvenient, with that approach.  Instead, you create your own certificate authority and use the local CA to sign all your node keys.

Miranda uses encryption because I found myself in situations where I wished that its predecessor, Prospero, did.  In particular, one of the obstacles to using Prospero in AWS was its lack of support for encryption.  Another problem was crossing availability zones.  If we had a node on the West coast, and another on the East coast, then they would probably talk across the internet.

As far as what to use, I thought SSL/TLS with their wide use, would be well supported, secure, cheap, and easy to use.  While they are indeed well supported, secure and inexpensive I have not found SSL/TLS to be at all easy to use.  I have run across a problem that has forced me to do all my development work "in the clear."  I refer to the dreaded "Invalid signature" problem that I posted on Stack Overflow about.

At any rate, that is why Miranda uses local certificate authorities and encryption in general.

All hail netty!

Thursday, February 23, 2017

On Terminology

It is incredibly important for readers to understand what I say.  Or rather, if anyone read this blog, then it would be.

At any rate, when I refer to something as "netty-like" in its simplicity or "netty-escque" then I want people to understand what I am am talking about.  Imagine, if you will, all the evil and rancor associated with the following:


the Adversary, 
Destroyer of Kings, 
Angel of the Bottomless Pit, 
Great Beast that is called Dragon, 
Prince of This World,
Father of Lies, 
Spawn of Satan, 
and Lord of Darkness *


And refer to the opposite as "netty."  So, when I refer to something as incredibly simple, or "netty-like" remember that the opposite is

the Adversary, 
Destroyer of Kings, 
Angel of the Bottomless Pit, 
Great Beast that is called Dragon, 
Prince of This World,
Father of Lies, 
Spawn of Satan, 
and Lord of Darkness *

and how truly fortunate, we are to have a tool like netty; and how truly blessed I feel as I while away my precious time on it.

TLS still doesn't work.

All hail netty!

* = This is taken, as anything good and original that you find on this blog, from someplace else.  In this case, it is taken from the book Good Omens: The Nice and Accurate Prophecies of Agnes Nutter, Witch, Mass Market Paperback – November 28, 2006 by Niel Gaiman and Terry Pratchett.

Wednesday, February 22, 2017

Subsystems and Threads

Miranda uses threads that run state machines and which communicate through BlockingQueues.

Each subsystem has its own thread and runs a Consumer; which basically sits around, waiting for new messages from the thread's BlockingQueue.  When the Consumer gets a new message, it hands it off to the Consumer's current state for processing.

There are two advantages to this arrangement.  One, the module is inherently multi-threaded, and two, each state only has to worry about the messages relevant to it and not all the other messages that could occur.

When a state receives a message that it does not recognize, it asks the superclass to take a look at it and goes about normal processing.  This allows you to factor out common behavior into super states (superclasses of states).

If I'm going to be honest about it, this is kind of the way Erlang does things, except in Erlang each object has its own thread and queue.  Also Erlang is a functional language, that is, you can't do things like "i++" in Erlang; wheras Java is an imperative language.

One upside to this approach is that, in theory, testing should be very easy.  You just send a message, wait a little, and then see what the subsystem sent in response.  I say "in theory" because in practice instead you discover very embarrassing bugs.

I think this was a Good Idea, instead of A Very Bad Idea, but we shall see.

TLS is still not working.

All hail netty!

Tuesday, February 21, 2017

Miranda Testing

Currently, the strategy is to create a test class for each Miranda class, and put the tests for that class there.

When I say "every Miranda class," I don't actually mean it.  Classes that don't have interesting behavior (or put another way, boring classes) are exempt.

One of the perks of testing, is getting a chance to refactor code.  Classes which seemed like a good idea at the time can be "garbage collected" and removed.

TLS is still not working...

All hail netty!

Monday, February 20, 2017

Testing

Miranda is supposed to be an example of how I would like to go about designing and implementing a system.

This means testing.

Testing and I have a long and contentious history.  The problem is that I'm lazy.  I don't want to go to the extra effort of writing tests.  In addition to writing the test itself, you have to setup preconditions and evaluate post conditions making the effort on par with writing the code that it is trying to test.

Bottom line: tests are as expensive as the code itself.

And if the underlying system changes, the the tests have to change as well; making changes even more expensive.  Thus certain people (read me), want to delay writing tests until the system is "finished" (like that ever happens).

So I don't like writing tests.

But I'm getting to the point where one part of the system is stepping on another part when I try to test some aspect; so off I go to write tests.

Oh the joy.

Looking at the alternatives, I will use JUnit unless I find something better.

I can just taste the excitement.

All hail netty!

Sunday, February 19, 2017

Property Defaults

Some of the Miranda properties and their default values:

Name Default Description
com.ltsllc.miranda.ClusterFle data/custer.json The cluster file.
com.ltsllc.miranda.UsersFile data/users.json The users file.
com.ltsllc.miranda.TopicsFile data/topics.json The topics file.
com.ltsllc.miranda.SubscriptionsFile data/subscriptions.json The subscriptions file.
com.ltsllc.miranda.MessagesDirectory data/messages The directory where messages files are kept.
com.ltsllc.miranda.DeliveriesDirectory data/deliveries The directory where deliveries files are kept.
com.ltsllc.miranda.TopicsPort 443 The port that Miranda listens to for new messages
com.ltsllc.miranda.ClusterPort 6789 The port that Miranda listens to for new cluster nodes.
com.ltsllc.miranda.Log4jFile log4j.xml The log4j configuration file.
com.ltsllc.miranda.MessageFileSize 100 The number of events (messages) in a single event file.
com.ltsllc.miranda.Truststore truststore The file that contains the certificate that Miranda expects everything to be signed with. The file should be in JKS format.
com.ltsllc.miranda.TruststorePassword none The password for the truststore. You must supply a value for this property!
com.ltsllc.miranda.TruststoreAlias ca The alias (name) of the certificate to use from the truststore.
com.ltsllc.miranda.ServerKeystore serverkeystore The filename of the keystore that contains the server key. This key is used when a new node connects or a client posts an event (message). The file should be in JKS format.
com.ltsllc.miranda.ServerKeystorePassword none The password for the server keystore. You must supply a value for this!
com.ltsllc.miranda.ServerKeystoreAlias server The alias (name) of the key to use for the server key.
com.ltsllc.miranda.cluster.HealthCheckPeriod 10000 The time (in milliseconds) to perform a health check on the cluster.
com.ltsllc.miranda.cluster.Timeout 604800000 The time (in milliseconds) to wait before dropping a node.
com.ltsllc.miranda.cluster.HttpPort 443 Which port to wait for HTTP requests on.
com.ltsllc.miranda.cluster.GarbageCollectionPeriod 3600000 (once/hour) The amount of time (in milliseconds) to wait between garbage collections.

I decided to prefix all the properties with "com.ltsllc.miranda" to keep them separate from other properties.

I will add to this as new properties occur to me.