Friday, February 24, 2017

The Invalid Signature Problem

For some time now, I have been dealing with a problem where connections don't work when I try to use a local certificate authority with netty and transport layer security (TLS).  The code for this problem is available on gihub at

    https://github.com/ClarkHobbie/ssltest

When I try and connect, I get the following exception:

io.netty.handler.codec.DecoderException: javax.net.ssl.SSLKeyException: Invalid signature on ECDH server key exchange message.

The complete commands are:

    java -cp target\ssl-test-1.0-SNAPSHOT.jar;netty-all-4.1.6.Final.jar Server

and

    java -cp target\ssl-test-1.0-SNAPSHOT.jar;netty-all-4.1.6.Final.jar Client

I have modified the program to work with "remote CAs" like google, running the program this way, doesn't work (google isn't setup to send messages), but I don't get the invalid signature exception either.

The complete command to run against google is:

    java -cp target\ssl-test-1.0-SNAPSHOT.jar;netty-all-4.1.6.Final.jar Client remote google.com 443

Turning off encryption entirely works, it can be done with the following commands:

    java -cp target\ssl-test-1.0-SNAPSHOT.jar;netty-all-4.1.6.Final.jar Server nossl

and

    java -cp target\ssl-test-1.0-SNAPSHOT.jar;netty-all-4.1.6.Final.jar Client nossl

I have tried a variety of things, none of which work.  If anyone knows of a solution, I'm all ears.  Till then I've posted a question on Stack Overflow at:

    http://stackoverflow.com/questions/42445115/invalid-signature-on-ecdh-server-key-exchange-message

All hail netty!

The complete exception is:

io.netty.handler.codec.DecoderException: javax.net.ssl.SSLKeyException: Invalid signature on ECDH server key exchange message
        at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:442)
        at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:248)
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:373)
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:359)
        at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:351)
        at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1334)
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:373)
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:359)
        at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:926)
        at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:129)
        at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:651)
        at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:574)
        at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:488)
        at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:450)
        at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:873)
        at io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:144)
        at java.lang.Thread.run(Unknown Source)
Caused by: javax.net.ssl.SSLKeyException: Invalid signature on ECDH server key exchange message
        at sun.security.ssl.Handshaker.checkThrown(Unknown Source)
        at sun.security.ssl.SSLEngineImpl.checkTaskThrown(Unknown Source)
        at sun.security.ssl.SSLEngineImpl.readNetRecord(Unknown Source)
        at sun.security.ssl.SSLEngineImpl.unwrap(Unknown Source)
        at javax.net.ssl.SSLEngine.unwrap(Unknown Source)
        at io.netty.handler.ssl.SslHandler.unwrap(SslHandler.java:1097)
        at io.netty.handler.ssl.SslHandler.unwrap(SslHandler.java:968)
        at io.netty.handler.ssl.SslHandler.decode(SslHandler.java:902)
        at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:411)
        ... 16 more
Caused by: javax.net.ssl.SSLKeyException: Invalid signature on ECDH server key exchange message
        at sun.security.ssl.HandshakeMessage$ECDH_ServerKeyExchange.(Unknown Source)
        at sun.security.ssl.ClientHandshaker.processMessage(Unknown Source)
        at sun.security.ssl.Handshaker.processLoop(Unknown Source)
        at sun.security.ssl.Handshaker$1.run(Unknown Source)
        at sun.security.ssl.Handshaker$1.run(Unknown Source)
        at java.security.AccessController.doPrivileged(Native Method)
        at sun.security.ssl.Handshaker$DelegatedTask.run(Unknown Source)
        at io.netty.handler.ssl.SslHandler.runDelegatedTasks(SslHandler.java:1123)
        at io.netty.handler.ssl.SslHandler.unwrap(SslHandler.java:1008)
        ... 18 more

No comments:

Post a Comment