Issue with Kibana -> ElasticSearch / TLSv1.3 " Protocol TLSv1.3 is not supported."

Security
1

Hi

After updating from 0.8.0 to 0.9.0, kibana cannot connect to ES because auf TLS:

[2019-05-07T15:09:59,900][WARN ][i.n.c.ChannelInitializer ] [es-kibana4] Failed to initialize a channel. Closing: [id: 0x19b22a3b, L:/127.0.0.1:9200 - R:/127.0.0.1:60814]
java.lang.IllegalArgumentException: Protocol TLSv1.3 is not supported.
        at io.netty.handler.ssl.ReferenceCountedOpenSslEngine.setEnabledProtocols(ReferenceCountedOpenSslEngine.java:1516) ~[netty-handler-4.1.32.Final.jar:4.1.32.Final]
        at com.amazon.opendistroforelasticsearch.security.ssl.DefaultOpenDistroSecurityKeyStore.createHTTPSSLEngine(DefaultOpenDistroSecurityKeyStore.java:525) ~[opendistro_security_ssl-0.9.0.0.jar:0.9.0.0]
        at com.amazon.opendistroforelasticsearch.security.ssl.http.netty.OpenDistroSecuritySSLNettyHttpServerTransport$SSLHttpChannelHandler.initChannel(OpenDistroSecuritySSLNettyHttpServerTransport.java:115) ~[opendistro_security_ssl-0.9.0.0.jar:0.9.0.0]
        at io.netty.channel.ChannelInitializer.initChannel(ChannelInitializer.java:115) [netty-transport-4.1.32.Final.jar:4.1.32.Final]
        at io.netty.channel.ChannelInitializer.handlerAdded(ChannelInitializer.java:107) [netty-transport-4.1.32.Final.jar:4.1.32.Final]
        at io.netty.channel.DefaultChannelPipeline.callHandlerAdded0(DefaultChannelPipeline.java:637) [netty-transport-4.1.32.Final.jar:4.1.32.Final]
        at io.netty.channel.DefaultChannelPipeline.access$000(DefaultChannelPipeline.java:46) [netty-transport-4.1.32.Final.jar:4.1.32.Final]
        at io.netty.channel.DefaultChannelPipeline$PendingHandlerAddedTask.execute(DefaultChannelPipeline.java:1487) [netty-transport-4.1.32.Final.jar:4.1.32.Final]
        at io.netty.channel.DefaultChannelPipeline.callHandlerAddedForAllHandlers(DefaultChannelPipeline.java:1161) [netty-transport-4.1.32.Final.jar:4.1.32.Final]
        at io.netty.channel.DefaultChannelPipeline.invokeHandlerAddedIfNeeded(DefaultChannelPipeline.java:686) [netty-transport-4.1.32.Final.jar:4.1.32.Final]
        at io.netty.channel.AbstractChannel$AbstractUnsafe.register0(AbstractChannel.java:514) [netty-transport-4.1.32.Final.jar:4.1.32.Final]
        at io.netty.channel.AbstractChannel$AbstractUnsafe.access$200(AbstractChannel.java:427) [netty-transport-4.1.32.Final.jar:4.1.32.Final]
        at io.netty.channel.AbstractChannel$AbstractUnsafe$1.run(AbstractChannel.java:486) [netty-transport-4.1.32.Final.jar:4.1.32.Final]
        at io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:163) [netty-common-4.1.32.Final.jar:4.1.32.Final]
        at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:404) [netty-common-4.1.32.Final.jar:4.1.32.Final]
        at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:474) [netty-transport-4.1.32.Final.jar:4.1.32.Final]
        at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:909) [netty-common-4.1.32.Final.jar:4.1.32.Final]
        at java.lang.Thread.run(Thread.java:834) [?:?]

Here is some more TLS info:

    2019-05-07T14:55:18,397][INFO ][c.a.o.s.s.DefaultOpenDistroSecurityKeyStore] [es-kibana4] JVM supports TLSv1.3
    [2019-05-07T14:55:19,373][INFO ][c.a.o.s.s.DefaultOpenDistroSecurityKeyStore] [es-kibana4] TLS Transport Client Provider : OPENSSL
    [2019-05-07T14:55:19,374][INFO ][c.a.o.s.s.DefaultOpenDistroSecurityKeyStore] [es-kibana4] TLS Transport Server Provider : OPENSSL
    [2019-05-07T14:55:19,374][INFO ][c.a.o.s.s.DefaultOpenDistroSecurityKeyStore] [es-kibana4] TLS HTTP Provider             : OPENSSL
    [2019-05-07T14:55:19,374][INFO ][c.a.o.s.s.DefaultOpenDistroSecurityKeyStore] [es-kibana4] Enabled TLS protocols for transport layer : [TLSv1.2, TLSv1.1]
    [2019-05-07T14:55:19,374][INFO ][c.a.o.s.s.DefaultOpenDistroSecurityKeyStore] [es-kibana4] Enabled TLS protocols for HTTP layer      : [TLSv1.3, TLSv1.2, TLSv1.1]

I have tried explicitly enabling TLSv1.3 for http in elasticsearch.yml:

opendistro_security.ssl.http.enabled_protocols:
  - "TLSv1.1"
  - "TLSv1.2"
  - "TLSv1.3"

I am using openssl by installing package “apr” and placing netty-tcnative to the opendistro plugin directory:

remote_file '/usr/share/elasticsearch/plugins/opendistro_security/netty-tcnative-2.0.25.Final-linux-x86_64-fedora.jar' do
    source 'https://repo1.maven.org/maven2/io/netty/netty-tcnative/2.0.25.Final/netty-tcnative-2.0.25.Final-linux-x86_64-fedora.jar'
    owner 'root'
    group 'root'
    mode '0644'
    action :create
end

I’m not aware of any settings on Kibana’s end to disable TLS v.1.3
I’m using OpenJDK 11.0.3 if that is of any concern.

Any help how to get kibana properly connected to ES would be appreciated.

Thanks
Michel

2

Weirdly enough, the “opposite” helped. I.e. specifying TLSv1.1 and TLSv1.2 as opendistro_security.ssl.http.enabled_protocols

3

Thanks @MichelZ for reporting this.

Can you please verify what OpenSSL version are you using?

OpenSSL 1.1.1 onward supports TLS 1.3. And all before version only supports till TLSv1.2.
That means this line is a problem deprecated-security-ssl/DefaultOpenDistroSecurityKeyStore.java at 4df028603529df9def93e862116ee0f3c6e5dabb · opendistro-for-elasticsearch/deprecated-security-ssl · GitHub

I have created an issue on github to fix this: Protocol TLSv1.3 is not supported for OpenDistro 0.9 · Issue #415 · opensearch-project/security · GitHub

4

Created a PR to fix this issue: Fixing unsupported TLSv1.3 exception. by hardik-k-shah · Pull Request #10 · opendistro-for-elasticsearch/deprecated-security-ssl · GitHub

5

Thanks Hardik

That would be OpenSSL 1.0.2k-fips