Kibana fails to authenticate with elasticsearch when security plugin is set up

I’ve set up elasticsearch with the security plugin and generated own certificates.
Kibana also has the security plugin (https://opendistro.github.io/for-elasticsearch-docs/docs/kibana/plugins/#security)

I’ve configured kibana.yml with the certificates like so:

server.name: kibana
server.host: "0"

server.ssl.enabled: true
server.ssl.key: config/certs/esnode-key.pem
server.ssl.certificate: config/certs/esnode.pem

elasticsearch.ssl.certificateAuthorities: config/certs/root-ca.pem
elasticsearch.hosts:
  - https://elasticsearch:9200

But kibana does not manage to start.

Here is its output:

{"type":"error","@timestamp":"2019-06-27T14:37:53Z","tags":["connection","client","error"],"pid":1,"level":"error","error":{"message":"139743954802560:error:1408F09C:SSL routines:ssl3_get_record:http request:../deps/openssl/openssl/ssl/record/ssl3_record.c:242:\n","name":"Error","stack":"Error: 139743954802560:error:1408F09C:SSL routines:ssl3_get_record:http request:../deps/openssl/openssl/ssl/record/ssl3_record.c:242:\n"},"message":"139743954802560:error:1408F09C:SSL routines:ssl3_get_record:http request:../deps/openssl/openssl/ssl/record/ssl3_record.c:242:\n"}
{"type":"log","@timestamp":"2019-06-27T14:37:59Z","tags":["error","security"],"pid":1,"message":"authType is basicauth"}
{"type":"log","@timestamp":"2019-06-27T14:37:59Z","tags":["error","security"],"pid":1,"message":"Basic Auth Has Been Matched"}
{"type":"error","@timestamp":"2019-06-27T14:38:03Z","tags":["connection","client","error"],"pid":1,"level":"error","error":{"message":"139743954802560:error:1408F09C:SSL routines:ssl3_get_record:http request:../deps/openssl/openssl/ssl/record/ssl3_record.c:242:\n","name":"Error","stack":"Error: 139743954802560:error:1408F09C:SSL routines:ssl3_get_record:http request:../deps/openssl/openssl/ssl/record/ssl3_record.c:242:\n"},"message":"139743954802560:error:1408F09C:SSL routines:ssl3_get_record:http request:../deps/openssl/openssl/ssl/record/ssl3_record.c:242:\n"}
{"type":"log","@timestamp":"2019-06-27T14:38:03Z","tags":["status","plugin:elasticsearch@6.7.1","error"],"pid":1,"state":"red","message":"Status changed from yellow to red - Authentication Exception","prevState":"yellow","prevMsg":"Waiting for Elasticsearch"}
{"type":"error","@timestamp":"2019-06-27T14:38:12Z","tags":["connection","client","error"],"pid":1,"level":"error","error":{"message":"139743954802560:error:1408F09C:SSL routines:ssl3_get_record:http request:../deps/openssl/openssl/ssl/record/ssl3_record.c:242:\n","name":"Error","stack":"Error: 139743954802560:error:1408F09C:SSL routines:ssl3_get_record:http request:../deps/openssl/openssl/ssl/record/ssl3_record.c:242:\n"},"message":"139743954802560:error:1408F09C:SSL routines:ssl3_get_record:http request:../deps/openssl/openssl/ssl/record/ssl3_record.c:242:\n"}

I did not find anything in the docs regarding the error “Authentication Exception”, nor does it provide enough information to go on

NOTE: If it’s relevant to the issue, the setup is done on kubernetes, (elasticsearch as a stateful set and kibana as a deployment)

I would appreciate some guidance if you can offer it

Hi @LutraMan, at first glance, I’m only seeing a couple things to check out. Can you try adding elasticsearch.username: kibanaserver and elasticsearch.password: kibanaserver (or whatever you configured in internal_users.yml)? Also, can you make sure you’re running the OSS version of Kibana? We’ve seen a similar issue before with people who were running the default distribution of Kibana, which includes some overlapping, proprietary plugins: https://github.com/opendistro-for-elasticsearch/security-kibana-plugin/issues/23

Here’s the “out of the box” configuration of kibana.yml:

server.name: kibana
server.host: "0"
elasticsearch.hosts: https://localhost:9200
elasticsearch.ssl.verificationMode: none
elasticsearch.username: kibanaserver
elasticsearch.password: kibanaserver
elasticsearch.requestHeadersWhitelist: ["securitytenant","Authorization"]

opendistro_security.multitenancy.enabled: true
opendistro_security.multitenancy.tenants.preferred: ["Private", "Global"]
opendistro_security.readonly_mode.roles: ["kibana_read_only"]