Connect external Kibana to AWS Elasticsearch service

Hi,

I appreciate OpenDistro for ES and the huge effort put into it.

I was glad to see that AWS Elasticsearch service now includes the security plugin starting Feb 11, so I started a test cluster with fine grain security and everything is really smooth.

Kibana that comes with AWS ES service is working fine, however, I’m trying to connect my own Kibana instance to AWS ES service. I have everything in place, but I don’t know what’s the equivalent for kibanaserver user for AWS ES service?

I tried pulling the security config by calling /_opendistro/_security/api/securityconfig from the working Kibana that comes with AWS ES, and it shows that kibana server_username is AmazonESKibanaServerUser. I couldn’t find a way to get that user’s password to have my external Kibana up and running.

Any thoughts on that?

Thanks

2 Likes

This is the exact configuration I’m trying for. I am able to get the external Kibana to authenticate, and startup with authentication by setting these properties in kibana.yml

(This comes from your Amazon ES management dashboard) Also create a user in your internal user database in the Amazon ES Kibana, under the padlock icon.

  • elasticsearch.username
  • elasticsearch.password

Here’s my setup and goals:

  1. I have a managed Amazon ES cluster up, with its own managed Kibana. It can’t run custom plugins, such as LogTrail, and neither can Amazon ES run XPack components like Logs (local app with real-time log updates)

  2. I want to run my own Kibana (“Kibana2”) on an EC2 instance, and I am, and it authenticates. When I click ANYWHERE around in the app after its is up and green, I get:

Blockquote
error [15:20:49.668] Error: Authentication Exception

at respond (/home/ec2-user/work/kibana-7.4.2-linux-x86_64/node_modules/elasticsearch/src/lib/transport.js:349:15)

at checkRespForFailure (/home/ec2-user/work/kibana-7.4.2-linux-x86_64/node_modules/elasticsearch/src/lib/transport.js:306:7)

at HttpConnector.<anonymous> (/home/ec2-user/work/kibana-7.4.2-linux-x86_64/node_modules/elasticsearch/src/lib/connectors/http.js:173:7)

at IncomingMessage.wrapper (/home/ec2-user/work/kibana-7.4.2-linux-x86_64/node_modules/elasticsearch/node_modules/lodash/lodash.js:4929:19)

at IncomingMessage.emit (events.js:194:15)

at endReadableNT (_stream_readable.js:1103:12)

at process._tickCallback (internal/process/next_tick.js:63:19)

Frustrations:

  • It looks like the Kibana2 needs a PEM for SSL communications to the server (kibana.yml server.ssl.certificate)
  • Amazon won’t let you generate an API token (“not allowed”)
  • Amazon DOES let you set a custom key (KMS) for at-rest encryption, but Amazon DOES NOT let you “get into the server and generate a PEM and set it in there” like a random Apache etc…

Thanks in advance for any help on this. It seems like a good pattern, but, I’m wondering if I’m stepping on some “Amazon Only” configuration issue.

G.