OpenId Keycloak Docker

[nicholson]

Hi everyone,

I have been trying for days to set up an OpenId opendistro stack together with a keycloak server in a docker-compose. After the login with keycloak is completed, I get a kibana opendistro screen with the message:

Authentication failed - please provide a new token.

This occurs with version 1.9 and earlier of opendistro. With versions 1.10 and 1.11 the observed behavior is an endless loop between keycloak and kibana in the browser, not ending in any kibana screen. I do not get much useful information from the logs. I do manage to run an OpenId opendistro stack directly installed on my physical debian system.

I would be more than interested in exchanging with others on this topic. I especially have two questions. First, is someone able to provide an example of a docker-compose configuration for OpenId with opendistro, or some parts of such a configuration. Second, is it required to create users with some specific roles in Elasticsearch (for instance using the opendistro security REST API) and the same users in Keycloak ? It would be very useful for me to get details on this part too.

Best regards,

Nicholson

[beachjf]

Hi, did you found a solution for the infinite loop ?

I’m experiencing the same issue.

Thanks a lot

[mmamaenko]

For second question: You do not need to create the same user in Elasticsearch. With user in keycloak and subject_key: email in elasticsearch config.yaml file you may add email address to role mapping yaml file and user will have role assigned to it. Also you may have roles_key: roles in config.yaml file and create role in keycloak with same name as in Kibana, assign to user ( in clients config add User Client Roles with name roles and Token Claim Name: roles) and this role will be assigned to user in Kibana too. I will check my docker-compose config files and post them here later

[mmamaenko]

This configuration is working for me: Docker-compose file for Elasticsearch+Kibana+Keycloak with configuration files · GitHub . Before it will start working you need to add admin user to oauth2 realm in keycloak. I used “admin” password and admin@example.com email address. Rest of configuration files are default from odfe docker-compose. Admin user for keycloak: admin/admin. The same is for kibana/elasticsearch: admin/admin. Also remember that kibana has to start after keycloak is ready, otherwise security plugin will fail to initialize. I usually restart kibana after login to keycloak console

[beachjf]

Thank you so much for your answer and time.
I’m using Helm to install the opendistro version on GCP (Kubernetes)

Helm

I’ll try to convert what you proposed from your docker-compose and configuration file in the values.yaml file.

I’ll keep you posted.

Thanks again

[mrkimmy]

Hi, mmamaenko

I’m learning to use Opendistro security for Elasticsearch. I looking sample to work with keycloak. I found your docker-compose file

I tried to use docker-compose file from your GitHub to start Opendistro Elasticsearch Security using Keycloak as OpenID backend.

I follow your instruction to use docker-compose file as below.

1. add admin user with email address admin@example.com to oauth2 realm in keycloak

2. Create admin role in client kibana-sso

3. Add admin role to admin user in step 1

4. Create roles mapper for kibana-sso client
   

   

5. open kibana page then kibana redirect to authentication with Keycloak.

6. Login with user created in step 1. I got an error ERR_TOO_MANY_REDIRECTS

Could suggest what is my mistake to make docker-compose file working.

Best regards,
Maethee

[mrkimmy]

Hi, beachjf

Have you resolve the infinite loop problem. I found save issue with Nicolson’s docker-compose file.

Thank you.

[mmamaenko]

did you try to restart kibana service? Sometimes kibana starts before keycloak is ready and fails to load security plugin… I will spin efk with my files later to see if your error will popup

have docker-compose file working for me…can you post your config files for kibana and elasticsearch plus log from elasticsearch

[mrkimmy]

Hi, mmamaenko

Thank you for you quick kindly reply

I found more information in log. I found error when elastic node talk to keycloak about certificate error. I regenerate all certificates and use your docker-compose file again then the system working as expected.

Thank you so much.

[Kirill]

Hi!
I’m experiencing the same issue with infinity loop
I want to reproduce your docker-compose file
I have a question
Where can i get volumes data1 sysctl?
Can you give me more detailed instructions to start it, please

Thanks

[Kirill]

I founded the solution
in my test enviroment keycloak and opendistro were on the same host

first I regenerated certificates for elasticsearch nodes, admin cert
it describes here

Generate Certificates

i issued new cert to keycloak with the same root-ca and configure nginx to use it
My keycloak behind nginx-reverse proxy
(if you try to configure it and see blank page after loging to keycloak admin console look to
Keycloak blank page behind nginx reverse proxy - Server Fault)

configure kibana log to easy debugging
/etc/kibana/kibana.yml

logging.dest: "/var/log/kibana/kibana.log"
logging.root.level: "debug"

but I got error which i don’t know how to resolve

{"type":"log","@timestamp":"2021-07-08T06:08:14Z","tags":["error","plugins","opendistroSecurityKibana"],"pid":47793,"message":"{ Error: self signed certificate\n    at TLSSocket.onConnectSecure (_tls_wrap.js:1088:34)\n    at TLSSocket.emit (events.js:198:13)\n    at TLSSocket._finishInit (_tls_wrap.js:666:8)\n  code: 'DEPTH_ZERO_SELF_SIGNED_CERT',\n  trace:\n   [ { method: 'GET',\n       url:\n        'https://localhost:8443/auth/realms/kibana/.well-known/openid-configuration' } ],\n  isBoom: true,\n  isServer: true,\n  data: null,\n  output:\n   { statusCode: 502,\n     payload:\n      { message: 'Client request error: self signed certificate',\n        statusCode: 502,\n        error: 'Bad Gateway' },\n     headers: {} } }"}
{"type":"log","@timestamp":"2021-07-08T06:08:14Z","tags":["warning","environment"],"pid":47793,"message":"Detected an unhandled Promise rejection.\nError: Failed when trying to obtain the endpoints from your IdP"}
{"type":"log","@timestamp":"2021-07-08T06:08:14Z","tags":["info","plugins-system"],"pid":47793,"message":"Starting [48] plugins: [opendistroAlertingKibana,usageCollection,telemetryCollectionManager,telemetry,kibanaUsageCollection,securityOss,newsfeed,mapsLegacy,kibanaLegacy,share,legacyExport,embeddable,expressions,data,home,console,apmOss,management,indexPatternManagement,advancedSettings,opendistroSecurityKibana,opendistroIndexManagementKibana,opendistroAnomalyDetectionKibana,opendistroReportsKibana,opendistroQueryWorkbenchKibana,savedObjects,dashboard,opendistroNotebooksKibana,visualizations,visTypeMarkdown,visTypeVega,visTypeTimelion,timelion,visTypeTable,tileMap,regionMap,inputControlVis,opendistroGanttChartKibana,visualize,bfetch,esUiShared,charts,visTypeVislib,visTypeTimeseries,visTypeTagcloud,visTypeMetric,discover,savedObjectsManagement]"}

to workaround this
edit
/usr/share/elasticsearch/plugins/opendistro_security/securityconfig/config.yml

            openid_connect_url: "http://localhost:8080/auth/realms/kibana/.well-known/openid-configuration"
            enable_ssl: false
            verify_hostnames: false

but infinity loop was continued
to relove it change
/etc/kibana/kibana.yml
opendistro_security.cookie.secure: false