OpenID Connect Authentication Issue

I’m attempting to use OIDC (Keycloak IdP) for user SSO in Kibana. My security config is as follows:

    basic_internal_auth_domain:
        description: "Authenticate via HTTP Basic against internal users database"
        http_enabled: true
        transport_enabled: true
        order: 0
        http_authenticator:
          type: basic
          challenge: false
        authentication_backend:
          type: intern
      openid_auth_domain:
        http_enabled: true
        transport_enabled: true
        order: 1
        http_authenticator:
          type: openid
          challenge: false
          config:
            subject_key: preferred_username
            roles_key: roles
            openid_connect_url: https://domain/auth/realms/myrealm/.well-known/openid-configuration
            enable_ssl: true
            verify_hostnames: false
        authentication_backend:
          type: noop

And my kibana config:

elasticsearch.hosts: https://localhost:9200
elasticsearch.ssl.verificationMode: none
elasticsearch.username: kibanaserver
elasticsearch.password: kibanaserver
elasticsearch.requestHeadersWhitelist: ["securitytenant","Authorization", "security_tenant"]

opendistro_security.auth.type: "openid"
opendistro_security.openid.connect_url: "https://domain/auth/realms/myrealm/.well-known/openid-configuration"
opendistro_security.openid.client_id: "elasticsearch"
opendistro_security.openid.client_secret: "secret"
opendistro_security.openid.base_redirect_url: "https://my.tld"

opendistro_security.cookie.secure: true
opendistro_security.cookie.password: "redacted"

opendistro_security.multitenancy.enabled: true
opendistro_security.multitenancy.tenants.preferred: ["Private", "Global"]
opendistro_security.readonly_mode.roles: ["kibana_read_only"]

newsfeed.enabled: false
telemetry.optIn: false
telemetry.enabled: false

However, upon loading the kibana dashboard, I get the error:
{"statusCode":500,"error":"Internal Server Error","message":"An internal server error occurred."}.

The corresponding Kibana log entry:

Jul 23 19:10:11 ubuntu1804 kibana[1445]: {"type":"log","@timestamp":"2020-07-23T19:10:11Z","tags":["error","http"],"pid":1445,"message":"{ Error: Authentication Exception\n    at respond (/usr/share/kibana/node_modules/elasticsearch/src/lib/transport.js:349:15)\n    at checkRespForFailure (/usr/share/kibana/node_modules/elasticsearch/src/lib/transport.js:306:7)\n    at HttpConnector.<anonymous> (/usr/share/kibana/node_modules/elasticsearch/src/lib/connectors/http.js:173:7)\n    at IncomingMessage.wrapper (/usr/share/kibana/node_modules/elasticsearch/node_modules/lodash/lodash.js:4929:19)\n    at IncomingMessage.emit (events.js:203:15)\n    at endReadableNT (_stream_readable.js:1145:12)\n    at process._tickCallback (internal/process/next_tick.js:63:19)\n  status: 401,\n  displayName: 'AuthenticationException',\n  message: 'Authentication Exception',\n  path: '/.kibana/_doc/config%3A7.8.0',\n  query: {},\n  body: 'Authentication finally failed',\n  statusCode: 401,\n  response: 'Authentication finally failed',\n  toString: [Function],\n  toJSON: [Function],\n  isBoom: true,\n  isServer: false,\n  data: null,\n  output:\n   { statusCode: 401,\n     payload:\n      { statusCode: 401,\n        error: 'Unauthorized',\n        message: 'Authentication Exception' },\n     headers:\n      { 'WWW-Authenticate': 'Basic realm=\"Authorization Required\"' } },\n  reformat: [Function],\n  [Symbol(ElasticsearchError)]: 'Elasticsearch/notAuthorized',\n  [Symbol(SavedObjectsClientErrorCode)]: 'SavedObjectsClient/notAuthorized' }"}
Jul 23 19:10:11 ubuntu1804 kibana[1445]: {"type":"error","@timestamp":"2020-07-23T19:10:11Z","tags":[],"pid":1445,"level":"error","error":{"message":"Internal Server Error","name":"Error","stack":"Error: Internal Server Error\n    at HapiResponseAdapter.toInternalError (/usr/share/kibana/src/core/server/http/router/response_adapter.js:69:19)\n    at Router.handle (/usr/share/kibana/src/core/server/http/router/router.js:163:34)\n    at process._tickCallback (internal/process/next_tick.js:68:7)"},"url":{"protocol":null,"slashes":null,"auth":null,"host":null,"port":null,"hostname":null,"hash":null,"search":null,"query":{},"pathname":"/","path":"/","href":"/"},"message":"Internal Server Error"}

And finally, the log for elastic:

[2020-07-23T20:00:37,915][TRACE][c.a.o.s.h.XFFResolver    ] [ubuntu1804] no xff done (enabled or no netty request) false,class org.elasticsearch.rest.RestRequest,{},{}
[2020-07-23T20:00:37,915][TRACE][c.a.o.s.a.BackendRegistry] [ubuntu1804] Rest authentication request from 127.0.0.1:57104 [original: /127.0.0.1:57104]
[2020-07-23T20:00:37,915][DEBUG][c.a.o.s.a.BackendRegistry] [ubuntu1804] Check authdomain for rest internal/0 or 2 in total
[2020-07-23T20:00:37,915][TRACE][c.a.o.s.a.BackendRegistry] [ubuntu1804] Try to extract auth creds from basic http authenticator
[2020-07-23T20:00:37,915][TRACE][c.a.o.s.a.BackendRegistry] [ubuntu1804] No 'Authorization' header, send 403
[2020-07-23T20:00:37,915][DEBUG][c.a.o.s.a.BackendRegistry] [ubuntu1804] Check authdomain for rest noop/1 or 2 in total
[2020-07-23T20:00:37,915][TRACE][c.a.o.s.a.BackendRegistry] [ubuntu1804] Try to extract auth creds from jwt-key-by-oidc http authenticator
[2020-07-23T20:00:37,915][TRACE][c.a.o.s.a.BackendRegistry] [ubuntu1804] No 'Authorization' header, send 403
[2020-07-23T20:00:37,915][DEBUG][c.a.o.s.a.BackendRegistry] [ubuntu1804] User still not authenticated after checking 2 auth domains
[2020-07-23T20:00:37,915][WARN ][c.a.o.s.a.BackendRegistry] [ubuntu1804] Authentication finally failed for null from 127.0.0.1:57104
[2020-07-23T20:00:37,915][TRACE][c.a.o.s.a.i.AuditLogImpl ] [ubuntu1804] Check for REST category:FAILED_LOGIN, effectiveUser:null, request:/.kibana/_doc/config:7.8.0

Any help would be appreciated - thank you!

Have you tried this https://docs.search-guard.com/latest/troubleshooting-openid

I’m not using Search Guard, but I did try all of Open Distro’s OIDC debugging steps.

I have the exact same problem.

In my case the only diference is i dont have any elasticsearch logs.
Are you using the Helm chart? and how did you turned the logs on?

I’m not sure what a Helm chart is so I guess not. Logging configuration is in /etc/elasticsearch/log4j2.properties. I ended up not using Open Distro and using https://github.com/novomatic-tech/keycloak-kibana instead.

Anyone using helm charts that can provide the OIDC configuration that is being passed on the values.yaml file ?
Much appreciated

I am using the same configurations that were posted on this post, but they dont work, even if they are the ones on the documentation

I resolved my problem, It was being cause by using persistance in the cloud. For some reason elk was selectively choosing some settings and removing others.

Without persistance it runs all configurations i send him and was able to connect to ldap.