Opensearch and OpenID (Azure) issue

rtarek · August 9, 2021, 10:02pm

Hello,

I have an issue configuring Opensearch authentication with OpenID (Azure). I think the same recommendations could apply to Opendistro for elasticsearch.

I registred an app in Azure AD with a redirect URL as http://localhost:5601/auth/openid/login .

Next I added the following lines in config.yml and executed securityadmin.sh script.

      openid_auth_domain:
        http_enabled: true
        transport_enabled: false
        order: 0
        http_authenticator:
          type: openid
          challenge: false
          config:
            subject_key: preferred_username
            roles_key: roles
            openid_connect_url: https://login.microsoftonline.com/----/v2.0/.well-known/openid-configuration
        authentication_backend:
          type: noop

In opensearch-dashboard.yml, I added the following lines :

opensearch_security.auth.type: "openid"
opensearch_security.openid.connect_url: "https://login.microsoftonline.com/----/v2.0/.well-known/openid-configuration"
opensearch_security.openid.client_id: "------------------------"
opensearch_security.openid.client_secret: "------------"
opensearch_security.openid.scope: "openid"
opensearch_security.openid.base_redirect_url: "http://localhost:5601"

When accessing http://localhost:5601, I’m being redirected to microsoft login portal, but once authenticated I receive the following error :

Beside the error, I could not find where to configure the roles mapping in order to affect privileges for each authenticated user.

Thank you in advance !!

searchymcsearchface · August 10, 2021, 3:40pm

I’m moving this to the security category - I think it fits it better here.

rtarek · August 10, 2021, 8:07pm

Thank you @searchymcsearchface

Anthony · August 10, 2021, 9:56pm

Hi @rtarek
The configuration you provided looks ok and I have just tested the same with opensearch and it works as expected.

Do you see any errors in the logs? Do you have anything running in front of kibana?

rtarek · August 11, 2021, 11:43pm

Hi @Anthony

According to the output console :

opensearch-node2         | [2021-08-11T23:33:11,332][WARN ][c.a.d.a.h.j.AbstractHTTPJwtAuthenticator] [opensearch-node2] Failed to get subject from JWT claims, check if subject_key 'preferred_username' is correct.
opensearch-node2         | [2021-08-11T23:33:11,333][ERROR][c.a.d.a.h.j.AbstractHTTPJwtAuthenticator] [opensearch-node2] No subject found in JWT token
opensearch-node2         | [2021-08-11T23:33:11,333][WARN ][o.o.s.h.HTTPBasicAuthenticator] [opensearch-node2] No 'Basic Authorization' header, send 401 and 'WWW-Authenticate Basic'
opensearch-dashboards    | {"type":"log","@timestamp":"2021-08-11T23:33:11Z","tags":["error","plugins","securityDashboards"],"pid":1,"message":"OpenId authentication failed: Error: Authentication Exception"}

Anthony · August 12, 2021, 8:27am

@rtarek
It seems to be complaining about preferred_username not being present in the JWT.

Could you paste the openid_connect_url in the browser and check if preferred_username is listed under claims_supported.

Also, try changing subject_key in config.yml to just name

rtarek · August 14, 2021, 6:56pm

Hey @Anthony

The “preferred_username” keyword is indeed under the “claims_supported”, and I already tried some couple values from the “claims_supported”, but the result is the same, they are always claiming that the “preferred_username” or whatever is not found in the JWT token … which is very weird …

Thank you very much

Anthony · August 16, 2021, 7:33am

@rtarek Can you try commenting out the line in config.yml:

#subject_key: preferred_username

Ideally would be useful to get the JWT that is being produced, but as you’ve seen from other posts, it doesn’t seem to be possible right now.

rtarek · August 16, 2021, 5:11pm

@Anthony

Opensearch output :

No 'Basic Authorization' header, send 401 and 'WWW-Authenticate Basic'

Opensearch dashboards output :

 {"type":"log","@timestamp":"2021-08-16T17:08:34Z","tags":["error","plugins","securityDashboards"],"pid":1,"message":"OpenId authentication failed: Error: Authentication Exception"}

Anthony · August 16, 2021, 5:19pm

@rtarek can you increase the logging to debug using below:

PUT /_cluster/settings
{“transient”:{“logger._root”:“DEBUG”}}

and DM me the logs from elasticsearch, from the moment you try to connect using OpenID.

rtarek · August 18, 2021, 7:39pm

Hi @Anthony

Same output …

Thank you !!

JackBlack · November 4, 2021, 9:23am

Hi @rtarek , I’m now having the same issue with you before

Opensearch Dashboards output:

{“type”:“log”,“@timestamp”:“2021-11-04T09:12:54Z”,“tags”:[“error”,“plugins”,“securityDashboards”],“pid”:1,“message”:“OpenId authentication failed: Error: Authentication Exception”}

Opensearch output:

{“type”:“log”,“@timestamp”:“2021-11-04T09:13:37Z”,“tags”:[“error”,“plugins”,“securityDashboards”],“pid”:1,“message”:“OpenId authentication failed: Error: Authentication Exception”}

Please let me know how did you fix this issue?

Anthony · November 4, 2021, 10:07am

@JackBlack
Can you provide your config.yml and opensearchDashboard.yml files?

Also, how are you running opensearch, binaries or docker?

JackBlack · November 4, 2021, 10:57am

Hi @Anthony
I’m running both Opensearch and Opensearch dashboards from docker-compose.

config.yml

 type: "config"
  config_version: 2
config:
  dynamic:
    http:
      anonymous_auth_enabled: false
    authc:
      openid_auth_domain:
        http_enabled: true
        transport_enabled: true
        order: 0
        http_authenticator:
          type: openid
          challenge: false
          config:
            subject_key: preferred_username
            roles_key: roles
            openid_connect_url: https://login.microsoftonline.com/---------------------------/v2.0/.well-known/openid-configuration
        authentication_backend:
          type: noop

opensearch_dashboards.yml

server.host: "0"
opensearch.hosts: ["http://localhost:9200"]
opensearch.ssl.verificationMode: none
opensearch.username: "kibanaserver"
opensearch.password: "kibanaserver"
opensearch.requestHeadersWhitelist: [ "Authorization", "security_tenant"]
opensearch_security.multitenancy.enabled: true
opensearch_security.multitenancy.tenants.preferred: ["Private", "Global"]
opensearch_security.readonly_mode.roles: ["kibana_read_only"]
 #Use this setting if you are running opensearch-dashboards without https
opensearch_security.cookie.secure: false
#Config ssl
server.ssl.enabled: true
server.ssl.certificate: /usr/share/opensearch-dashboards/config/client.pem
server.ssl.key: /usr/share/opensearch-dashboards/config/client-key.pem
opensearch.ssl.certificateAuthorities: [ "/usr/share/opensearch-dashboards/config/root-ca.pem"]
#Add for sso by Azure AD
opensearch_security.auth.type: "openid"
opensearch_security.openid.connect_url: "https://login.microsoftonline.com/---------------------------/v2.0/.well-known/openid-configuration"
opensearch_security.openid.client_id: "--------------------------------------"
opensearch_security.openid.client_secret: "-------------------------------------------"
opensearch_security.openid.scope: "openid"
opensearch_security.openid.base_redirect_url: "https://my_ip_addr:5601"

Anthony · November 4, 2021, 2:27pm

@JackBlack Can you provide the full config.yml, as basic_auth is surely enabled if you are using it (“kibanaserver”)

Also sent you DM with more questions

JackBlack · November 5, 2021, 4:18am

Hi @Anthony,
I’m running both of them with docker-compose.

Full config.yml:

---
_meta:
  type: "config"
  config_version: 2

config:
  dynamic:
    http:
      anonymous_auth_enabled: false
    authc:
      openid_auth_domain:
        http_enabled: true
        transport_enabled: true
        order: 0
        http_authenticator:
          type: openid
          challenge: false
          config:
            subject_key: preferred_username
            roles_key: roles
            openid_connect_url: https://login.microsoftonline.com/-----------------------------/v2.0/.well-known/openid-configuration
        authentication_backend:
          type: noop

And here is the docker-compose.yml:

version: '3'
services:
  opensearch-node1:
    image: opensearchproject/opensearch:1.1.0
    container_name: opensearch-node1
    environment:
      - cluster.name=opensearch-cluster
      - node.name=opensearch-node1
      - discovery.seed_hosts=opensearch-node1
      - cluster.initial_master_nodes=opensearch-node1
      - bootstrap.memory_lock=true
      - "OPENSEARCH_JAVA_OPTS=-Xms512m -Xmx512m"
    ulimits:
      memlock:
        soft: -1
        hard: -1
      nofile:
        soft: 65536 # maximum number of open files for the OpenSearch user, set to at least 65536 on modern systems
        hard: 65536
    volumes:
      - opensearch-data1:/usr/share/opensearch/data
      - ./config.yml:/usr/share/opensearch/plugins/opensearch-security/securityconfig/config.yml
    ports:
      - 9200:9200
      - 9600:9600 # required for Performance Analyzer
    networks:
      - opensearch-net
  opensearch-dashboards:
    image: opensearchproject/opensearch-dashboards:1.1.0
    container_name: opensearch-dashboards
    volumes:
      - ./opensearch_dashboards.yml:/usr/share/opensearch-dashboards/config/opensearch_dashboards.yml
      - ./client.pem:/usr/share/opensearch-dashboards/config/client.pem
      - ./client-key.pem:/usr/share/opensearch-dashboards/config/client-key.pem
      - ./root-ca.pem:/usr/share/opensearch-dashboards/config/root-ca.pem
    ports:
      - 5601:5601
    expose:
      - "5601"
    environment:
      OPENSEARCH_HOSTS: '["https://opensearch-node1:9200"]' # must be a string with no spaces when specified as an environment variable
    networks:
      - opensearch-net

volumes:
  opensearch-data1:

networks:
  opensearch-net:

Anthony · November 5, 2021, 8:39am

@JackBlack You need to enabled basic_auth for OSDashboards to authenticate with opensearch, like so:

basic_internal_auth_domain:
        description: "Authenticate via HTTP Basic against internal users database"
        http_enabled: true
        transport_enabled: true
        order: 0
        http_authenticator:
          type: basic
          challenge: false
        authentication_backend:
          type: intern
      openid:
        http_enabled: true
        transport_enabled: false
        order: 1
        http_authenticator:
          type: openid
          challenge: true
          config: ...

JackBlack · November 5, 2021, 10:35am

@Anthony , I have changed the config.yml file as you mentioned, but it’s still having the same problem.
This is my new config.yml:

config:
  dynamic:
    http:
      anonymous_auth_enabled: false
    authc:
      basic_internal_auth_domain:
        description: "Authenticate via HTTP Basic against internal users database"
        http_enabled: true
        transport_enabled: true
        order: 0
        http_authenticator:
          type: basic
          challenge: false
        authentication_backend:
          type: intern
      openid_auth_domain:
        http_enabled: true
        transport_enabled: false
        order: 1
        http_authenticator:
          type: openid
          challenge: true
          config:
            subject_key: preferred_username
            roles_key: roles
            openid_connect_url: https://login.microsoftonline.com/---------------------------------------------/v2.0/.well-known/openid-configuration
        authentication_backend:
          type: noop

Anthony · November 5, 2021, 11:22am

@JackBlack The Issue could be from Azure side, which is why I sent you DM with quickest method to troubleshoot using separate docker-compose file.

sastorsl · November 24, 2021, 1:11pm

Please ensure that you apply the configuration as well Apply changes with securityadmin.sh - OpenSearch documentation

I ended up reversing the order of authentication as well.
OIDC first, but configure “skip_users” for internal users.

I also think you need to change to challenge: false.

config:
  dynamic:
    authc:
      #
      # OIDC configuration
      #
      openid_auth_domain:
        http_enabled: true
        transport_enabled: true
        order: 0
        http_authenticator:
          type: openid
          challenge: false
          config:
            enable_ssl: true
            verify_hostnames: true
            roles_key: "groups"  # "roles" / "groups", requires the GroupMembers api permission
            subject_key: preferred_username  # In Azure "upn" is only added in v1.0 tokens by default.
            openid_connect_url: https://login.microsoftonline.com/<uuid>/v2.0/.well-known/openid-configuration
            #
            # Users to skip in OIDC to ensure that admin, etc can login with Basic Auth
            #
            skip_users:
              - admin
              - kibanaserver
              - kibanaro
              - logstash
              - readall
              - snapshotrestore
        authentication_backend:
          type: noop
      #
      # Basic auth - used for admin authentication and tooling such as logstash and opensearch-dashboards
      #
      basic_internal_auth_domain:
        description: "Authenticate via HTTP Basic against internal users database"
        http_enabled: true
        transport_enabled: true
        order: 1
        http_authenticator:
          type: basic
          # challenge must be false for oidc to work
          challenge: false
        authentication_backend:
          type: intern
    authz: {}

Topic		Replies	Views
Azure sso with openid issue OpenSearch	0	222	July 13, 2022
Integrate Opensearch with Azure AD using OIDC Security configure	5	1175	June 10, 2022
Azure AD OpenID infinite redirection OpenSearch troubleshoot , configure	0	278	October 20, 2022
Opensearch Dashboard OpenID not working Security troubleshoot	9	1384	May 27, 2022
Integrating openid connect with opensearch throws an 401 unauthroised error , using azure ad as our idp Security troubleshoot , configure	37	1149	November 27, 2023

Opensearch and OpenID (Azure) issue

Related Topics