No permissions for [cluster:monitor/health]

While bootstrapping the cluster, I created a custom internal user for monitoring purpose with readall_and_monitor built-in role by putting the user spec in internal_users.yml file.

$ cat plugins/opendistro_security/securityconfig/internal_users.yml 
# This is the internal user database
# The hash value is a bcrypt hash and can be generated with plugin/tools/hash.sh
_meta:
  type: "internalusers"
  config_version: 2
# Define your internal users here

admin:
  hash: $2a$12$cTiFk0k0Ag2JEGlu4i6BDu.ESh0qzFHivIE/5uvaBdFpKMAo3ZPRa
  reserved: true
  backend_roles:
  - admin
  description: "Demo admin user"

# <--------------here----------->
readall_monitor: 
  hash: $2a$12$j/8/2MmfAvtPQYpClNX1/OadvXj.imjke8ePRTLs4xXlPVZVHwI1C
  backend_roles:
  - readall_and_monitor

Cluster is up and running but when I try to check health with this user credential it shows the following error:

$ curl -XGET "https://localhost:9200/_cluster/health?pretty" -u "readall_monitor:fuuax6j2" --insecure
{
  "error" : {
    "root_cause" : [
      {
        "type" : "security_exception",
        "reason" : "no permissions for [cluster:monitor/health] and User [name=readall_monitor, backend_roles=[readall_and_monitor], requestedTenant=null]"
      }
    ],
    "type" : "security_exception",
    "reason" : "no permissions for [cluster:monitor/health] and User [name=readall_monitor, backend_roles=[readall_and_monitor], requestedTenant=null]"
  },
  "status" : 403

  • What am I missing here, Do I need to create role mapping too?

Hello @kamolhasan, as i is described in the documentation - Users and Roles - Open Distro Documentation

Backend roles differ from security roles. Backend roles are arbitrary strings that you specify or that come from an external authentication system (e.g. LDAP/Active Directory)

Can you try to do a role mapping as you supposed.

@stmx38 I’m little confused.

How can I use those roles? Users and Roles - Open Distro Documentation

As you mentioned, these are security roles, so it shouldn’t be mentioned under backend_roles? right?

readall_monitor: 
  hash: $2a$12$j/8/2MmfAvtPQYpClNX1/OadvXj.imjke8ePRTLs4xXlPVZVHwI1C
  backend_roles:
  - readall_and_monitor

But the following configuration doesn’t work either.

readall_monitor:
  hash: $2a$12$5FS2ApvIzJpMi41ezyqr8eYzBY7B8p6mFD/QERjV6Z4cMOYQlKYBK
  opendistro_security_roles:
  - readall_and_monitor

Can you please guide me stepwise? :cry: I’m new to elasticsearch.

Role mapping is done under roles_mapping.yml file and looks like:

readall_and_monitor:
  description: "Maps readall_monitor user to readall_and_monitor role"
  reserved: false
  users:
  - "readall_monitor"

Then, you should apply modifications using securityadmin.sh

You can also do it via Kibana UI:

Kibana --> Security --> Role mappings --> +

Role: read_and_monitor
User: + Add User --> readall_monitor
--> Submit

And then you can try to check your permissions modifications.

@stmx38 then what are the functionalities of opendistro_security_roles & backend_roles fields in userSpec:

<username>:
    hash:
    opendistro_security_roles:
    backend_roles:

Since all those roles are needed to map again to this <username> from role_mapping.yml?

Probably I don’t know, but you can start reading from Security/Access Control/Users and Roles

We use backend Roles but for Roles and mappings to map some LDAP groups:

ELK-Readers:
  backend_roles:
  - "ELK-TEST_Readers"

ELK-Readers - Kibana Role with specific permissions
ELK-TEST_Readers - LDAP Group
All users from LDAP group will have permissions from mapped role.

We also use backend_roles in roles_mapping.yml for anonymous access:

opendistro_security_anonymous:
  backend_roles:
  - "opendistro_security_anonymous_backendrole"

Terminology for users and roles in Open Distro security plug-in

Backend roles
Backend roles differ from security roles. Backend roles are external roles that come from an external authentication system, such as LDAP. If you do not use such an external system, that is, if you use the internal user database, you can ignore backend roles.

‘you can ignore backend roles’ :slight_smile:

You can ignore backend roles, but depending on your use case you might not want to. Backend roles don’t necessarily have to relate to external authentication system, they can just be arbitrary text strings you use to group together multiple (non backend) roles. As the ODFE doc describes them:

[Backend roles] Arbitrary strings that you specify or that come from an external authentication system (e.g. LDAP/Active Directory). Backend roles can help simplify the role mapping process. Rather than mapping a role to 100 individual users, you can map the role to a single backend role that all 100 users share.

In my project dealing with log monitoring in Kubernetes, we’re using namespace-specific backend roles to limit users access to log messages from a specific namespace AND the existing kibana_user role to grant access to Kibana. This allows to grant/limit access to both things just by assigning the user to the single namespace-specific backend role. That’s easier to manage than having to do rolemappings between the users and each of the (non backend) roles directly.

Hi @kamolhasan Did you get the roles working based on the details in above thread?

I am facing the same issue, while deploying my opensearch cluster, a custom internal user for monitoring purpose with custom role by putting the user spec in internal_users.yml file. But user is not being created. Below are the details

internal_users.yml:
        prometheus:
          hash: "$2y$12$Sfi1XzL7j94vpAjRnK0wOeMzJ60XNBSeN0BLh/srmweHnfpgnhdMC"
          reserved: false
          description: "Prometheus metrics user"


roles.yml:
        # Allows prometheus users to get the metrics
        prometheus:
          reserved: false
          cluster_permissions:
            - 'cluster:monitor/health'
            - 'cluster_monitor'
roles_mapping.yml:
        prometheus:
          reserved: false
          users:
          - "prometheus"

Could anyone please help me out to discover the issue

@kkasha We use the Elasticsearch Exporter to collect metrics from OpenSearch and make them available to Prometheus… We configure the Elasticsearch Exporter to use a user we call “metricgetter”. Here’s how we define that user and the corresponding role (we did not need to create any special role-mappings).

The MetricGetter User definition

# MetricGetter User
metricgetter:
  hash: "some_hashed_password"
  reserved: false
  description: "Metricgetter user"
  opendistro_security_roles:
  - metric_collector

The Metric Collector role definition

# Metric collector
metric_collector:
  reserved: false
  index_permissions:
    - index_patterns:
      - "*"
      allowed_actions:
        - indices_monitor
  cluster_permissions:
    - cluster_monitor
    - manage_snapshots

We’re currently using OpenSearch 1.3.2 (although I believe these were originally created when we worked with OpenDistro for Elasticsearch 1.13)