Use Audit Logging from Elasticsearch.yml Only?

Security
#1

Hi,

OpenDisto 1.12.0

I have a strange misconfiguration between our clusters. We have 5 ES clusters. 2 of them get their audit configuration from elasticsearch.yml. On these I can’t turn on audit logging from Kibana and the endpoint _opendistro/_security/api/audit is NOT IMPLEMENTED. The other 3 ignore audit settings elasticsearch.yml and get their configuration from index.

I actually want them to get their configuration from elasticsearch.yml.

Is there some way to disable the dynamic/hot audit configuration and use the config from elasticsearch.yml? Would it work if I somehow deleted the audit type from .opendistro_security?

Thanks!

#2

@jshrue can you show what configuration is being read from elasticsearch.yml as to the best of my knowledge audit log configuration was moved to kibana UI from 1.11.0

#3

Thanks @Anthony,
What really happened was I was struggling with my newer ES clusters, where the audit logs would begin but then completely die off. Investigating this I wanted to know why the older clusters were configured via elasticsearch.yml and the newer ones via Kibana. I think I have upgraded the older ones a couple times, so they probably started out from 1.9 or 1.10. So this is probably why.

i found a statement in the logs that if I re-run the security configuration it should add the audit type to the security index. Although I don’t like my clusters configured differently, this is probably not worth the risk. I doubt there is a way to make the newer clusters use the elasticsearch.yml config.

opendistro_security.audit.type: internal_elasticsearch
opendistro_security.audit.threadpool.size: 50
opendistro_security.audit.config.disabled_rest_categories: NONE
opendistro_security.audit.config.disabled_transport_categories: NONE

The real problem I was struggling with turned out to be that the newer clusters are gettting more traffic and the increase in the compliance logging wasn’t able to write fast enough to a single shard (the audit index defaults to 1 primary, 2 replicas). Turns out being overwhelmed it virtually stopped writing. I configured a template for the audit index, increasing the number of primaries, and this fixed the real problem.