Upgrading from ODFE 1.11.0 to 1.12.0 - Deprection logs complain about a lot

Open Source Elasticsearch and Kibana
#1

Hi,

Not sure if this is actually an issue or purely cosmetic but i just upgraded from ODFE 1.11.0 to 1.12.0 and the deprecation log is filling up with logs such as this when the node has just been restarted:

[2020-12-15T07:47:08,418][DEPRECATION][o.e.d.i.m.IdFieldMapper ] [test-es01] Loading the fielddata on the _id field is deprecated and will be removed in future versions. If you require sorting or aggregating on this field you should also include the id in the body of your documents, and map this field as a keyword field that has [doc_values] enabled

And when i upgraded ODFE Kibana the deprecation logs complain about the following:

[2020-12-15T07:48:57,136][DEPRECATION][o.e.d.a.b.BulkRequestParser] [test-es01] [types removal] Specifying types in bulk requests is deprecated.
[2020-12-15T07:48:57,137][DEPRECATION][o.e.d.c.m.IndexNameExpressionResolver] [test-es01] this request accesses system indices: [.kibana_-152937574_admintenant_5, .kibana_1441287829_iveronil_3, .kibana_5, .opendistro-anomaly-results-history-2020.07.11-000002], but in a future major version, direct access to system indices will be prevented by default

Is this an issue or purely a cosmetic issue? How can i fix this?

3 Likes
#2

I got bit by this also but on the .kibana_3 index

Any thoughts on how to remove these from the logs.about 20 to 40 entries per second in the logs…

Wondering if this is being triggered by logstash ingestion…

[2021-01-20T13:56:00,663][DEPRECATION][o.e.d.a.b.BulkRequestParser] [myserver] [types removal] Specifying types in bulk requests is deprecated.
[2021-01-20T13:56:00,664][DEPRECATION][o.e.d.c.m.IndexNameExpressionResolver] [myserver] this request accesses system indices: [.kibana_3], but in a future major version, direct access to system indices will be prevented by default

#3

Hi, I am also getting a lot of the above mentioned DEPRECATION logs.
Platform: Kubernettes Cluster 1.16.15-gke.6000
Elasticsearch Version: x-content-7.10.0
Open Distro Security Version: 1.12.0.0
Integrated: LDAP, Fluentbit

[2021-01-29T09:50:51,383][DEPRECATION][o.e.d.c.m.IndexNameExpressionResolver] [opendistro-es-client-bd868b588-jjx62] this request accesses system indices: [.kibana_-1925406847_user1_1, .kibana_1, .kibana_1361082134_user2_1, .kibana_1694441575_user3_1, .kibana_1760453036_user4_1, .kibana_92668751_admin_1], but in a future major version, direct access to system indices will be prevented by default

I would really like to know the cause of this. Could it be the fluentbit output Type, flb_type

Regards

#4

Hi,

as we ship these logs to AWS CloudWatch I first tried tried to disable the logs to cut costs by executing

PUT /_cluster/settings
{
  "persistent" : {
    "logger.deprecation.level" : "ERROR"
  }
}

as described on Logs - Open Distro for Elasticsearch Documentation. This didn’t stop the deprecation log.

Then I executed

PUT /_cluster/settings
{
  "persistent" : {
    "logger.org.elasticsearch.deprecation" : "ERROR"
  }
}

as described on Turn off ES deprecation Logs Dynamically - Elasticsearch - Discuss the Elastic Stack.
Now deprecation logs are silent.

Regards

Clifford

3 Likes
#5

Thank you, this solved my issue.

#6

for

[2021-01-20T13:56:00,663][DEPRECATION][o.e.d.a.b.BulkRequestParser] [myserver] [types removal] Specifying types in bulk requests is deprecated.

I’m currently looking at

#7

for

[2021-01-20T13:56:00,664][DEPRECATION][o.e.d.c.m.IndexNameExpressionResolver] [myserver] this request accesses system indices: [.kibana_3], but in a future major version, direct access to system indices will be prevented by default

I had a look at the code [1] and the pull request [2]. I also found [3] which explains the reason for the deprecation.

To my understanding this will be fixed, when 8.0 will be released. I decided to disable the deprecation logging until then.

[1] elasticsearch/IndexNameExpressionResolver.java at master · elastic/elasticsearch · GitHub
[2] https://github.com/elastic/elasticsearch/pull/60945
[3] System Indices · Issue #50251 · elastic/elasticsearch · GitHub

Regards,

Clifford

#8

I’m seeing this as well and it definitely correlated to Logstash writing logs.

[2021-05-20T13:53:57,862][DEPRECATION][o.e.d.c.m.IndexNameExpressionResolver] [es-node-1] this request accesses system indices: [.kibana_1, .kibana_task_manager_1], but in a future major version, direct access to system indices will be prevented by default

Sure, disabling the deprecation log hides the issue but I think that the more pressing question is: Why is Logstash even attempting to access Kibana indices? Is something really wrong or is the deprecation warning detection itself broken?