I’ve been experimenting with applying policies to my indices. I tried applying the example hot/cold policy to and index, but when I assign the policy to the index the status gets stuck in “Initializing”. Ive tried creating other policies and applying those, but they get stuck as well. I followed the instructions in the OpenDistro docs for ISM. Do you have any ideas what to do?
Are you using ODFE or are you on Amazon Elasticsearch Service?
There is a job_interval setting which controls how fast jobs are executed (only configurable on ODFE). The default for ODFE is 5 minutes and the default for the service is 30-48 minutes. So it should show Initializing for that amount of time. Did your indices get stuck on initializing for longer than that?
I did test this with an index and an alias and the alias was rolled over to a new index; however, the policy was not applied to the new index. I’m trying to use the policy to create a rolling window of documents that get aged out and deleted after 30 days. I’d like to have an old index for older data, and a current index for new data to be added to, but to get this to work I need the same policy applied to the new index created by the rollover. Do I have to do this manually each time an index is rolled over? It may be I’m going about this all wrong, but I haven’t seen a suggested way of doing this in the documentation or elsewhere on line.
You should be able to use an index template to do this. You would create an index template that applies to your group of indices with the policy_id and rollover_alias settings. And then whenever a new index is created from the rollover action it will have the settings applied where it will bootstrap the policy and use the same rollover_alias for rolling over.
Thanks Drew. I’ve looked through the OpenDistro and documentation on index templates at https://opendistro.github.io/for-elasticsearch-docs/docs/elasticsearch/index-data/ , but I don’t see a way to specify a policy inside the template. I also looked through the definitions for mappings and settings in the ElasticSearch documentation and I didn’t find anything there either. Also, my Kibana Index Management panel doesn’t have support for specifying templates. What am I missing?
This would create an index template that matches all indices with the pattern some-index-pattern*. Whenever an index that matches this pattern is created it will automatically apply these settings, policy_id and rollover_alias.
So in your rollover case, you have a policy that is rolling over the index. When it executes rollover it will create a new index and roll over the alias. This new index will match this template pattern and automatically apply the policy_id and rollover_alias. It will then be managed by ISM and keep repeating for newly rolled over indices.
The Kibana Index Management plugin does not have a place to manage Index Templates yet although it is something we want to add. For now you would have to create them through the Elasticsearch APIs or in the Kibana DevTools.