Index_management_exception when setting ism_template

Prior to version 1.13.0, we were attaching ISM policies to indices via the template api like below.

PUT /_template/some-index
{
      "index_patterns": "example*",
      "order": order,
      "settings": {
          "index": {
            "number_of_shards": shards,
            "number_of_replicas": replicas
           "opendistro.index_state_management.policy_id": ism_policy
          }
        }

After version 1.13.0, opendistro.index_state_management.policy_id is deprecated, and we had to use ism_template like in the code snippet to achieve the same behaviour.

PUT _opendistro/_ism/policies/ism_policy
{
  "policy": {
    "description": "Delete index after 14d",
    "default_state": "search",
    "states": [
      {
        "name": "search",
        "actions": [],
        "transitions": [
          {
            "state_name": "delete",
            "conditions": {
              "min_index_age": "14d"
            }
          }
        ]
      },
      {
        "name": "delete",
        "actions": [
          {
            "delete": {}
          }
        ],
        "transitions": []
      }
    ],
    "ism_template": {
      "index_patterns": ["example*"]
    }
  }
}

However, we are faced with this error when running the above command in the kibana dev tools. We tried using different priority numbers and it randomly works and errors out. Not sure why this is happening.

{
  "error" : {
    "root_cause" : [
      {
        "type" : "index_management_exception",
        "reason" : "New policy short has an ISM template with index pattern [example*] matching existing policy templates, please use a different priority than 44"
      }
    ],
    "type" : "index_management_exception",
    "reason" : "New policy short has an ISM template with index pattern  [example*]  matching existing policy templates, please use a different priority than 44",
    "caused_by" : {
      "type" : "exception",
      "reason" : "java.lang.IllegalArgumentException: New policy short has an ISM template with index pattern  [example*]  matching existing policy templates, please use a different priority than 44"
    }
  },
  "status" : 400
}

[Welcome @lajansa - I moved your question to a better category]

Hey @lajansa,

Since we have moved the index matching logic onto the policies themselves in the ism_template field we need a way to know which policy to choose if there are conflicting templates. That’s why there is a priority field; we use this to deal with conflicts. We also explicitly reject new policies that match another policies index pattern and has the same priority because in this case we won’t know which policy you meant for us to apply to the index. Based on that error I am assuming you have another policy that overlaps with that index pattern and has the same priority of 44.

1 Like

Hi @dbbaughe thank you for your reply. Hmm, as far as I can tell, we don’t have overlapping index patterns. However, we were updating the existing policies with the same index patterns that used to be updated using the index template before. This is because we have a script that does this periodically. Do you think this could have caused the problem?

@lajansa

Could you share the policies you currently have stored (can trim them down to just policy id and ism templates) and the one you are trying to create or update?

Hi @dbbaughe sorry for the late reply. You were right before that there were overlapping index patterns. We thought the index patterns were unique enough previously (e.g. cat** and dog) but the wildcard does cause it to be overlapping. Thanks for your help.

1 Like