Index alias for multiple indices

Hi,

I am currently working with Opendistro elasticsearch and we have been using ISM. The process is when a new Logstash configuration is created with a new index, we manually create a template for the index and create a new index with an alias and place the alias in Logstash.

We have a new requirement, where we will need to setup the same way we use currently but for new application. For example the new index will look like logstash-%%{app} where the app will be a tag, or a field from filebeat.

I am stuck now, because in this situation we may not know the application name in advance and hence not sure how to proceed. Can someone help here?

Any response for this issue?

Same issue here. We are looking for some ISM options in in the Logstash plugin.
Something like this -> https://www.elastic.co/guide/en/logstash/current/plugins-outputs-elasticsearch.html#plugins-outputs-elasticsearch-ilm

Also same issue here. I have logs being sent via logstash to indices named syslog-YYYY.MM.DD.

logstash.conf output looks like this:

output {
   elasticsearch { 
       hosts => ["https://localhost:9200"]
       index => "%{type}-%{+YYYY.MM.dd}"
       ssl => true

These are associated to an alias named syslog using the index regex syslog-20*

{
    "actions": [
      {
        "add": {
          "index": "syslog-20*",
          "alias": "syslog"
        }
      }
    ]
  }

Since the indices are named with a date suffix, and not the -00001 format that the rollover function looks for, I need to create a separate index named syslog-00001 and set is_write_index: true for the syslog alias. Then my logs get rolled, but I’m constantly incrementing a separate (albeit smaller) index…

It would be nice to be able to configure logstash output to specify alias/pattern as @ larwied mentioned or have the option to rollover indices matching a custom regex and not just “^.*-\d$”

@dbbaughe Don’t know if you saw this since it wasn’t under the Index Management category…can you provide any guidance here?

I did not see this @GSmith, thanks :slight_smile:
Moving to ISM section and will look into this

Bumping for any updates?
We have similar indices named with variables such as beat version and date, that we don’t know in advance. Our setup is FB>LS>ES and we want to roll based on size.
filebeat_testing-7.2.4
filebeat_testing-7.5.0

filebeat_testing-7.5.0-2020-07-29

Any help is appreciated,
Currently we are using the following template with no luck. Bootstrapping works only if LS outputs to a constant “filebeat_testing”.

PUT _template/ism-test-template
{
“index_patterns”: [
“filebeat_testing*”
],
“order”: 10,
“settings”: {
“opendistro.index_state_management.rollover_alias”: “filebeat_testing”,
“opendistro.index_state_management.policy_id”: “simple_rotate_100GB”,
“number_of_shards”: 4,
“number_of_replicas”: 1,
“refresh_interval”: “30s”
}
}