DSL query for indices not receiving any docs

Plugins Alerting
1

Hello,

We want to set up an alert using opendistro for indices not receiving any docs for the past 5 mins. I used the must_not exist field “@timestamp” since it always exists in all documents but i get no results… Does anyone have any workaround? I intentionally stopped logstash to stop sending events to the indices but im getting 0 hits for the below query. Any help would be greatly appreciated!

{
    "size": 0,
    "query": {
        "bool": {
            "filter": [
                {
                    "range": {
                        "@timestamp": {
                            "from": "{{period_end}}||-5m",
                            "to": "{{period_end}}",
                            "include_lower": true,
                            "include_upper": true,
                            "format": "epoch_millis",
                            "boost": 1
                        }
                    }
                }
            ],
            "must_not": [
                {
                    "exists": {
                        "field": "@timestamp",
                        "boost": 1
                    }
                }
            ],
            "adjust_pure_negative": true,
            "boost": 1
        }
    },
    "aggregations": {}
}
2

You could try to query for presence of documents and then the trigger condition could be looking for zero results which can invoke a action.

3

@jathin12 any clue on how to implement this using DSL? I thought about it but the count is returned only after the query is run. So i dont know how to make this a nested query

4

you dont need to do that operation in the monitor section… you write a condition on “trigger” section of your alerts.
check examples here- API - OpenSearch documentation

5

oh i see. I also thought of that, but this means that i have to do the same thing for every index… I want to have a single monitor for all of the indices for dynamic alerting no matter the index changes. This is why i went with the must_not query filter. The question here, since timestamp isnt picked up, can i do any other changes to be able and check the metadata fields for every document? E.g checking if an _id doesnt exist within a time period? This would solve the problem.

Regards,
Tony

6

you dont have to… you can create a monitor on multiple indices and maybe aggregate on some unique field, and when any one or more return 0 hits then it can trigger the action which can display all the indices that are not getting any records.

check Create bucket-level monitor section on examples wiki.

7

Thanks @jathin12 , the problem here is that the only common field in all is @timestamp this is why i was trying to set that up initially.