Hello, I have a question regarding resources management when there are Anomaly Detection jobs on the cluster.
We have standard scenario:
- we have a cluster used for monitoring (ETL jobs deliver the data, users are consuming dashboards)
- some users are doing additionall ad-hoc data analysis
- some users are defining new ML jobs using Anomaly Detection module.
As I understand by default AD jobs are using datanodes, which can result in all cluster nodes to have problems if somebody will use too much resources for AD.
On “standard elastic” there is a way of dedicating nodes for ML jobs by assigning them ml role.
So is there a way of assigning AD nodes like ML role?
Or how you guys are managing limitation or making sure there is a resource pool for core cluster actions so the AD jobs will not cause problems like too high load on data nodes or Out Of Memory errors?