I am testing the OD Helm chart that can be found here: opendistro-build/helm at main · opendistro-for-elasticsearch/opendistro-build · GitHub
The default supported functionalities like rolling out Master and Data nodes work fine within a single zone, and configuring
affinity by creating each node type on dedicated k8s nodes (VMs) also work.
Now I want to go to the next step of configuring the ES nodes into multiple zones. I have set-up a node pool in 2 zones. The way to achieve this is by adding
node.attr.zone to the
elasticsearch.yaml on each ES node. The value of
node.attr.zone should be the name of the zone of the k8s node where the pod is being created. Then we need to add to each master node or apply as cluster setting the
cluster.routing.allocation.awareness.attributes: zone. And then we are done! How to achieve this with this helm chart?
Another repo discussed this issue and there is a workaround solution offered by using initContainers, see Dynamic pod configuration within a nodeSet for zone awareness · Issue #3933 · elastic/cloud-on-k8s · GitHub
The commands that need to be run in the initContainer are these:
ZONE=$(kubectl get node $NODE_NAME -o json | jq -r '.metadata.labels[\"failure- domain.beta.kubernetes.io/zone\"]') \ CONFIG_FILE=/usr/share/elasticsearch/config/elasticsearch.yml \ CONFIG_FILE_BACKUP=/usr/share/elasticsearch/config/elasticsearch.yml.backup; \ mv $CONFIG_FILE $CONFIG_FILE_BACKUP && \ cat $CONFIG_FILE_BACKUP <(echo node.attr.zone: $ZONE) > $CONFIG_FILE
I tried this solution, but I am getting RBAC issues. And when starting this initContainer, the
elasticsearch.yaml file cannot be found either. Has anyone else tried? I would really like to use Helm for this, instead of explictly creating multiple deployments for each zone.