Nodes fall out of the cluster es 7.9.1

Open Source Elasticsearch and Kibana
#1

Hello, we have the following problem: nodes can periodically fall out of the cluster

[2020-11-19T09:00:37,415][INFO ][o.e.c.s.MasterService    ] [h1-es03] node-left[{h1-es02}{qgmMV2UbT-ScN9uRr6YM8g}{klj1K8UMRHGtYpYozIuqsA}{192.168.57.102}{192.168.57.102:9300}{dimr} reason: followers check retry count exceeded], term: 87,
 version: 236727, delta: removed {{h1-es02}{qgmMV2UbT-ScN9uRr6YM8g}{klj1K8UMRHGtYpYozIuqsA}{192.168.57.102}{192.168.57.102:9300}{dimr}}
[2020-11-19T09:00:39,763][INFO ][o.e.c.s.ClusterApplierService] [h1-es03] removed {{h1-es02}{qgmMV2UbT-ScN9uRr6YM8g}{klj1K8UMRHGtYpYozIuqsA}{192.168.57.102}{192.168.57.102:9300}{dimr}}, term: 87, version: 236727, reason: Publication{term
=87, version=236727}
[2020-11-19T09:00:47,890][INFO ][o.e.c.s.MasterService    ] [h1-es03] node-join[{h1-es02}{qgmMV2UbT-ScN9uRr6YM8g}{klj1K8UMRHGtYpYozIuqsA}{192.168.57.102}{192.168.57.102:9300}{dimr} join existing leader], term: 87, version: 236730, delta:
 added {{h1-es02}{qgmMV2UbT-ScN9uRr6YM8g}{klj1K8UMRHGtYpYozIuqsA}{192.168.57.102}{192.168.57.102:9300}{dimr}}
[2020-11-19T09:00:52,713][INFO ][o.e.c.s.ClusterApplierService] [h1-es03] added {{h1-es02}{qgmMV2UbT-ScN9uRr6YM8g}{klj1K8UMRHGtYpYozIuqsA}{192.168.57.102}{192.168.57.102:9300}{dimr}}, term: 87, version: 236730, reason: Publication{term=8
7, version=236730}

3 nodes in a cluster, each can be a master and is a datanode. Elasticsearch 7.9.1 build opendistro from amazon.
In addition, we write to Elasticsearch using Apache Metron and when we try to write sometimes, we get errors:

java.io.IOException: listener timeout after waiting for [30000] ms
#2

Do you have the logs for h1-es02 at that time to see what happened with the node?

#3

We faced with this problem again. Now I have log, but can’t understang how to load it to this forum. Seems that I can’t change my initial message, and can’t attach to new message any file that not a picture.