Are there plans of adding Observability/Uptime support on OpenSearch Dashboard? Or any viable alternatives to get these installed on the current OpenSearch distribution
I assume you’re talking about the Elastic Observability stuff? There is no way to install Elastic-provided plugins on OpenSearch (the internal namespace is different to start). I’m pretty sure this also not open source-licensed.
IIRC this is all based on Heartbeat and you can use a specific version of that with OpenSearch - see this page Agents and ingestion tools - OpenSearch documentation .
There is also a team working on OpenSearch Observability features, but I’m not sure this would be included in their charge (it’s more about things like traces).
If the focus here is availability monitoring using Elastic as a backend is not a good idea. Metrics are not well suited to be stored in elastic due to the inability to do math across large series of measures. This is why most are using backends like Prometheus, Influx, or others for metric data. Using Prometheus you can use the “black box exporter” for this use case.
Thanks @jkowall for the inputs. Since the upstream Elastic project supports it out of the box and its the same elasticsearch backend, there must be a reason for everyone to think its not a “anti-pattern”
The unified database concept is a nice one in theory, unfortunately all of the databases are not well suited for any data. You can use elastic for metrics (ex: metricbeat) but I can do so for about 1/20th of the cost and 100x the speed with Prometheus at scale. Also the query language for Elastic is not well suited for time series, looking at PromQL you have a much more powerful query language especially for observability where you want to build quantiles/histograms and other buckets which are simply not possible in Elastic. I can tell you from first-hand experience, we build a metric platform on Elastic and had to abandon it, we even created specific capabilities to deal with rollups and other ways to save on data cost, it’s simply not a good approach to use the Elastic backend for these use cases with any kind of scale.
3 Likes
Thank you @jkowall for useful insights. Will try prom/grafana. Its just an additional bit of infra to manage which I wanted to avoid
Hoping there is progress on using OpenSearch Dashboards with Prometheus, which would be very helpful 
2 Likes