Hi,
Is there any way to perform event correlation on Opensearch or ODFE. If not, is there any workaround to get a correlation results.
Thanks !!
@rtarek How do you define “event correlation” exactly? This is a very overloaded and vague term in our industry 
Thanks @jkowall thats a very good point !!
So imagine I’m looking for the following behavior:
A user account X created and deleted in a short period.
You can issue some DSL / KQL queries to get user creation events or user deletion event. But there is no shuch DSL or KQL queries to get users account created and deleted in a short period of time. So this requires events correlation.
I hope my example was clear.
This sounds like UEBA (User and Entity Behavioral Anomaly Detection) which is a security use case for SIEM products. This is not something simple to do since you have to create user behavior profiles. In order to do that you must normalize the data and have a common schema to understand various types of user actions. We have built this on top of ElasticSearch today, but we sell a product that does it. By no means is this a simple query or plugin, but part of a product that we sell a lot of at my company. We are still building more capabilities but the alerting system and various other behavioral analysis capabilities are required.
If you have a specific simple requirement to look for user creation on a single logfile and creating alerts you can do this with the alerting ODFE plugin, but you’d have to create and manage all of the rules and it wouldn’t have a behavioral pattern detection. I would have a look at Alerting - Open Distro for Elasticsearch Documentation
1 Like
Thank you @jkowall
Actually, you can still detect a lot of UEBA use-cases using just simple DSL / KQL queries. However, there are some cases where you need correlation (Like in the example). In the Elastic version of ELK, the EQL (Event Query Language) offers the possibility to perform event correlation.
Besides that, can I get more information about the product you are selling ?
Thank you !
Query language is the same on OpenSearch, so I don’t see why the same wouldn’t work. Either way I don’t consider that event correlation personally. As I said we do this more complex multi-step rule evaluation in our alerting system at logz.io which is more sophisticated than the open source alerting in ODFE.
1 Like
The Event Query Language is not present on Elasticsearch OSS neither in ODFE or OpenSearch.
I think you are talking about this:
https://docs.logz.io/user-guide/siem/security-correlated-queries/
And Indeed, this is what I was looking for, and in fact, EQL can do the same job.
You can read about EQL here : Create a detection rule | Elastic Security Solution [8.4] | Elastic
Thank you !
Correct, these are both alerting systems and not query languages. The ODFE alerting system would need to be enhanced for this use case. The logz.io alerting could be open-sourced, but it would be a lot of work to make it run outside of our RBAC system and other elements it relies upon. It runs outside of ElasticSearch completely today.