Is OpenDistro/OpenSearch exposed to ReDoS attack?

Hello,
By using a specific regExp query, I am able to cause 100% cpu for a long time.
I would expect the internal Lucene regExp engine to limit the execution after short period. But unfortunately, it doesn’t.

Is there any way to overcome this?
If not, can it be fixed in next version?
I think Open Search must protect itself - and us the developers :slight_smile: - in this case.

Here is how to reproduce it (opendistro 1.12):

POST regexp-test1/_doc/test01
{
    "stringvalue" : "aaaaaaaaaaaaaaaaaaaaaaaaaasssssssssssssssssssssssssssssss"
}


GET regexp-test1/_search
{
  "query": {
    "regexp": {
      "stringvalue": {
        "value": "(.*a){2000}"
      }
    }
  }
}


GET /_nodes/stats/process
{
  "_nodes" : {
    "total" : 2,
    "successful" : 2,
    "failed" : 0
  },
  "cluster_name" : "odfe-cluster",
  "nodes" : {
    "BlLGbL6ZQEOtI-Aj3jaXMg" : {
      "timestamp" : 1620681797816,
      "name" : "odfe-node2",
      "transport_address" : "172.21.0.4:9300",
      "host" : "172.21.0.4",
      "ip" : "172.21.0.4:9300",
      "roles" : [
        "data",
        "ingest",
        "master",
        "remote_cluster_client"
      ],
      "process" : {
        "timestamp" : 1620681797816,
        "open_file_descriptors" : 639,
        "max_file_descriptors" : 65536,
        "cpu" : {
          **"percent" : 25,**
          "total_in_millis" : 6514870
        },
        "mem" : {
          "total_virtual_in_bytes" : 5553577984
        }
      }
    },

Thank you,
Ori.

@orid I don’t see anything Open Distro specific in this query, so if this problem exists it’s stemming from Elasticsearch.

Should be interesting to see if this is still present in OpenSearch. It has an updated version of Lucene vs ODFE 1.12. Once the beta comes out it would be worth while to see if it’s still reproducible. If it is, open an issue for sure.

Hi @searchymcsearchface ,
I’ve downloaded latest OpenSearch and built it from sources.
The issue is easily reproduced as well:


top - 11:46:43 up 50 days,  4:13,  6 users,  load average: 0.59, 0.38, 1.76
Tasks: 223 total,   1 running, 222 sleeping,   0 stopped,   0 zombie
%Cpu(s): 25.6 us,  0.3 sy,  0.0 ni, 74.1 id,  0.0 wa,  0.0 hi,  0.0 si,  0.0 st
KiB Mem : 16266524 total,  7994040 free,  5803556 used,  2468928 buff/cache
KiB Swap:  4063228 total,  4039932 free,    23296 used. 10108712 avail Mem

  PID USER      PR  NI    VIRT    RES    SHR S  %CPU %MEM     TIME+ COMMAND
15818 ori       20   0 4217756 705716  12296 S 100.3  4.3   1:00.82 java

I will open a Github issue and update.

Thank you,
Ori.

Thanks Ori! Good catch too.

Sidenode: I wish I could bet on the cause of software problems in Las Vegas. I’d always put my money on RegExp. I’d be a millionaire.

2 Likes

https://github.com/opensearch-project/OpenSearch/issues/687

1 Like