OpenSearch docker image much bigger than ES

opensearchproject/opensearch 1.48GB
elasticsearch-oss-7.10.2 699MB

double the size ?

2 Likes

Moved to build category

Keep in mind that OpenSearch is not only OSS Elasticsearch. It includes all of the Open Distro plugins as well. The Elasticsearch X-Pack container is also larger than the OSS container as it include all of the X-Pack code.

We currently don’t need any of those plugins.
So, it’s just unnecessary weight for us.

The plan is also to have container images with data inside and that will change weekly.
So we will have a bunch of images around.
Having 100s of 1.5 GB or 0.7 GB images is quite a difference in managing, uploading, downloading etc.
It will probably also increase the cost of our CI/CD as those images will be part of integration testing.

So, we’d much prefer something much more lightweight.

Regards,
Petar

I was just explaining why it is bigger, which seemed to be your question.

Given your use-case you may be better of sticking to elasticsearch-oss-7.10.2. However you should keep in mind Docker’s layered images. Once all of the layers are present on a system, they do not have to be transferred or stored again just because another container is referencing them.

If you build a new container starting with FROM opensearchproject/opensearch:1.0.0-rc1 you will not be adding another copy of opensearchproject/opensearch:1.0.0-rc1 if it already exists on the system. Your new container will simply reference the layer’s already present. A well designed use of containers will consider the maximum re-use of existing layers. This can greatly reduce the storage and network transfer requirements, even if you are using elasticsearch-oss-7.10.2.

Hmm, yes you’re right.

Will have to check if this docker layering applies also to AWS ECR.

I still think having the pure opensearch image without plugins would be a good idea.

Thanks,
Petar

@PetarR Minimal artifacts (w/o plugins) are already on the roadmap for the 1.0 GA release, specifically tracked in this issue.

Just keep in mind the minimal artifact does not include security (TLS/Authz/Authc/etc.) so unless you have those taken care of otherwise, go for the non-minimal or at least install the security plugin.

We also opened an issue on this - seems like there are some known optimizations that can slim down the docker image.

Thanks to both of you.
Our ES 7.10.2 cluster is inside our VPC.
We need this docker image mostly on our development boxes and our CI/CD pipeline.
So, no need for security plugins.

Regards,
Petar