Replacing demo certificates with my organization certificates is not working

Security
1

Hi,

I am trying to setup open distro elasticsearch in docker container using docker-compose file. I am setting it up for my organization, so I am using our own company certificates signed by our internal CA. I have also added the certs in elasticsearch.yml file.

However when I try to run the securityadmin.sh script using my certs, I am getting below error

Open Distro Security Admin v7
Will connect to localhost:9300 … done
10:55:11.720 [elasticsearch[client][transport_worker][T#1]] ERROR com.amazon.opendistroforelasticsearch.security.ssl.transport.OpenDistroSecuritySSLNettyTransport - Exception during establishing a SSL connection: javax.net.ssl.SSLHandshakeException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
javax.net.ssl.SSLHandshakeException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target

Do I need add the certs anywhere else? Please suggest if I am missing anything here.

Thanks

2

It would help if you provided the complete command line you used that resulted in the error message above. Off the top, I would guess that your CA certificate and your admin certificates don’t match … but having the whole command line would help narrow things down.

3

thanks for your reply, here is the command that I used which resulted in above error message.

[root@359bb31eda70 config]# /usr/share/elasticsearch/plugins/opendistro_security/tools/securityadmin.sh -cacert /usr/share/elasticsearch/config/root-ca.pem -cert /usr/share/elasticsearch/config/node.pem -key /usr/share/elasticsearch/config/sandbox.elk.mydomain.com.key -f /usr/share/elasticsearch/plugins/opendistro_security/securityconfig/config.yml -t config
Open Distro Security Admin v7
Will connect to localhost:9300 … done
06:14:26.783 [elasticsearch[client][transport_worker][T#1]] ERROR com.amazon.opendistroforelasticsearch.security.ssl.transport.OpenDistroSecuritySSLNettyTransport - Exception during establishing a SSL connection: javax.net.ssl.SSLHandshakeException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
javax.net.ssl.SSLHandshakeException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target

I have converted my node and root CA certificates from .crt to .pem

The reason of using .key file in above command is that I tried to convert my key to .pem and tried using it in above command, but it says “File does not contain valid private key”, hence I tried the .key file and got above error.

4

Hello @ravis85. Would you mind sharing your elasticsearch.yml file?
Is your root-ca self-signed certificate?

5

@ravis85 securityadmin.sh requires an admin certificate to execute administrative commands. The demo admin cert is kirk.pem, which is also defined as the value of the opendistro_security.authcz.admin_dn: in elasticsearch.yml.

If you’d like to replace demo admin cert with your own, then you have to add or replace admin_dn with your custom one. opendistro_security.authcz.admin_dn: is an array and can hold more than one value.

6

Hi @pablo I tried what you suggested above, still getting the same error message.

My elasticsearch.yml is as below -

cluster.name: “docker-cluster”
network.host: 0.0.0.0

# minimum_master_nodes need to be explicitly set when bound on a public IP
# set to 1 to allow single node clusters
# Details: Enforce `discovery.zen.minimum_master_nodes` is set when bound to a public ip by bleskes · Pull Request #17288 · elastic/elasticsearch · GitHub
discovery.zen.minimum_master_nodes: 1
# Breaking change in 7.0
# Breaking changes in 7.0 | Elasticsearch Guide [7.0] | Elastic
cluster.initial_master_nodes:
- elasticsearch1
- docker-test-node-1

######## Start OpenDistro for Elasticsearch Security Demo Configuration ########

WARNING: revise all the lines below before you go into production

opendistro_security.ssl.transport.pemcert_filepath: node.pem
opendistro_security.ssl.transport.pemkey_filepath: node-key.pem
opendistro_security.ssl.transport.pemtrustedcas_filepath: root-ca.pem
opendistro_security.ssl.transport.enforce_hostname_verification: false
opendistro_security.ssl.http.enabled: true
opendistro_security.ssl.http.pemcert_filepath: node.pem
opendistro_security.ssl.http.pemkey_filepath: node-key.pem
opendistro_security.ssl.http.pemtrustedcas_filepath: root-ca.pem
opendistro_security.allow_unsafe_democertificates: true
opendistro_security.allow_default_init_securityindex: true
opendistro_security.authcz.admin_dn:

  • CN=admin,OU=Global,O=Company,L=Newyork, C=US

opendistro_security.audit.type: internal_elasticsearch
opendistro_security.enable_snapshot_restore_privilege: true
opendistro_security.check_snapshot_restore_write_privileges: true
opendistro_security.restapi.roles_enabled: [“all_access”, “security_rest_api_access”, “adminrole”]
opendistro_security.system_indices.enabled: true
opendistro_security.system_indices.indices: [“.opendistro-alerting-config”, “.opendistro-alerting-alert*”, “.opendistro-anomaly-results*”, “.opendistro-anomaly-detector*”, “.opendistro-anomaly-checkpoints”, “.opendistro-anomaly-detection-state”, “.opendistro-reports-", ".opendistro-notifications-”, “.opendistro-notebooks”, “.opendistro-asynchronous-search-response*”]
cluster.routing.allocation.disk.threshold_enabled: false
node.max_local_storage_nodes: 3
######## End OpenDistro for Elasticsearch Security Demo Configuration ########

And the command results is as below -

./securityadmin.sh -cd …/securityconfig/ -icl -nhnv -cacert …/…/…/config/root-ca.pem -cert …/…/…/config/admin.pem -key …/…/…/config/admin.key
Open Distro Security Admin v7
Will connect to localhost:9300 … done
15:15:15.637 [elasticsearch[client][transport_worker][T#1]] ERROR com.amazon.opendistroforelasticsearch.security.ssl.transport.OpenDistroSecuritySSLNettyTransport - Exception during establishing a SSL connection: javax.net.ssl.SSLHandshakeException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
javax.net.ssl.SSLHandshakeException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target

My question is that do I need to add the admin cert to keystore/truststore file to make this work, if yes, then could you please tell me which keystore file I need add the certificate in?

Thank you.

7

@ravis85 I think your issue is in elasticsearch certificate.
By default securityadmin.sh uses localhost as the default target and will try to connect on the transport layer (ports 9300-9400).

Since you’ve replaced your certificates with custom certs you might forget to add localhost to SAN (Subject Alternative Name) of the node.pem certificate defined in opendistro_security.ssl.transport.pemcert_filepath.

You have two solutions here.
First, if you wish to keep using localhost as a target, you should regenerate the transport certs and add localhost to SAN.
Second, you can use -h option with either IP address or FQDN of the Elasticsearch node. Be sure that either IP address or FQDN are in CN or SAN of the node.pem certificate.

i.e.
./securityadmin.sh -cd …/securityconfig/ -icl -nhnv -cacert …/…/…/config/root-ca.pem -cert …/…/…/config/admin.pem -key …/…/…/config/admin.key -h <IP_or_FQDN_of_ES_node>

8

Hi @pablo Thanks for your reply.

I will try the suggested solutions and update.