Filebeat not connecting

hello all,
Seems that open distro instructions are not updated or not correct. After following the instructions for setting up Suricata module I always get the same error :
Failed to connect to backoff(elasticsearch()): Connection marked as failed because the onConnect callback failed: cannot retrieve the elasticsearch license from the /_license endpoint, Filebeat requires the default distribution of Elasticsearch. Please make the endpoint accessible to Filebeat so it can verify the license.: could not retrieve the license information from the cluster: 500 Internal Server Error: {“error”:{“root_cause”:[{“type”:“security_exception”,“reason”:“Unexpected exception indices:admin/get”}],“type”:“security_exception”,“reason”:“Unexpected exception indices:admin/get”},“status”:500}

My config file is the following.

filebeat.config.modules:
  enabled: true
  path: ${path.config}/modules.d/*.yml

setup.ilm.overwrite: true
setup.ilm.enabled: false

output.elasticsearch:
  ssl.verification_mode: none
  hosts: ["localhost"]
  username: "admin"
  password: "admin"
setup.kibana:
  host: "localhost"

logging.level: debug
logging.to_files: true
logging.files:
  path: /var/log/filebeat
  name: filebeat
  keepfiles: 7
  permissions: 0644

I followed the instructions found in app/kibana#/home/tutorial/suricataLogs
IS NOT WORKING!!

Which version of filebeat have you installed?

Looks like can be related to the non-OSS version fo the Filebeat - Filebeat connection issue

I used the version that the /home/tutorial/suricataLogs suggests.
When i run filebeat setup I get this message. If i run filebeat -c filebeat.yml -e it seems that is running.
I have the same problem with Zeek also.
I think tutorial is insufficient .

@clouca, in order to be able to connect to Elasticsearch OSS you should use Filebeat OSS.
https://www.elastic.co/downloads/beats/filebeat - can connect to the Elasticsearch
https://www.elastic.co/downloads/beats/filebeat-oss - can connect to the Elastichsearch OSS

If you use Open Distro for Elasticsearch, it by default install Elasticsearch OSS and to be abble to connnect to it using Filebeat, you must use OSS version of it.

I used the correct versions but no success.
Now i get the error x509: certificate signed by unknown authority

You probably can try to configure Filebeat with appropriate certificates of the your Elasticsearch cluster like it is described in the documentation - Configure SSL.
You also can use ssl.verification_mode: none to check if it is related to your self-signed certificates.

I have already done that but seems that filebeat does not taking into account this option

What is in your output.elasticsearch is specified now?
From the documentation: Configure the Elasticsearch output:

output.elasticsearch:
  ssl.verification_mode: none
  hosts: ["https://localhost:9200"]
  # hosts: ["localhost"]
  # protocol: https

As you can find in my first post which i posted my config the ssl.verification_mode: none is already there. Every time I make a setup of opendistro different things happened. Cannot explain this instability.

From the documentation: Configure the Elasticsearch output:

output.elasticsearch:
  ssl.verification_mode: none
  hosts: ["https://localhost:9200"]
  # hosts: ["localhost"]
  # protocol: https

You a trying to connect to the Elasticsearch on localhost via http or https, where is your protocol specification?

Hey @cloucat,

not sure if that is still working but in case it doesn’t I wanted to just add one point which may help.

From your configuration file I see that you are missing one of the configuraiton:

setup.ilm.check_exists: false

as described here: https://opendistro.github.io/for-elasticsearch-docs/docs/troubleshoot/#beats .

Give it a try (just saying as I’m not able to reproduce the issue).

Best regards,