Hello,
I have an issue with the Logstash-OSS with Opensearch plugin. I keep getting the following error and it just loops in my terminal until I kill the process. I’m totally lost as to what could be causing the issue. According to the error the issue seems to be in the pipelines.yml file but I have compared what I have to the example in the pipelines.yml file, the documentation on the Opensearch and Elastic websites and even went as far as to look at the source code on Github and it matches up but I still get the error and the failed loop. Does anyone have any ideas to what I’m doing wrong?
Below is the DEBUG logs:
[2021-12-20T13:42:48,531][DEBUG][logstash.instrument.periodicpoller.jvm] collector name {:name=>"ParNew"}
[2021-12-20T13:42:48,531][DEBUG][logstash.instrument.periodicpoller.jvm] collector name {:name=>"ConcurrentMarkSweep"}
[2021-12-20T13:42:50,923][DEBUG][logstash.config.source.local.configpathloader] Skipping the following files while reading config since they don't match the specified glob pattern {:files=>["/home/user/logstash-7.16.1/config/conf.d", "/home/user/logstash-7.16.1/config/config", "/home/user/logstash-7.16.1/config/jvm.options", "/home/user/logstash-7.16.1/config/log4j2.properties", "/home/user/logstash-7.16.1/config/logstash-sample.conf", "/home/user/logstash-7.16.1/config/logstash.yml", "/home/user/logstash-7.16.1/config/startup.options"]}
[2021-12-20T13:42:50,924][DEBUG][logstash.config.source.local.configpathloader] Reading config file {:config_file=>"/home/user/logstash-7.16.1/config/pipelines.yml"}
[2021-12-20T13:42:50,925][DEBUG][logstash.agent ] Converging pipelines state {:actions_count=>1}
[2021-12-20T13:42:50,927][DEBUG][logstash.agent ] Executing action {:action=>LogStash::PipelineAction::Create/pipeline_id:main}
^C[2021-12-20T13:42:50,938][ERROR][logstash.agent ] Failed to execute action {:action=>LogStash::PipelineAction::Create/pipeline_id:main, :exception=>"LogStash::ConfigurationError", :message=>"Expected one of [ \\t\\r\\n], \"#\", \"input\", \"filter\", \"output\" at line 16, column 1 (byte 614) after ", :backtrace=>["/home/user/logstash-7.16.1/logstash-core/lib/logstash/compiler.rb:32:in `compile_imperative'", "org/logstash/execution/AbstractPipelineExt.java:187:in `initialize'", "org/logstash/execution/JavaBasePipelineExt.java:72:in `initialize'", "/home/user/logstash-7.16.1/logstash-core/lib/logstash/java_pipeline.rb:47:in `initialize'", "/home/user/logstash-7.16.1/logstash-core/lib/logstash/pipeline_action/create.rb:52:in `execute'", "/home/user/logstash-7.16.1/logstash-core/lib/logstash/agent.rb:383:in `block in converge_state'"]}
[2021-12-20T13:42:50,942][WARN ][logstash.runner ] SIGINT received. Shutting down.
Below is the snippet from the pipelines.yml:
# List of pipelines to be loaded by Logstash
#
# This document must be a list of dictionaries/hashes, where the keys/values are pipeline settings.
# Default values for omitted settings are read from the `logstash.yml` file.
# When declaring multiple pipelines, each MUST have its own `pipeline.id`.
#
# Example of two pipelines:
#
# - pipeline.id: test
# pipeline.workers: 1
# pipeline.batch.size: 1
# config.string: "input { generator {} } filter { sleep { time => 1 } } output { stdout { codec => dots } }"
# - pipeline.id: another_test
# queue.type: persisted
# path.config: "/tmp/logstash/*.config"
- pipeline.id: syslog.hosts
pipeline.workers: 1
pipeline.output.workers: 1
path.config: "/home/user/logstash-7.16.1/config/conf.d/03-syslog-input.conf"
- pipeline.id: syslog.switches
pipeline.workers: 1
pipeline.output.workers: 1
path.config: "/home/user/logstash-7.16.1/config/conf.d/04-switches-input.conf"
queue.type: persisted
- pipeline.id: beats
pipeline.workers: 1
pipeline.output.workers: 1
path.config: "/home/user/logstash-7.16.1/config/conf.d/02-beats-input.conf"
queue.type: persisted
# Available options:
#
# # name of the pipeline
# pipeline.id: mylogs
Below is one of the conf files, not posting all because the configs are more or less the same minus the obvious:
input {
tcp {
port => 5514
type => syslog
}
udp {
port => 5514
type => syslog
}
}
filter {
if [type] == "syslog" {
grok {
match => { "message" => "%{SYSLOGTIMESTAMP:syslog_timestamp} %{SYSLOGHOST:syslog_hostname} %{DATA:syslog_program}(?:\[%{POSINT:syslog_pid}\])?: %{GREEDYDATA:syslog_message}" }
add_field => [ "received_at", "%{@timestamp}" ]
add_field => [ "received_from", "%{host}" ]
}
date {
match => [ "syslog_timestamp", "MMM d HH:mm:ss", "MMM dd HH:mm:ss" ]
}
}
}
output {
opensearch {
hosts => ["https://localhost:9200"]
ssl => true
ssl_certificate_verification => false
cacert => "/home/user/logstash-7.16.1/config/config/root-ca.pem"
user => X
password => X
ilm_enabled => auto
manage_template => false
index => "logstash-hosts-%{+YYYY.MM.dd}"
}
}
Server Info:
Opensearch 1.2.2
Dashboards 1.2.0
Logstash-OSS from Opensearch 7.16.1