Logstash Custom Config filter (types)

Open Source Elasticsearch and Kibana
1

Hi there,

we have some problems with logstash configurations which was working quite well under the logstash version from elastic directly.

if [type] == "syslog" {

        output {
          opensearch {
            hosts => ["https://opensearch-node1:9200", "https://opensearch-node2:9200"]
            index => "syslog-%{+YYYY.MM.dd}"
            ssl => true
            ssl_certificate_verification => false
            user => "xx"
            password => "xx"
          }
        }
}

With a logstash docker container (image opensearchproject/logstash-oss-with-opensearch-output-plugin:7.13.2) we got the following error:

logstash | [2021-11-15T10:07:20,566][ERROR][logstash.agent ] Failed to execute action {:action=>LogStash::PipelineAction::Create/pipeline_id:main, :exception=>“LogStash::ConfigurationError”, :message=>“Expected one of [ \t\r\n], "#", "input", "filter", "output" at line 1, column 1 (byte 1)”, :backtrace=>[“/usr/share/logstash/logstash-core/lib/logstash/compiler.rb:32:in compile_imperative'", "org/logstash/execution/AbstractPipelineExt.java:187:in initialize’”, “org/logstash/execution/JavaBasePipelineExt.java:72:in initialize'", "/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:47:in initialize’”, “/usr/share/logstash/logstash-core/lib/logstash/pipeline_action/create.rb:52:in execute'", "/usr/share/logstash/logstash-core/lib/logstash/agent.rb:389:in block in converge_state’”]}

Are there new “if” filter types or is there any replacement available? We face a similar problem with the type “beats”.

If the first and last line is commented, that logstash service is starting normally.

Regards

2

Nothing new/changed should have been introduced. That error message doesn’t look like it’s from the output plugin.

Are you migrating versions of Logstash?

3

If i remove the filter (see quote) logstash is starting normally. So the problem is not the output, it is the filter.

4

The important part of the error message is:

Expected one of [ \t\r\n], “#”, “input”, “filter”, “output” at line 1, column 1 (byte 1)

In this case, an output block should be defined and everything else then goes inside it:

output {
  if [type] == "syslog" {
    opensearch { ... }
  }
}
5

Thats it. Thank you. For wthatever reason this was changed at our end…

1 Like