Metricbeat not working through Logstash

Graylog Central (peer support)
1

Hello Everyone, i was wondering if anyone could provide some assistance in the following:

We have a setup in which we output from Multiple beats into logstash which outputs into an Elasticsearch Cluster, and Graylog Cluster simultaneously as an additional layer of redundancy. the setup is Beats > Logstash > Graylog / Elasticsearch. now all the beats besides Metricbeat can successfully send data and i can see the messages in the inputs coming in with no issues. i know that the beat can connect and send data, as when i point it directly to graylog and set up a Beats input and not a GELF UDP input it works fine. This is the configuration:

Metricbeat:

metricbeat.config.modules:
path: ${path.config}/modules.d/*.yml
reload.enabled: false

setup.template.settings:
index.number_of_shards: 1
index.codec: best_compression

setup.kibana:
host: “10.226.100.30:5601”

output.logstash:
hosts: [“logstash01:6969”]

Logstash

input {

beats {
port => 6969
}

}

output {

elasticsearch {

hosts => ["elasticnode01", "elasticnode02"]
index => "%{[@metadata][beat]}-%{[@metadata][version]}"
ilm_enabled => true

}

gelf {

host => "graylognode01"
port => 12201
short_message => "Output to Node 1"

}
gelf {

host => "graylognode02"
port => 12201
short_message => "Output to Node 2"

}

}

And for Gelf input i set up a Global GELF UDP input bound to 0.0.0.0:12201 and have turned off Firewalld and SElinux to see if the problem is there by any chance. All the machines are running on Centos7 if it makes any difference. Any information would be more than Welcome as i cant seem to figure out why this is happening with only one of the beats

2

did you checked the graylog server.log or the elasticsearch log (from the ES cluster that is used by Graylog) for error messages?

4

so what is in your index at the field [beats_type]?

You might want to split indices by ingest type or force with a custom template a specific store.

5

Sorry i didnt mention that i am really new with these systems so i dont know how to check the field in the index

6

Ok after some digning around, by pointing the beats to graylog directly i managed to find that [beats_type] filed holds a string metricbeat

7

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.