Graylog GELF warning "is missing mandatory "host" field"


(Hass Sea) #1

I saw something related in this post below however a couple of items are not clear.

I’m using a logstash plugin to pull Azure NSG Flow logs from Azure storage accounts and then forward them to Graylog. Everytime the forward happens my Graylog server log is filled with:

2018-02-01T12:20:55.232-05:00 WARN [GelfCodec] GELF message <42bd8ec5-0774-11e8-ace9-000d3a18d769> (received from <172.18.4.17:50998>) is missing mandatory “host” field.

I tried adding a “Add static field” within the Graylog Input section but this seemed to make no difference and it’s not clear what Graylog is actually looking for and maybe I should us a different input type?

FYI, The Logstash Plugin information is below"

Thanks

Hass


(Jochen) #2

That’s just a warning and your messages are being processed nonetheless (given that there are no error messages following that warning).

The static fields are “too late”. The GELF codec runs to decode GELF messages received via the network and create Graylog messages from them. The static fields are added to the Graylog created messages, not the received GELF messages.

If you’re using Logstash, you could add a field named “host” which contains the desired value in the configuration of the GELF output:
https://www.elastic.co/guide/en/logstash/5.6/plugins-outputs-gelf.html#plugins-outputs-gelf-custom_fields


(Hass Sea) #3

Thanks Jochen, I’ll test that


(123dev) #4

We have a similar problem with code using pygelf.
Somehow the host field is missing, and adding the host field into the code using pygelf did not make a difference.

Also would drool help in this case, or that would be too late as well?

Is there any easy way to see exactly what the GELF input is receiving without resorting to tcpdump?

Thanks


(Jochen) #5

The Drools filter also runs at a later stage.


(Hass Sea) #6

Just a heads up based I have now resolved this by adding a new field “Host” field during my logstash transform which has cleared the warning:

Example below:

filter {
	mutate{
		add_field => {
					"host" => "Enter-Host-Name-Here"
						}
			}
		}

(system) #7

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.