Missing fields host, protocol

[daniel]

Hi out there,

I am putting apache2 access logs via gelf with fluentd to graylog. Messages in the td-agent.log leave the host like this:

2019-07-30 08:22:25.000000000 +0200 apache.access: {"host":"x.x.x.x","user":"-","method":"GET","path":"/urz/nagvis/server/core/ajax_handler.php?mod=Multisite&act=getMaps&_ajaxid=1564467745","protocol":"HTTP/2.0","code":"200","size":"495","referer":"https://host/urz/check_mk/side.py","agent":"Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/75.0.3770.142 Safari/537.36"}

In graylog the host filed is missing and the protocol field always shows ‘0’. If I rename host to client ip address is shown correct. If I rename protocol to proto field has correct value ‘HTTP/2.0’ and protocol is still ‘0’.

What’s the reason for this and how can I use the fields ‘host’ and ‘protocol’?

[jan (Jan Doberstein)]

but your message is not a valid GELF message - what kind of Input did you have?

[daniel]

Its a default /var/log/apache2/access.log

[jan (Jan Doberstein)]

Sorry I need to be more clear - what kind of Input did you use on Graylog to ingest that messages?

[daniel]

Hi Jan,

on the webserver fluntd agent is running with
/etc/td-agent/td-agent.conf:

<source>
  @type tail
  path /var/log/apache2/access.log
  format apache2
  pos_file /tmp/access.log.pos
  tag apache.access
</source>
<match apache.access.**>
  type copy
  buffer_type memory
  buffer_chunk_limit 256m
  buffer_queue_limit 128
  flush_interval 1s
  disable_retry_limit false
  retry_limit 17
  retry_wait 1s
  <store>
    type gelf
    host (logserver)
    port 12201
  </store>
  <store>
    type stdout
  </store>
</match>

On Graylog the following input is configured als GELF UDP:

* bind_address:  0.0.0.0
* decompress_size_limit: 8388608
* number_worker_threads: 2
* override_source: *<empty>*
* port: 12201
* recv_buffer_size: 262144