Packebeat Input

[kartone]

Hi, just installed Graylog 3.1.0+aa5175e and it works great.

I’ve been dealing with a few issues related to sending DNS log with NXLOG. I’ve been searching for solutions on issue with empty mandatory "short_message" field but everything I tried was uneffective.

So, I switched to Packetbeat as everyone seems suggesting. The environment is Windows 2012r2 and Packetbeat is on the latest version. Its configuration file is approved by the verification command .\packetbeat.exe test config .\packetbeat.yml -e and it outputs a good Config OK

With this in my pocket, I created a new Graylog Beats input sitting on port 5046, with no TLS, but when I run it, in the log I see:

2019-08-28T14:00:48.992+02:00 ERROR [AbstractTcpTransport] Error in Input [Beats/5d666cae63b2cc0299c9655f] (channel [id: 0x92b66b48, L:/10.5.1.5:5046 ! R:/10.5.1.10:61999]) (cause io.netty.handler.codec.DecoderException: java.lang.IllegalStateException: Unknown beats protocol version: 69)

I’ve been googling on this for days now…can somebody helps?

Thanks.

[jan (Jan Doberstein)]

what packetbeat version did you run?

[kartone]

Hi Jan, thanks for your support.

Actually I solved the problem. I’m writing the solution right here in the hope somebody would find it useful.

I found that the only way to have Packetbeat working with Graylog is to let it be run by Sidecar. So I needed to configure a new log collector manually (I found only winlogbeat, filebeat and nxlog template, despite on what I read on the website announcing version 3) and pushing this configuration:

# Needed for Graylog
fields_under_root: true
fields.collector_node_id: ${sidecar.nodeName}
fields.gl2_source_collector: ${sidecar.nodeId}
path:
  data: C:\Program Files\Graylog\sidecar\cache\packetbeat\data
  logs: C:\Program Files\Graylog\sidecar\logs
tags:
 - windows
packetbeat.interfaces:
 device: 0
packetbeat.protocols:
 dns:
   ports: [53]
output:
 logstash:
   hosts: ["graylog:5046"]

Input is of Beats type. I needed also to configure a processing pipeline to fill the source and message fields that were sent null from packetbeat collectors. Everything seems to work with these two rules:

rule "Fill source field"
when
    has_field("beats_type")
then
    set_field("source", $message.packetbeat_collector_node_id);
end

rule "Fill message field"
when
    has_field("beats_type")
then
    set_field("message", $message.packetbeat_query);
end

All Packetbeat messages are pushed in their reserved Packetbeat DNS stream and the Pipeline is applied only on this stream.

Do you suggest better ways to approach this?

Thanks a lot!