Sending DNS Logs to GrayLog

[crazydiamond (Jimmy)]

Good Morning Everyone,
Fairly new Graylog user here. I installed Graylog on an ubuntu 22.04 server with the intention of it being a security log server. Logs from firewalls, dns and dhcp logs, etc. I have the firewall logs working with no issues. I have installed packet beat on my DNS/DHCP server and am trying to get the config correct for sending those logs. I uncommented the ports that I need to send and specified the device that would be doing the sniffing. Then when I go to the output section, this is where I am stuck a little bit. I see output sections for elastic search and logstash. The install of graylog that I ran (which is located here https://www.howtoforge.com/how-to-install-graylog-4-on-ubuntu-22-04/) installed elastic search and not logstash. The input I created in Graylog is looking for those logs to come in as GELF UDP and on port 1053. However, in the Elasticsearch output section, it pretty much specifies either http or https and wants the username and password as well as some kind of api key.
So my questions are as follows: Are http and https the only ways to send logs from packetbeat to graylog? Can I add an output section in the packetbeat.yml for Graylog and the specific port I need to send them to? Is there a better way to send those specific logs to Graylog?

Any assistance, pointers, or advice is greatly appreciated.

[m_mlk]

Hi @crazydiamond

Welcome to the Community!

This Sending in log data is usually a good page to read for cases like yours.

HTH

[crazydiamond (Jimmy)]

I did read that one. But it doesn’t answer those specific question i have. I was sure hoping it was there.

[m_mlk]

Sorry, I haven’t used Packetbeat at all…
I’d suggest you do a search for “packetbeat” in the forum…

HTH

[StefanAustin]

First,
Graylog 4 and Ubuntu 22.04 will work with some tricks, but it is not supported.

I never worked with packetbeat either, sorry. I used syslog, auditbeat or filebeat for such things.

[gsmith]

Hello @crazydiamond

Packetbeat on Docker/Virtual machine, etc…

Using Graylog 4.x.x, Opensearch 1.3, MongoDb 4.4.x on CentOS 7.
I installed GL-sidecar /w packetbeat for my Docker Containers. I have used WinlogBeat/Packetbeat for my Hyper-v servers.

@crazydiamond

So my questions are as follows: Are http and https the only ways to send logs from packetbeat to graylog?

Its perferred using TCP/TLS by means of sending data from pint A to point B.
The TYPE of data being sent the awser is NO,
Sending Network logs to Graylog example;

PacketBeat.yml

packetbeat.interfaces.device: any
packetbeat.interfaces.internal_networks:
  - private
packetbeat.flows:
  timeout: 30s
  period: 10s
packetbeat.protocols:
- type: icmp
  enabled: true
- type: amqp
  ports: [5672]
- type: cassandra
  ports: [9042]
- type: dhcpv4
  ports: [67, 68]
- type: dns
  ports: [53]
- type: http
  ports: [80, 8080, 8000, 5000, 8002]
- type: memcache
  ports: [11211]
- type: mysql
  ports: [3306,3307]
- type: pgsql
  ports: [5432]
- type: redis
  ports: [6379]
- type: thrift
  ports: [9090]
- type: mongodb
  ports: [27017]
- type: nfs
  ports: [2049]
- type: tls
  ports:
    - 443   # HTTPS
    - 993   # IMAPS
    - 995   # POP3S
    - 5223  # XMPP over SSL
    - 8443
    - 8883  # Secure MQTT
    - 9243  # Elasticsearch
- type: sip
  ports: [5060]
  _source.enabled: true
output.logstash:
  hosts: ["10.10.10.100:5066"]
processors:
  - # Add forwarded to tags when processing data from a network tap or mirror.
    if.contains.tags: forwarded
    then:
      - drop_fields:
          fields: [host]
    else:
      - add_host_metadata: ~
  - add_cloud_metadata: ~
  - add_docker_metadata: ~
  - detect_mime_type:
      field: http.request.body.content
      target: http.request.mime_type
  - detect_mime_type:
      field: http.response.body.content
      target: http.response.mime_type
logging.level: info
logging.to_files: true
logging.files:
  path: /var/log/packetbeat
  name: packetbeat
  keepfiles: 7
  permissions: 0640

With Fiewall, AD DC/DNS logs You can attach WinLogbeat /w Packetbeat on Graylog Sidecar.

EDIT: to make this more clear this section is my Graylog server ip addess and INPUT port number.

@gsmith

output.logstash:
  hosts: ["10.10.10.100:5066"]

If im readining your statment correct, I think you have Graylog sidecar also? Using just packetbeat installed, the above example should get you where you want to go. Perhaps a couple adjustments for your environment.

Example packebeat with filebeat. Filebeat (port 5044) is going to a different INPUT then Packetbeat (port 5066), because I have the INPUTS routed to a different index set.

[borjam (Borja Marcos Ea2 Ekh)]

You can use syslog or, depending on your DNS server software, DNSTap.

There is a DNSTap implementation witten in Go that can export JSON records which Graylog will parse directly.

[crazydiamond (Jimmy)]

Thanks very much for the gsmith. I think I am good on the indexes and as I said, the graylog server is working correctly thus far as I am sending Firewall logs to it and I can confirm they are there. The part I am wondering about is since logstash was not installed on my graylog server (the installation writeup I used included elasticsearch but not logstash). So will editing the logstash output section in the config file still send logs to the elasticsearch input port that I have created? In other words, if I have a packet beat input created on the graylog/elasticsearch server listening on port 1053 and i edit the logstash output in the yml to read x.x.x.x:1053, will that work? if so, does it change the format of the logs themselves? My apologies if these are just trivial questions, as I am not much of a linux/ubuntu user.

[tmacgbay]

When you use output.logstash: in your client side yml its just because Packentbeat doesn’t have a configuration for outpu.graylog: so the x.x.x.x:1053 port should match between the client ‘logstash’ and the Graylog Input. The packetbeat data is sent to the Graylog Input for manipulation (extractors, pipelines, alerts…) with the resulting data shunted to Elasticsearch/OpenSearch for storage and searchability. I don’t know id packetbeat changes the format off hand since I also don’t use it… beats log shippers will generally break out fields for you but usually retain the data nicely.

[crazydiamond (Jimmy)]

Thank you guys very much. It seems like maybe packetbeat isnt the way to go. You aren’t the first person I have heard say they don’t use or have never used packetbeat. I certainly don’t know anything about it. I only found it because I was researching ways to get the DNS/DHCP logs my Windows server for investigation purposes. Need to be able to search for particular queries without having to copy the log off the windows box and then search it every time I need to see where a query came from. So, I think I will drop back and punt and try to find a better more supported solution for this one.

[tmacgbay]

I played around with windows DNS logs where I had the service dumping to a log file and I was picking it up with filebeat. In the end it wasn’t really giving me what I wanted so it fell into a list of future-projects.

Just in case it can give you what you want, here is the simple sidecar configuration I was using to pickup the logs with filebeat:

# Needed for Graylog
fields_under_root: true
fields.collector_node_id: ${sidecaah.nodeName}
fields.gl2_source_collector: ${sidecaah.nodeId}
output.logstash:
   hosts: 
   - ${some_ip_here}
path:
  data: C:\Program Files\Graylog\sidecar\cache\winlogbeat\data
  logs: C:\Program Files\Graylog\sidecar\logs
tags:
 - DNS
filebeat:
  inputs:
    - type: log
      paths:
        - C:\Windows\Logs\DNSServer\dns.log