Good Morning Everyone,
Fairly new Graylog user here. I installed Graylog on an ubuntu 22.04 server with the intention of it being a security log server. Logs from firewalls, dns and dhcp logs, etc. I have the firewall logs working with no issues. I have installed packet beat on my DNS/DHCP server and am trying to get the config correct for sending those logs. I uncommented the ports that I need to send and specified the device that would be doing the sniffing. Then when I go to the output section, this is where I am stuck a little bit. I see output sections for elastic search and logstash. The install of graylog that I ran (which is located here https://www.howtoforge.com/how-to-install-graylog-4-on-ubuntu-22-04/) installed elastic search and not logstash. The input I created in Graylog is looking for those logs to come in as GELF UDP and on port 1053. However, in the Elasticsearch output section, it pretty much specifies either http or https and wants the username and password as well as some kind of api key.
So my questions are as follows: Are http and https the only ways to send logs from packetbeat to graylog? Can I add an output section in the packetbeat.yml for Graylog and the specific port I need to send them to? Is there a better way to send those specific logs to Graylog?
Any assistance, pointers, or advice is greatly appreciated.
With Fiewall, AD DC/DNS logs You can attach WinLogbeat /w Packetbeat on Graylog Sidecar.
EDIT: to make this more clear this section is my Graylog server ip addess and INPUT port number.
If im readining your statment correct, I think you have Graylog sidecar also? Using just packetbeat installed, the above example should get you where you want to go. Perhaps a couple adjustments for your environment.
Example packebeat with filebeat. Filebeat (port 5044) is going to a different INPUT then Packetbeat (port 5066), because I have the INPUTS routed to a different index set.
Thanks very much for the gsmith. I think I am good on the indexes and as I said, the graylog server is working correctly thus far as I am sending Firewall logs to it and I can confirm they are there. The part I am wondering about is since logstash was not installed on my graylog server (the installation writeup I used included elasticsearch but not logstash). So will editing the logstash output section in the config file still send logs to the elasticsearch input port that I have created? In other words, if I have a packet beat input created on the graylog/elasticsearch server listening on port 1053 and i edit the logstash output in the yml to read x.x.x.x:1053, will that work? if so, does it change the format of the logs themselves? My apologies if these are just trivial questions, as I am not much of a linux/ubuntu user.
When you use output.logstash: in your client side yml its just because Packentbeat doesn’t have a configuration for outpu.graylog: so the x.x.x.x:1053 port should match between the client ‘logstash’ and the Graylog Input. The packetbeat data is sent to the Graylog Input for manipulation (extractors, pipelines, alerts…) with the resulting data shunted to Elasticsearch/OpenSearch for storage and searchability. I don’t know id packetbeat changes the format off hand since I also don’t use it… beats log shippers will generally break out fields for you but usually retain the data nicely.
Thank you guys very much. It seems like maybe packetbeat isnt the way to go. You aren’t the first person I have heard say they don’t use or have never used packetbeat. I certainly don’t know anything about it. I only found it because I was researching ways to get the DNS/DHCP logs my Windows server for investigation purposes. Need to be able to search for particular queries without having to copy the log off the windows box and then search it every time I need to see where a query came from. So, I think I will drop back and punt and try to find a better more supported solution for this one.
I played around with windows DNS logs where I had the service dumping to a log file and I was picking it up with filebeat. In the end it wasn’t really giving me what I wanted so it fell into a list of future-projects.
Just in case it can give you what you want, here is the simple sidecar configuration I was using to pickup the logs with filebeat: