Looking for some help. it just may be my lack of understanding of how Graylog works and what is needed.
Summary - How do I configure Graylog to read non-syslog type logs? We have home-grown applications running on a Linux system (Redhat 7.4) that produces logs. The format and methodology for creation of these logs were defined in-house along time ago and don’t meet any current format standards. I can’t seem to get Graylog to read/input these logs. I’ve been able to follow the Graylog “tutorial” and get Syslogs to show up but not our application specific logs
Background - I’m running VMs running RHL 7.4. I’m using Graylog 2.4.3, ElasticSearch 5.6.8 and Filebeat 6.2.3. I did not download any of the side-car collectors.
ElasticSearch (5.6.8) is running fine, I can see the “harvester” running in the Filebeat logs. I’ve tried following the same steps for Syslogs to create inputs for our custom logs but selected “Beats”, but nothing shows up, no messages, input stream, etc. Is this an issue because I’m using Filebeat Version 6 and Graylog is not compatible with Filebeat version 6.x (same incompatibility with ElasticSearch 6) Do I need to install a previous version of Filebeat?
Or do I need to I need a side-car collector for custom logs? Please let me know if I’ve not provided enough/adequate information
Well, with what you’ve said to go off of, it’s hard to tell where this is failing. It sounds, from what you’ve said, like the logs aren’t being shipped at all. If that’s the case, and the logs aren’t being handled by a syslog daemon (syslog, rsyslog, syslog-n, etc.) that can forward the logs to the GL Input, then you need some form of syslog service to get them into GL. The collector-sidecar is just for managing the configuration of your 3rd party (often Windows-based) log collectors.
If the logs are already being logged with any of the above syslog services, they can be set up to ship the logs to a remote point. You will want to use the Raw/Plaintext Input type, and set up some Extractors to populate fields with useful data from the logs.
However, once you’ve got them shipping in, I feel like your job is just beginning. In my experience, custom log formats tend to be annoying to deal with. Often they will write multi-line log messages as individual log entries, leading to big problems with readability, since most of the time these lines all get written in the same second, and sent as individual log entries. So often times the lines will be completely out of order, with no way to automatically organize them. If it sends stack traces in this fashion, things will be tricky for you.
Another way to get this working, if the app doesn’t use any normal syslog service, would be to get them to add a GELF log output to the app code directly, and have it send them directly to a GELF Input on the GL box.
Thank you for your responses! It does answer my question. Sounds like the side-car collector isn’t going to help me. BUT you did give me some options I can present and/or try! Thanks!
as already written by @Grakkal you need a log shipper for your logfiles.
Filebeat is fine for that and can be configured manual to collect the logfile and ship it to Graylog. On Graylog you define a beat input and in the filebeat configuration you use a logstash output.
You might need to configure filebeat for you logfile (if multiline), but that should be straight forward.
After your logfile is ingested to Graylog you might want to extract the information you really need and normalize the data. This way you would be able to work with that data in the best way. But that normalization is a local and uniq to your environment.
Thanks I’ll give that a shot.
I’ve configured an “Input” as “Beat” and have filebeat running.
When you say “in the filebeat configuration you use a logstash output” are you referring to the filebeat.yml file where you specify the “output”? I currently have that set to “elasticsearch” and the “hosts” entry is set to the elasticsearch host and port 9200. Should that be set to my Graylog host and port 5044?
