I am running Suricata on a PfSense 2.4.5-p1 firewall. Suricata is configured to output logs to EVE JSON, type Syslog. Under logging settings, “Send alerts to System log” is checked, the PfSense successfully sends these logs to Graylog (4.0.1) on a Syslog UDP Input. All is received fine.
However, there are three distinct ‘message’ files: ‘full_message,’ ‘json,’ and ‘message.’ I’ve noticed that some other sources end up with a ‘full_message’ and ‘message’ field as well.
This obviously adds a lot of storage space and seems repetitious. What is the best practice for dealing with this? Leave them all, leave only one (if so which)? Would you use Pipeline rules to drop the unwanted message fields, or Extractors? Any help or suggestions (including if you have them any actual rules) would be greatly appreciated.
full_message field is stored, if you enable it on Input configuration in graylog: → System - Inputs - Edit Input, checkbox: Store full message? If you don’t want to store full_message field, just uncheck it in input configuration.
There is no best practice for it. If your company have requirements (policy, government, certification) to save also full_message (which is full syslog message before parsed to another fields, like message, level, source) then enable it. If you want to save storage, just disable it.
