Any best practices how to take in syslog from multiple different vendors that have each a bit different way of sending it. Like FortiGates have the data in key=value pairs which is different from Cisco switches, timestamp are in different format etc…
Our first try of Graylog uses different ports for different vendors, then adds a static attribute which we again match in streams. Each stream matches logs from certain vendor and puts them in their own index. Idea was that each index wouldn’t contain that many different fields, though not sure if it’s a problem with Elasticsearch at all? Other that as the “date” field might be in different formats it would make analyzing the logs harder.
Now that I’ve looked at couple more expensive product, they seem to take in al the log in on standard 514 port and then parse all the dates and timestamp etc to a common format once they recognize what the device is.
Is there a reasonable way in Graylog to do similar? For example doing groups of source IP addresses and map them to different streams? Should I store all the syslogs in one set of indices, or continue creating different indices for different vendors?
Thanks for any ideas!