At first we didn’t use collector on the first few servers and instead just ran NXLOG by itself. After seeing the power and benefits of using the collector we now use it EVERYWHERE. It just makes managing hundreds of instances so much easier. So yes. Install collector and NXLOG on your the servers you want to ship logs from. You can ship them raw but that would mean you would have to create extractors for all the fields you need in graylog itself. This can be a resource intensive operation for graylog, especially if not keeping regex and GROK pattern usage under control. As such we try to keep the need for extractors to a minimum. Do this by having NXLOG do the work for us as a log shipper. In a perfect world all logs would be in an easily parsed format like JSON but unfortunately this isn’t always possible. (IIS,NGINX,etc) NXLOG helps with this by not only shipping your local logs to graylog but it can also reformat the outgoing messages and output them in whatever format you require. I’d recommend JSON or GELF as these are natively supported and easily parsed by Graylog as the fields are predefined before being shipped. On the graylog side you would create an input that you point the NXLOG to output to. Say like in our case NXLOG reformats our IIS logs to JSON and outputs GELF to our Graylog GELF input at whatever IP and Port you set. I recently shared an NXLOG config in another post to help out another member with the IIS log shipping via NXLOG. Here is a link to that post from last week. NXLOG IIS CONFIG That config is a very good starting point for IIS but the same concepts apply to using it to ship any other type of log. Once logs start flowing in you should find it super simple to start searching and creating dashboards/reports in graylog for whatever you need to pull out of them. Let us know how it goes. If you have any more questions just ask.
2 Likes