Is GrayLog Right for Me?

Graylog Central (peer support)
,
1

Hey Guys -

I have a fairly extensive home lab and wish to centrally manage / review log files for various applications & devices including:
Most Important

  • Text-based log files from multiple different applications (mostly built on Python)
  • ~x12 Windows OS Event logs (Mix of workstation & server)
  • LEDE / OpenWRT Router

Nice to Have but Not Required

  • x3 VMWare ESXi Servers
  • vCenter Server
  • SCCM Primary Server
  • Docker host (not incredibly important)

I’ve considered setting up Graylog using the OVA or Docker installation. Before doing so, I wanted to post to see if Graylog would be the best solution for me, if its overkill, or if I should use an alternative.

Any suggestions would be appreciated - Thank You!

2

if you think Graylog is overkill than every other solution for centralized log management would be too …

Just start using it and you could always destroy the setup and stop having a central log station if you find that it gives you no benefits and only work …

3

Yes… use Graylog. It’s a great product. Capable of doing what you described. Whether it’s right for you or not is up to you. It’s free… install it and try it out… OVA makes it easy, but do yourself a favor and skip the ova. If you decide to keep it, you’ll be glad you did.

4

Thanks, Guys -

Just set it up on vCenter via OVA, performed initial setup, and just have one question; please.

What input option would the best to add text-based log files produced by Windows applications? For example, below are 3 lines from one of the files (although their format can differ)

19-9-2 13:00:19.8|Info|ExistingSubImporter|Found 0 existing sub files
19-9-2 13:00:20.0|Info|ExistingOtherExtraImporter|Found 0 existing other extra files
19-9-2 13:00:20.0|Info|ExistingExtraFileService|Found 0 extra files

I’m looking through the marketplace, but wanted to ask first before taking the time to add all of them. Not looking for anything too advanced.

Thanks again!

5

Look into filebeat via the graylog sidecar

https://www.elastic.co/downloads/beats/filebeat

http://docs.graylog.org/en/3.1/pages/sidecar.html

6

Thanks -

I did that and set everything up to monitor a single log, but can’t get it to work. I created an API, installed Sidecar on the Windows Server, created a configuration, then assigned it to the Sidecar as documented. When starting the service, it says Running, but quickly changes to Failed. I can verify that the config I made was pushed to the sidecar. Below are logs and the config file. I’ll make another post if suggested:

Environment

  • IP of Graylog VM: 192.168.0.199/24
  • IP of server with sidecar: 192.168.0.35/24

sidecar.log

  • level=error msg=“[filebeat] Collector configuration file is not valid, waiting for the next update.”
  • level=error msg=“[filebeat] Validation command output: Exiting: no modules or inputs enabled and configuration reloading disabled. What files do you want me to watch?\n”

filebeat

  • ERROR fileset/modules.go:118 Not loading modules. Module directory not found: C:\Program Files\Graylog\sidecar\module
  • ERROR instance/beat.go:743 Exiting: no modules or inputs enabled and configuration reloading disabled. What files do you want me to watch?

sidecar.yml (Uncommented only)

server_url: “http://192.168.0.199:9000/api”
server_api_token: “q7mq81ino9okp7duq8j5k9nfsqjbvgpjntm7bqcac0o3a6i1u8o”
node_id: “file:C:\Program Files\Graylog\sidecar\node-id”
node_name: “kelnetmedia”
update_interval: 10
tls_skip_verify: false
send_status: true

filebeat.conf

fields_under_root: true
fields.collector_node_id: kelnetmedia
fields.gl2_source_collector: df03af40-381c-4756-8ff4-860091d323f3

output.logstash:
   hosts: ["192.168.0.199:5044"]
path:
  data: C:\Program Files\Graylog\sidecar\cache\filebeat\data
  logs: C:\Program Files\Graylog\sidecar\logs
tags:
 - windows
filebeat.inputs:
  type: log
  enabled: true
  paths:
    - C:\ProgramData\Application\logs\applog.txt

Notes

  • All paths within config files are valid
  • Initially had issue getting sidecar service to appear in Windows post install
  • Know comm between client and Graylog VM plus file permissions are ok since it pushed config to it successfully
  • Tried restarting sidecar service multiple times plus reinstalling running as admin
  • Tried changing Log On account for service to admin user then restarting with same results
  • Show Messages on sidecar displays nothing in web interface
  • Just saw that filebeat service isn’t starting

Any suggestions? Thanks

7

Nevermind - Got it to work by copying over some of the original filebeat files and playing with config. Thanks

8

glad to hear it… good luck.

9

only to have it said - you do not need to use the sidecar - that is just one option.

You only need to have a filebeat that collects a file and push that to a beats input at Graylog.

10

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.