Collector-sidecar configuration provisioning

[th3n3rd (Marco Garofalo)]

Hi,

I am a fan of “automate all the things” and I am currently evaluating graylog for some application. I am using the dockerised version on my local machine but anyway my problem, as per subject, is around automating the client/collector configuration provisioning.

Currently the collector relies (almost) completely on the server configuration, even though in my opinion the “client” is the one that should “know” what is going to send to the server.

If I got it correctly the configuration for the collector stays all on the server and will be fetched by tag from the collector, but as of now I cannot see any way to export/import the configurations and I really want to have a disposable environment to work with that I can setup on the fly and be ready to work.

It would be nice to “provide” the collector’s inputs/outputs configurations rather then having to input them in the server UI

I have also tried to go through the API but it actually requires to fire multiple requests for each configuration, plus all the inputs/outputs, and it might require to link requests depending on the returning entity IDs.

I was wondering if you have any suggestions on how to automate the collector configuration provisioning.

Thanks.

[jochen]

Issue: Move collector logic to graylog-server

[th3n3rd (Marco Garofalo)]

I am not quite sure the Issue/Feature requested in there is related to what I am doing.

IMHO the client/collector should be less dumb actually, and it should be able to be configurable from a local source (providing a path to a local configuration for filebeat, etc) as well as a remote source (fetching configuration from the server as it’s doing right now).

That said I eventually develop a python script firing API calls to populate the server on demand before starting the collector.

[jochen]

@th3n3rd

IMHO the client/collector should be less dumb actually, and it should be able to be configurable from a local source (providing a path to a local configuration for filebeat, etc)

This would kind of make the whole Collector Sidecar obsolete.

If you want to use your own configuration management (Puppet, Chef, Ansible, etc.) to configure the log collectors, you can certainly do so and don’t need the Collector Sidecar.