I’m moving an old Graylog instance from 2.2.3 to Graylog 4.0.5 + Elasticsearch 7.10.2. There’s a considerable number of servers running an old version of ‘collector-sidecar’.
The first issue I faced was that the names of the custom fields configured in Beats inputs get renamed with prefix
fields-app=system. The solutions offered here and here seem to apply to new sidecars only.
So, I tried to configure a new sidecar on one of the servers. sidecar.yml includes only the most necessary information:
server_api_token. Then, I created a configuration called ‘system’. I changed paths to logs, set up one field and changed the Graylog server URL:
# Needed for Graylog
- input_type: log
When I tried to run the new sidecar, it was assigned this configuration, but was in the ‘Failing’ state. I fixed one of the errors by replacing
-c %s in the log collector configuration with the exact path to
/etc/graylog/sidecar/generated/sidecar.yml, but it still fails with the message
Exiting: No prospectors defined.
I can go on with fixing the configuration, but somehow it doesn’t feel right. This is not what I read in Step-by-step guide. Should I go on fixing the errors as they appear or have I already lost my way?
where does this configuration came from? your sidecar.yml in your server or Graylog UI?
I’m asking because your error message is complaining about an option that is not in your configuration, I’m wondering if those files are the same.
This configuration is taken from Graylog UI (System - Sidecars - Configuration - my configuration - Edit).
As for sidecar.yml, it consists of only two lines, as I mentioned:
Here is the resulting filebeat.yml that causes the error, I assume:
As I suspected, those aren’t the same files. To work properly, it must be.
Via your graylog UI you could be able to send the right file to your server, or you can try to edit this file manually to see if it runs with no errors.
To edit your file manually keep in mind you need to replace this fields with the righ values:
Those informations you find in your Graylog UI following the path System > Sidecars
I think I know what I did wrong. Your answer helped me understand that:
- Changes in configuration can be applied
- collector-sidecar cannot be installed together with sidecar/filebeat, because collector-sidecar includes its own copy of filebeat
- filebeat configuration is not stored in
/etc/graylog/sidecar/generated/filebeat.yml, but in
However, I still can’t understand the difference between these two configuration files and why
/etc/graylog/sidecar/generated/filebeat.yml is not re-created after changes are made in configuration?
At this point, I think I can’t help you because I’m relative new with graylog to tell how things work under the hood.
Maybe @aaronsachs can.
On my ubuntu linux system, there is no directory
/etc/graylog/sidecar/generated. My guess is that is a legacy directory for the older collectors. I have sidecar 1.0.2 running and see the following in those directories:
# ls -l /etc/graylog/sidecar/
-rw-r--r-- 1 root root 36 Jul 23 2020 node-id
-rw-r--r-- 1 root root 3080 Jul 23 2020 sidecar.yml
This sidecar.yml is the core settings for sidecar configuration (server_url, server_api_token…etc…)
# ls -l /var/lib/graylog-sidecar/generated/
-rw------- 1 root root 539 Mar 31 09:49 filebeat.conf
This filebeat.conf is the configuration pushed from Graylog that tells the sidecar what and how to send stuff back to Graylog. Here is a simple one I built in our env that is pushed out from Graylog for you to compare:
# Needed for Graylog
- type: log
Here is what it looks like on the Graylog side:
Hope that helps to look at in comparison to your setup. If there are other parts you would like to see I can post.
I’ve just installed graylog-sidecar on another server and there are no files in /etc/graylog/sidecar/generated, you are right.
Thank you, folks, for your extensive answers.
This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.