I see that this topic has been covered a couple of times. And in each time I go through thread I replicate the solutions/checks and still have this issue.
a) Is it normal for file = /etc/graylog/collector-sidecar/generated/filebeat.yml to be missing after installing collector-sidecar 0.1.6? Is it OK if I take filebeat config from forum and customize it. Then replace the missing filebeat.yml?
b) My config for tags is as below. The tags show in Graylog 2.4 appliance I am using for demo. Does this mean this is OK?
Yes this is normal, because in my experience the Sidecar generates the file for you, based on the configuration you make in Graylog. As Jan points out, you need to first do some config in the Graylog GU.
Define a collector config, in System > Collectors > Manage configurations.
For example, a config called “Windows”, with the tag “windows”.
You edit the config by setting up an output (for example Winlogbeat targeted at the Graylog server) and an input (for example Window Event Log).
Once that is setup, you edit the Sidecar config file and i the tags section you add the new “windows” tag.
Once the Sidecar gets restarted, it’ll pick up that it has a config to retrieve. It’ll pull it from Graylog and then arrange the local filebeat/winlogbeat config.
@Totally_Not_A_Robot is it ok if I configure the collector then configure the GUI with the tags? Restarting the collector should then get it to collect the config?
As long as the collector sidecar is configured with tags, when you create the configs in the Graylog UI they will automatically be synched within a minute or so and the collector will restart filebeat and start sending. No need to restart the collector - unless, of course, you added tags to it’s configuration
So I have followed step by step. Not sure why node-id was an issue. Filebeat is failing to start in background. Graylog Server is not generating the required filebeat.yml file. Is it reasonable to create my own filebeat.yml as workaround? It seems like it will never be overwritten.
you latest error loglines give a clear picture … you have not configured anything in the Graylog UI OR you just have not have any tags given to the configuration that matches the tags you have in the sidecar configuration at the server.
From below image I got the impression that the tags have been configured as per config files above. “linux” and “ussdrica”. The names I configured in collector*.yml automatically showed up in the GUI.
This step 6 is missing - because you need to assign a tag to the configuration that Graylog knows what configuration should be written to what sidecar.
Thanks @jan, from the below, it may seem like I added the tags but the below picture clearly disagrees. So I need to spend more time on this step. Also I note that each time I look for the tags, I also see that there is no auto-suggest for the tag, while it shows the “windows” tag of a successful sidecar collector I did for windows. I will try again
Thanks @Totally_Not_A_Robot. I will also review those steps. Hopefully I do not need to specify the tags in other steps too (besides the above mentioned and the collector*.yml)
I made a mistake I made before. Perhaps it is because I last did a config about 6 months ago. I did not CREATE the tags. I only added the tag words and pressed update. I actually need to insert tag name, create option and then select update.
I have learnt a lesson I will never forget. Many thanks for your assistance.
