Is the simulator useless for Beats inputs?

Graylog Central (peer support)
, , ,
1

I’m trying to write pipeline rules to process messages received into Graylog from Beats.

I don’t have a way of seeing the Beats raw message which I can then input into the simulator to test the rules.

Anyone know a way around this or some example Beats messages that I could adapt?

This has been raised as an issue but in classic Graylog fashion its been open since 2016!

2
  1. Beast is automatically parsed to fields, so there is no raw input. But you can still run tcpdump on graylog box to record it.
  2. I usually use pipeline function debug() to write debug message to log
    https://docs.graylog.org/en/3.3/pages/pipelines/functions.html#debug
    Afterthat check output of log line PIPELINE DEBUG::
    sudo tail -f /var/log/graylog-server/server.log
  3. There is also special field gl2_processing_error which contains name of pipeline rule and line with error message if pipeline has a problem. Only show this field in left menu ico X1 (Field) - a click on all including reserved fields and add to table. You can also use search pattern to find all pipeline errors: _exists_:gl2_processing_error
  4. And also graylog log file contains error message if something goes wrong with pipeline rule:
    sudo tail -f /var/log/graylog-server/server.log

Hope it helps

3

Thanks, I was worried that would be the case, I’ve got something in the region of 1000 messages a second coming in so tcpdump is going to be a real pain.

You don’t happen to have a winlogbeat tcp dump lying around do you? I’d just modify the fields to match what I want for testing the rule.

4

Best way to debug something like winlogbeat is to create own index with log retension, and send all test logs to it, so you will not broke index of real data.

5

That’s the thing though, I’m not really debugging. I’m writing new rules.

I shouldn’t have to write a rule, wait for it to fail, grab logs, check logs, rewrite rule and then repeat until it works.

I should be able to debug during the creation of the rule, test it and then publish a working rule first time rather than having to go through the hassle of debugging by default.

It’s one of the things that’s really frustrating about Graylog, it seems to be perpetually unfinished!

6

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.