Standalone Pipeline "IDE"

Hello,

We manage several Graylog clusters; Hence, it would be pretty helpful for us to have a standalone (i.e. not part of a full Graylog installation) pipeline “IDE”, to be able to write and test rules easily. Does something of the sort exist, or is it under development somewhere? If not, how hard would it be to extract this functionality from the Graylog code?

Thanks in advance

2 Likes

Use a test server for this. Finally you should test it with multiple logs, so If I were you:

  • install a test system
  • set forward rule to the test system (only the necessary messages)
  • play with pipelines
  • copy it to the live system
  • remove the forward

That’s not really an option, I would like a permanent solution, not a temporary workaround.

it’s not a temporary workaround. If you would like to test without make performance/availability problem in your live system, you need a test system.
And also, I wrote many pipelines, but you can’t test it with 1 messages, you should test it with many massages. To get many and live messages the best if you get the live messages.

I also saw some solutions for test systems, when they forward about 50% of the live messages to the test system. In this case you can do whatever you want with the test system.

But it is your system, so do what do you want.

You’re right, it’s not a temporary workaround. It’s a part of the solution that should exist, and I plan to integrate something like it.

However, it isn’t what I’m looking for right here. I would like to have a development environment for pipelines: a syntax checker (like in the Graylog UI), and the ability to combine them and test them with a single (or a few) message.

Then, and only then, when I think that I managed to create a pipeline that does what I want, comes what you’re referring to: testing on a “live” test system, with a large number of messages, performance monitoring, etc. But that, IMO, shouldn’t be the first step.

In my dictionary it is the synonym of a test environment. Maybe it is my mistake.

I get what you are saying and I think it would actually be a great tool. Closest I’ve gotten is to write the rules in VSCode with the “java” syntax. Obviously it’s not exactly what you are asking for as I still have to copy the rules to a graylog system and send in another test message in order to see how it gets processed.

I wish I had a full answer, but let me know what you end up going with.

1 Like

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.