Dedicated Logstash Cluster

Graylog Central (peer support)
1

Hi there,

currently I am using Logstash and the GELF plugin to send logfiles to Graylog.

So: Logstash -> GELF -> Graylog

Basically it works, but sometimes Logstash goes crazy and consumes almost all CPU resources - which is particularly bad on the production system. One reason for sure are very complex grok filters.

To take some load off the production machines and make them immune to Logstash hickups I’m thinking about makind a dedicated Logstash cluster.

So something like this:
Filebeat -> Logstash Cluster -> GELF -> Graylog

Question is: Shoud I put a message broker, e.g. Kafka or RabbitMQ, inbetween?

For example:
Filebeat -> Kafka -> Logstash Cluster -> GELF -> Graylog

Is there kinda reference architecture out there? It should be a fairly common problem and I don’t wanna reinvent the wheel.

cheers,
Frizz

2

Why not get rid of Logstash altogether? Graylog has a Beats input.

Filebeat (or any beat) --(Beats protocol)--> Graylog
3

Because I have lots of Logstash configuration files with complex filter definitions (grok, etc…).

I don’t wanna use Graylog extractors to do the (grok) filtering because
(a) there are lots of existing Logstash config files and it would be a huge effort to re-write them in graylog extractors.
(b) I’d have to upscale my graylog cluster (currently 3 nodes) to n nodes so that it can handle the additional grok filtering nodes. imho its cleaner from an architectural point of view to have a separate Logstash cluster to do this.

What do you think?

4

Both options you’ve suggested in your original post are viable and each has some advantages and disadvantages over the other.

FWIW, I’d go with the option with the least number of moving parts (i. e. Filebeat → Logstash → Graylog).

5

OK, understood. But to use multiple Logstash nodes I’d need some kind of load balancing inbetween, right? Maybe HAProxy or nginx?

So Filebeat -> LB -> Logstash(n nodes) -> Graylog

6

Filebeat supports load-balancing with its Logstash output.

1 Like
7

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.