Graylog2 start again or not? (Solved)

Graylog Central (peer support)
1

Hi all, i got my graylog slow, and broken. a lot of issues:

cluster shards : 60 and unassigned shard: 60. I going arround but i can’t found how to solve it.

Journal always to high, and a can’t solved it.

and others things.

Maybe i should installing again, but what i doing wrong to get this two errors?
There is a good practices to maintain graylog healtly?

Thanks! Have a nice day.

2

Hi,

can you tell a little bit more about your environment? Is your Graylog environment just a single machine? Are you using the graylog virtual appliance or have you separated the Graylog server from the elasticsearch data node?
It seems to be due to high I/O that the elasticsearch database cannot process the log events quickly enough so that the Graylog server has to buffer those locally in the message journal.
But, I’m by no means an expert, so please take this with a grain of salt.

How much data are you currently ingesting into your graylog environment?
You might also want to check if you’re running out of diskspace, it could be that your message journal is low on diskspace. If you haven’t changed this, than the message journal will be located in: /var/lib/graylog-server/journal

If it’s not there then you can look up the exact path in your graylog server.conf

cheers,
theresa

3

slow processing can also happen due to inefficient extractors/pipeline functions.

4

Also; if you have only 1 data node, you must have number of replicas = 0

5

Sorry if my question is to wide, i’m new on these things

Hi theresa, is Graylog a single machine, virtual appliance (2 core, 2ghz, 4GB of ram and 40GB of disk) nothing separated of this.
The log traffic is about 2 or 3 GB per 24hs. About the disk space i make indices about 1GB (15 indices)

jtkarvo,

i got a lot of regex extractor, somethings like this : regex_value: .(msg=“SSL tunnel shutdown”)

I don’t know how see replicas. I have just one node. The quantity of nodes, depends of what?

So, make it clean, if i want to “reset” and kill all problems (shrads and etc) all without lose extractors and alarms. What sould i do.

6

What do the graylogs server log and elasticsearch log look like on the machine? Any error messages?

In a normal setup, the number of replicas is set in the graylogs server.conf file; I don’t know about the appliance.

You can see if an extractor is slow by lookin in the “details” from the manage extractors page. It has timing information.

7

In tail -f /var/log/graylog/elasticsearch/current i don’t see errors

In tail -f /var/log/graylog/server/current i don’t see errors neither.

But i stiil seem slow :

Hi jtkarvo.

$ cat /opt/graylog/conf/graylog.conf | grep replicas
!#How many Elasticsearch shards and replicas should be used per index? Note that this only applies to newly created indices.
elasticsearch_replicas = 1

Here is one of my extractor:

1145×298 38.5 KB

Sorry, i see this realy dirty, if i want to “reset” and kill all problems (shrads and etc) all without lose extractors and alarms. What sould i do?

Thanks!

8

First, change elasticsearch_replicas = 0
then reboot. After that there will be no more new unassigned shards. It also easy to get rid of the old unassigned ones, it is a single command line command. If you don’t find the answer with google, I’ll post sunday/monday.

The regex looks like it is very good normally, but the maximum number is big. It is well possible that the real problem is in extractors, so that they sometimes hang and need to be optimized.

9

I follow all the steps, the unassigned_shards are decremented when the index was renewing.

Now works better and i got:

  • “status” : “green”,
  • “unassigned_shards” : 0,

Thanks a lot!

10

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.