Lot of unprocessed messages

Graylog Central (peer support)
1

Hi,

I have an issue with my graylog.
A lot of messages aren’t integrate since the 22 August.
How can i do to integrate all unprocessed message ?

image
image.png725×118 8.01 KB

Please help

2

he @alban check your Graylog server.log why this happens.

I guess your Elasticsearch is having issues…

3

I have only this errors :

2018-08-27T10:27:22.605+02:00 ERROR [LookupDataAdapter] Couldn't start data adapter <abuse-ch-ransomware-domains/5b3639a41e32bb32fb57817c/@5ccf13f>
org.graylog.plugins.threatintel.tools.AdapterDisabledException: Abuse.ch service is disabled, not starting adapter. To enable it please go to System / Configurations.
        at org.graylog.plugins.threatintel.adapters.abusech.AbuseChRansomAdapter.doStart(AbuseChRansomAdapter.java:80) ~[?:?]
        at org.graylog2.plugin.lookup.LookupDataAdapter.startUp(LookupDataAdapter.java:59) [graylog.jar:?]
        at com.google.common.util.concurrent.AbstractIdleService$DelegateService$1.run(AbstractIdleService.java:62) [graylog.jar:?]
        at com.google.common.util.concurrent.Callables$4.run(Callables.java:122) [graylog.jar:?]
        at java.lang.Thread.run(Thread.java:748) [?:1.8.0_171]
		
2018-08-27T10:27:22.615+02:00 ERROR [LookupDataAdapter] Couldn't start data adapter <spamhaus-drop/5b3639a41e32bb32fb57817d/@106c98cf>
org.graylog.plugins.threatintel.tools.AdapterDisabledException: Spamhaus service is disabled, not starting (E)DROP adapter. To enable it please go to System / Configurations.
        at org.graylog.plugins.threatintel.adapters.spamhaus.SpamhausEDROPDataAdapter.doStart(SpamhausEDROPDataAdapter.java:68) ~[?:?]
        at org.graylog2.plugin.lookup.LookupDataAdapter.startUp(LookupDataAdapter.java:59) [graylog.jar:?]
        at com.google.common.util.concurrent.AbstractIdleService$DelegateService$1.run(AbstractIdleService.java:62) [graylog.jar:?]
        at com.google.common.util.concurrent.Callables$4.run(Callables.java:122) [graylog.jar:?]
        at java.lang.Thread.run(Thread.java:748) [?:1.8.0_171]

2018-08-27T10:27:22.629+02:00 ERROR [LookupDataAdapter] Couldn't start data adapter <abuse-ch-ransomware-ip/5b3639a41e32bb32fb578178/@6fef56d2>
org.graylog.plugins.threatintel.tools.AdapterDisabledException: Abuse.ch service is disabled, not starting adapter. To enable it please go to System / Configurations.
        at org.graylog.plugins.threatintel.adapters.abusech.AbuseChRansomAdapter.doStart(AbuseChRansomAdapter.java:80) ~[?:?]
        at org.graylog2.plugin.lookup.LookupDataAdapter.startUp(LookupDataAdapter.java:59) [graylog.jar:?]
        at com.google.common.util.concurrent.AbstractIdleService$DelegateService$1.run(AbstractIdleService.java:62) [graylog.jar:?]
        at com.google.common.util.concurrent.Callables$4.run(Callables.java:122) [graylog.jar:?]
        at java.lang.Thread.run(Thread.java:748) [?:1.8.0_171]
		
2018-08-27T10:27:22.639+02:00 WARN  [OTXDataAdapter] OTX API key is missing. Make sure to add the key to allow higher request limits.

2018-08-27T10:27:22.640+02:00 ERROR [LookupDataAdapter] Couldn't start data adapter <tor-exit-node/5b3639a41e32bb32fb57817b/@36749d8a>
org.graylog.plugins.threatintel.tools.AdapterDisabledException: TOR service is disabled, not starting TOR exit addresses adapter. To enable it please go to System / Configurations.
        at org.graylog.plugins.threatintel.adapters.tor.TorExitNodeDataAdapter.doStart(TorExitNodeDataAdapter.java:73) ~[?:?]
        at org.graylog2.plugin.lookup.LookupDataAdapter.startUp(LookupDataAdapter.java:59) [graylog.jar:?]
        at com.google.common.util.concurrent.AbstractIdleService$DelegateService$1.run(AbstractIdleService.java:62) [graylog.jar:?]
        at com.google.common.util.concurrent.Callables$4.run(Callables.java:122) [graylog.jar:?]
        at java.lang.Thread.run(Thread.java:748) [?:1.8.0_171]

What does that mean ?

4

That is related to the thread-intel plugin and the created lookup tables. Not related to your issue.

5

i saved directory /var/lib/graylog-server/journal/ , stopped elasticsearch and graylog and deleted directory journal.
Now, i don’t have anymore unprocessed messages.

But i would like to know if it’s possible that graylog can integrate files “index/log” waiting stored on journal directory ?

image
image.png664×673 10.7 KB

6

The journal holds all unproccessed messages. If you deleted the content and restart Graylog the messages are lost.

You should have tried to find the reason that the journal is not drained before doing that.

7

FS / var / lib / elasticsearch was complete (98% this morning). For me, it’s the root cause

8

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.