No logs displayed in streams

Hello.

I have a Graylog 3.0.2+1686930, single node with ES 6.8.

Logs are shipped this way: filebeat -> kafka -> graylog (input -> all messages -> pipelines -> route to 2 streams)

Shipping works well, i see the logs in the kafka topic.

Graylog accept and index the log, i can see the logs in elasticsearch. But since 2019-07-22 19:55:25.844 nothing appears in All Messages stream or in “show received message” in the input.

I have this message log from that time:

2019-07-22T19:55:25.610+02:00 INFO [AbstractRotationStrategy] Deflector index (index set <graylog_4>) should be rotated, Pointing deflector to new index now!
2019-07-22T19:55:25.970+02:00 INFO [MongoIndexSet] Cycling from <graylog_4> to <graylog_5>.
2019-07-22T19:55:25.970+02:00 INFO [MongoIndexSet] Creating target index <graylog_5>.
2019-07-22T19:55:26.222+02:00 INFO [Indices] Successfully created index template graylog-internal
2019-07-22T19:55:26.375+02:00 INFO [MongoIndexSet] Waiting for allocation of index <graylog_5>.
2019-07-22T19:55:26.547+02:00 INFO [MongoIndexSet] Index <graylog_5> has been successfully allocated.
2019-07-22T19:55:26.547+02:00 INFO [MongoIndexSet] Pointing index alias <graylog_deflector> to new index <graylog_5>.
2019-07-22T19:55:26.631+02:00 INFO [SystemJobManager] Submitted SystemJob [org.graylog2.indexer.indices.jobs.SetIndexReadOnlyAndCalculateRangeJob]
2019-07-22T19:55:26.631+02:00 INFO [MongoIndexSet] Successfully pointed index alias <graylog_deflector> to index <graylog_5>.
2019-07-22T19:55:56.641+02:00 INFO [SetIndexReadOnlyJob] Flushing old index <graylog_4>.
2019-07-22T19:55:57.122+02:00 INFO [SetIndexReadOnlyJob] Setting old index <graylog_4> to read-only.
2019-07-22T19:55:57.142+02:00 INFO [SystemJobManager] Submitted SystemJob [org.graylog2.indexer.indices.jobs.OptimizeIndexJob]
2019-07-22T19:55:57.156+02:00 INFO [OptimizeIndexJob] Optimizing index <graylog_4>.
2019-07-22T19:55:57.157+02:00 INFO [CreateNewSingleIndexRangeJob] Calculating ranges for index graylog_4.
2019-07-22T19:55:58.887+02:00 INFO [MongoIndexRangeService] Calculated range of [graylog_4] in [1729ms].
2019-07-22T19:55:58.890+02:00 INFO [CreateNewSingleIndexRangeJob] Created ranges for index graylog_4.
2019-07-22T19:55:58.891+02:00 INFO [SystemJobManager] SystemJob [org.graylog2.indexer.indices.jobs.SetIndexReadOnlyAndCalculateRangeJob] finished in 2259ms.
2019-07-22T19:58:25.523+02:00 INFO [SystemJobManager] SystemJob [org.graylog2.indexer.indices.jobs.OptimizeIndexJob] finished in 148380ms.

I suppose this behaviour is linked to the index rotation.

I had to recalculate index range in the index set to make it work again.

I have this messages in the log when i clicked:

2019-07-23T10:23:25.324+02:00 ERROR [Cluster] Couldn’t read cluster health for indices [graylog_*] (Could not connect to http://127.0.0.1:9200)
2019-07-23T10:23:25.324+02:00 INFO [IndexerClusterCheckerThread] Indexer not fully initialized yet. Skipping periodic cluster check.
2019-07-23T10:23:45.383+02:00 INFO [MongoIndexSet] Did not find a deflector alias. Setting one up now.
2019-07-23T10:23:46.194+02:00 INFO [MongoIndexSet] Pointing to already existing index target <graylog_5>

Is this some part of the configuration i missed? I don’t remember changing something related.

Thanks for any pointer

I guess your Elasticsearch has issues - you should check your Elasticsearch:

2019-07-23T10:23:25.324+02:00 ERROR [Cluster] Couldn’t read cluster health for indices [graylog_*] (Could not connect to http://127.0.0.1:9200)

indicates that Graylog could not reach elasticsearch.

I restarted it minutes before I clic to “recalculate index range”. ES is ok, graylog feeds correctly messages in it.

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.