Graylog don't write into elasticsearch anymore

[shinji7800]

Hello every one.

I recently have saturation on graylog. (var/ was full on my server). So I delete older indexes, from Graylog UI (System/Indices).

But graylog won’t write into elasticsearch anymore (out msg/s still at 0)

I try to recalculate index ranges and Rotate active write index but nothing change.

I tried to restart my server, nothing change to (mangoDB, elasticsearch and Graylog are running)

I don’t know what to do

Please forgive my approximate English I am not

Best regards

[jan (Jan Doberstein)]

he @shinji7800

what did you find in your server.log or the elasticsearch log? In addition what Graylog version did you use?

[shinji7800]

hello thanks for your reply

graylog-server/server.log don’t move

but 1 Gb of logs in 2 days for /elasticsearch/graylog.log

With in appearance only

All shards failed for phase [query]
java.lang.IllegalArgumentException: Expected numeric type on field [source], but got [text]

Graylog 2.3.2 on RHEL 7

[jan (Jan Doberstein)]

your issue might have multiple reasons, but you need to check them on your own.

• the Graylog journal might be corrupt because the filesystem was full
• the elasticsearch indices are in read only mode because the filesystem was full
• your elasticsearch server/data is corrupt
• the error message you have indicate that the elasticsearch server is not “green” - https://www.datadoghq.com/blog/elasticsearch-unassigned-shards/

[shinji7800]

thank you for give me hints to explore !

I let you know

best regards

[shinji7800]

hi

in Graylog UI/index set and via curl elasticseach is “green”

1 node
1 data nodes
16 active primary shards
16 active shards
the others value are 0
active shards percent as number 100.00

in graylog

evrything is green with this message :
4 indices with a total of {number going up every second} messages Under management

I Wonder if I is note better to erase all data …

[shinji7800]

for people can loose data …

I delete evey indices,
delete my extractors,
delete my host files,
and reload my server …

It works now