Find the source of index failures

I’ve got almost 25,000 index failures. I’m quite sure there’s a grok failure somewhere but there’s not enough information provided to track down the source. I see others have had the same problem as far back as 2018. Has there been any progress on this? I can resolve them if I know where to look. Any help is greatly appreciated.

Can you post a bit more about what you are seeing? Relevant text from log files, screen shots… Where are you seeing the index failures? Post them so we can see the wording. Here are some tips on posting questions so that you give information about your environment - it includes commands to watch Graylog and Elasticsearch log files, among other tips and ideas…

This is one of the failures. We’ve got somewhere around 30-40 Grok extractors and there’s just not enough information provided to know where to even start. Thank you for your help.

a minute ago cor-logs_2654 a52f04f1-f1a6-11ec-b98c-005056857e23 {“type”:“mapper_parsing_exception”,“reason”:“failed to parse field [Time] of type [date]”,“caused_by”:{“type”:“illegal_argument_exception”,“reason”:“Invalid format: "17:10:56" is malformed at ":10:56"”}}

There is no built in way to track that down that I know of - I had a similar issue where I needed to hunt down an error in my pipeline rules and I ended up dumping the rules into a Content Pack (System-> Content Packs) and searching the resulting JSON file for key words associated with my issues… in your case it would be the field Time inside of a GROK.

1 Like

Yup. This is something Graylog needs to fix. It’s going to take quite a while to look at all the extractors we’ve got. All Graylog would have to do is to provide the raw log with the error. That seems like a fix that would take a developer 10 minutes. Thank you for your replies. Cheers mate.

You can post into Github – even though development always longer that you/I want, I know they do look at the posts and triage issues pretty quickly. I have seen some of my ideas come to (partial) fruition… :smiley:

1 Like