Winlogbeat templates for Elasticsearch not complete?


(Tess) #1

Wouldn’t you know it? Exactly two weeks after my previous thread, the same issue arises again. And I’m two hours too late to necro the previous thread.

This weekend the indices rotated and I’m back to looking at 18k indexing failures. And like before, it revolves around the “winlogbeat_event_data_param2” field, which is incorrectly typed in the new index.

I know. I was dumb. I should not have assumed that the Winlogbeat template for Elastic was actually complete. Turns out that the “winlogbeat.template.json” that I used (which came from the official Github) is missing fields like the aforementioned “winlogbeat_event_data_param2”. I’m not yet sure whether this particular field is custom to Graylog, whether the absence is intentional by the designers, or whether something else is up.

I’ll poke around and see what else I may have missed :frowning:

EDIT 1 :
Found this old thread… could be useful.

In reply to that thread @jan wrote:

As long as you do not have the need to that data, I would drop it.

Now if only I had a passing notion of what he meant by that :smiley: I have no clue how to tell Graylog or whomever it is that makes that field to stop making it :smiley:

#EternalNewbie

EDIT 2:
Hmmmzzz, I guess that brings us to @jochen’s suggestion in this thread:

Something with pipelines and removing the field. Guess that could work, eh?

EDIT 3:
Okay, so it seems that what I want is to stop Graylog from splitting the “message” into “winlogbeat_event_data_param[1-30]”.

EDIT 4:
Okay! Found this thread on the ELK forums, from September 2016.

Basically it seems that I can’t easily prevent Winlogbeat from creating the fields. So either I add them all to the “winlogbeat.template.json”, or I use a Graylog pipeline to dike’m out…

EDIT 5:
If I go the route of the template for Elastic, it seems that I could get it sorted with the following:

"event_data_param1" : {
  "type" : "text",
  "include_in_all" : false,
  "index": no
},

Or leave of those false/no if you do want them indexed, but it really doesn’t serve any use.


Grok patterns not able to parse numbers after index rotation
(Jan Doberstein) #2

look at the data and act on that like I have mentioned in the edit1 linked thread.

Drop if no need for the data.


(Tess) #3

ACK… I’ll try and figure out how to do that with a pipeline. Thanks Jan.


(Ben van Staveren) #4

remove_field or even drop_message are your friends :wink:


(Tess) #5

I swear, I’m totally not a robot…


(system) #6

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.