Removing beats prefix breaking timestamp fields

Greetings,

I have had Graylog outputting in GELF format and being collected by Logstash. However, I wanted to remove the winlogbeats_ prefix which I did using the checkbox but this has resulted in errors from logstash:

[2020-01-22T10:45:38,409][WARN ][logstash.inputs.gelf     ] Gelf (tcp): client socket failed. {:exception=>#<TypeError: wrong argument type String (expected LogStash::Timestamp)>, :backtrace=>["org/logstash/ext/JrubyEventExtLibrary.java:95:in `set'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-input-gelf-3.2.0/lib/logstash/inputs/gelf.rb:281:in `block in strip_leading_underscore'", "org/jruby/RubyArray.java:1792:in `each'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-input-gelf-3.2.0/lib/logstash/inputs/gelf.rb:279:in `strip_leading_underscore'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-input-gelf-3.2.0/lib/logstash/inputs/gelf.rb:161:in `block in tcp_listener'"]}

I set up a netcat listener to look at the timestamps fields being passed through, they are like this:

WITH PREFIX (works)

timestamp":1.579706245057E9
winlogbeat@timestamp":“2020-01-22T15:17:25.057Z
_timestamp”:"2020-01-22T15:17:25.057Z

WITHOUT PREFIX (doesn’t work)

timestamp":1.579703908066E9
_timestamp":“2020-01-22T14:38:28.066Z
_@timestamp”:"2020-01-22T14:38:28.066Z

I have tried to write a pipeline to try and remove any leading underscores or @ signs and normalise the field name but it has seemed to have no effect.

rule "Timestamp Field Rename"

when
has_field("@metadata_beat")
then
rename_field("@timestamp", “timestamp”);
rename_field("winlogbeat
@timestamp", “timestamp”);
end

Can anyone point me in the right direction?

Many thanks.

he @morpheus

multiple questions:

  • what Graylog Version did you use?
  • what winlogbeat version did you use?
  • did you ingest from winlogbeat direct to beats input?
    • is that a legacy input?

The error indicate, that the leading underscore is the problem. Why not remove that öeading underscore?

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.