Greetings,
I have had Graylog outputting in GELF format and being collected by Logstash. However, I wanted to remove the winlogbeats_ prefix which I did using the checkbox but this has resulted in errors from logstash:
[2020-01-22T10:45:38,409][WARN ][logstash.inputs.gelf ] Gelf (tcp): client socket failed. {:exception=>#<TypeError: wrong argument type String (expected LogStash::Timestamp)>, :backtrace=>["org/logstash/ext/JrubyEventExtLibrary.java:95:in `set'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-input-gelf-3.2.0/lib/logstash/inputs/gelf.rb:281:in `block in strip_leading_underscore'", "org/jruby/RubyArray.java:1792:in `each'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-input-gelf-3.2.0/lib/logstash/inputs/gelf.rb:279:in `strip_leading_underscore'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-input-gelf-3.2.0/lib/logstash/inputs/gelf.rb:161:in `block in tcp_listener'"]}
I set up a netcat listener to look at the timestamps fields being passed through, they are like this:
WITH PREFIX (works)
timestamp":1.579706245057E9
winlogbeat@timestamp":“2020-01-22T15:17:25.057Z
_timestamp”:"2020-01-22T15:17:25.057Z
WITHOUT PREFIX (doesn’t work)
timestamp":1.579703908066E9
_timestamp":“2020-01-22T14:38:28.066Z
_@timestamp”:"2020-01-22T14:38:28.066Z
I have tried to write a pipeline to try and remove any leading underscores or @ signs and normalise the field name but it has seemed to have no effect.
rule "Timestamp Field Rename"when
has_field(“@metadata_beat”)
then
rename_field(“@timestamp", “timestamp”);
rename_field("winlogbeat@timestamp”, “timestamp”);
end
Can anyone point me in the right direction?
Many thanks.