Hi, Got an issue. I want to search my imported logs in Graylog by the timestamp of the logs themselves, not when Graylog received the log via syslog. How would I be able to do this? Thanks in advance

You would probably have to parse the time/date fields within the data as it comes in to change the date. If you can see the date/time in the message you should be able to do some basic text searching without having to parse/reimport.

@rmdashrf is correct. If the timestamp is not parsed from the syslog message itself (it should be though), you have to parse the timestamp manually, using our Processing Pipelines .

Some example based on @lennart ’s comment. I struggled 4 hours just to figure that out. :smiley: Hope this will save some time for others. E.g. I stored event timesamp value in event_ts, from apache combined log. I extracted with Grok. Processing pipeline to parse the values I groked and and stor…

Ye, Thanks for posting this. We currently do all of this in logstash/nxlog before the logs go into Graylog, and hadn’t really considering doing it the way you described. I don’t know if it’s proper or not, but it could reduce our need to use more complicated logstash configurations. Dustin Tennil…

@dustintennill You are right! We definitely recommend going with the way described by @yett . @yett , thanks for the example! Looks great. :slight_smile:

@lennart , Thanks for the pointer. I was bit unsure if I am on the right direction. Cheers.

Just to understand the thinking here: Does this mean that the extractors and converters are not the preferred method of handling raw input data (such as lines from log files)? Are they on the way out?

My timestamp still isn’t being set. I’ve tried the above parser as well as: rule "parse ACCESS log timestamp" when true then let new_date = flex_parse_date(to_string($message.event_ts)); set_field("timestamp", new_date); end The timestamp is still set to the time the message…

I think it should be set_field("timestamp",$new_date); I am also interested in how this works - I have had quite a lot timestamp problems, and still get occasionally failed indexing, due to wrong timestamp format.

The parser generates an error if I add the $new_date. [image]

Searching imported logs by log timestamp, not time Graylog received the log

Graylog Central (peer support)

Topic		Replies	Views
Use Timestamp of logfiles Graylog Central (peer support)	3	4729	August 9, 2018
Parse original log timestamp into Graylog Graylog Central (peer support) pipeline-rules , debuggingpl	0	745	August 14, 2020
Replacing default Graylog timestamp via Pipeline Processor Graylog Central (peer support) pipeline-rules , debuggingpl	12	11635	March 11, 2019
Changing the graylog "load" timestamp to the message log timestamp - solved Graylog Central (peer support) pipeline-rules	1	1760	November 9, 2018
Pipeline parsing and setting correct timestamp Graylog Central (peer support) pipeline-rules	11	13670	July 20, 2017

Searching imported logs by log timestamp, not time Graylog received the log

Related topics