Searching imported logs by log timestamp, not time Graylog received the log

Graylog Central (peer support)
,
14

Trying to catch up here - Same goal: Apache logs provide message timestamp, but Log Time is Graylog-ingested timestamp.

Out of the box COMMONAPACHELOG grok pattern parses the date field just fine into its own fields by out of the box %{HTTPDATE:timestamp} but timestamp is not set. I thought maybe I require the ^ pipeline rule, but testing pipeline looks like it’s not running thru the assigned grok pattern: Should it? There’s no explicit selector for it nor any indication that it tried or didn’t…

My method of testing is:

  • Defined Input for this specifically
  • Set custom grok expression on this input only
  • Set up pipeline with all the above
  • Copy/paste raw string into Simulator and Raw String as codec
    Message comes up “no changes” and timestamp shown in simulator results is always current time.

I suppose a better question is: With Graylog 2.3.1+9f2c6ef and standard Apache log files, what’s the expected Log Time timestamp? Ingested or message-provided? And is the above recipe required to get Log Time to be message-provided timestamp? How best to test/confirm?

Thanks

[update]
I have narrowed it down to that I can’t access $message.time (the json extractor shows me there’s a field “time”). I get the error message Invalid format: "".

Is there a way to inspect a pipeline rule? Console log? Dump $messages and see what I have access to?
(moved this part of my followup question for more eyeballs)
[/update]

[update]
In my case, the $message object was indeed not getting populated, because the order of message processors in Graylog config had Pipeline rules fire first and -then- JSON extraction second:

[/update]