# Timestamp Replacement Issues (Ingestion timestamp vs log timestamp)

**URL:** <https://community.graylog.org/t/timestamp-replacement-issues-ingestion-timestamp-vs-log-timestamp/9842>\
**Category:** Graylog Central (peer support)\
**Tags:** pipeline-rules, time-stamp-issuespl\
**Created:** [April 5, 2019, 11:21pm UTC](https://community.graylog.org/t/timestamp-replacement-issues-ingestion-timestamp-vs-log-timestamp/9842 "2019-04-05T23:21:46Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![nom666nom](https://avatars.discourse-cdn.com/v4/letter/n/f17d59/32.png) [@nom666nom](https://community.graylog.org/u/nom666nom)\
**Post date:** [April 5, 2019, 11:21pm UTC](https://community.graylog.org/t/timestamp-replacement-issues-ingestion-timestamp-vs-log-timestamp/9842/1 "2019-04-05T23:21:46Z")

</div>

Essentially I’m trying to replace the ingestion timestamp with the actual timestamp in the logs. I’ve read many of the posts about this but I don’t seem to be making any progress.

As you can see in the screenshot below the Timestamp(far left) and timestamp(in the middle) are the same and are the time of ingestion, not the time of the log. I’ve created an extractor to copy the timestamp from the log and put it into the timestamp\_orig(far right). As you can see in the screenshot.

 ![47%20PM](https://us1.discourse-cdn.com/flex016/uploads/graylog/original/2X/3/3a421f64eab301a28e828426318816823d6ca051.png)

I’ve already tried to copy/replace in the extractor to the timestamp field but that doesn’t change the timestamps.

I’ve then tried to make a pipeline to do it.

Pipeline Code:

> rule “parse right timestamp”  
> when  
> true  
> then  
> let new\_date = parse\_date(to\_string($message.timestamp\_orig));  
> set\_field(“timestamp”, new\_date);  
> end

That doesn’t seem to be doing anything to my timestamps.

I feel like I’m missing something fairly simple and help would be much appreciated.

---

<div class="post-metadata">

**Author:** ![nom666nom](https://avatars.discourse-cdn.com/v4/letter/n/f17d59/32.png) [@nom666nom](https://community.graylog.org/u/nom666nom)\
**Post date:** [April 6, 2019, 1:50am UTC](https://community.graylog.org/t/timestamp-replacement-issues-ingestion-timestamp-vs-log-timestamp/9842/2 "2019-04-06T01:50:54Z")

</div>

I did some troubleshooting and I can confirm that the pipeline is processing my messages, just isn’t doing what it’s supposed to do.

I change my pipeline rule to set a static value for a field.

> rule “parse right timestamp”  
> when  
> has\_field(“timestamp\_orig”)  
> then  
> let new\_date = parse\_date(to\_string($message.timestamp\_orig));  
> set\_field(“beat\_hostname”, “taco”);  
> end

 ![11%20PM](https://us1.discourse-cdn.com/flex016/uploads/graylog/original/2X/4/4473c0896170128acf44bf212a375f15d3d52f05.png)

When I just change the set\_field to the variable it reverts back the hostname and not the time stamp.

> rule “parse right timestamp”  
> when  
> has\_field(“timestamp\_orig”)  
> then  
> let new\_date = parse\_date(to\_string($message.timestamp\_orig));  
> set\_field(“beat\_hostname”, new\_date);  
> end

This leads me to believe that the parse\_date is that part not working.

---

<div class="post-metadata">

**Author:** ![nom666nom](https://avatars.discourse-cdn.com/v4/letter/n/f17d59/32.png) [@nom666nom](https://community.graylog.org/u/nom666nom)\
**Post date:** [April 6, 2019, 2:24am UTC](https://community.graylog.org/t/timestamp-replacement-issues-ingestion-timestamp-vs-log-timestamp/9842/3 "2019-04-06T02:24:54Z")

</div>

Success. The gl2\_processing\_error field was useful in figuring out what was going on since I wasn’t seeing anything in the logs.

> For rule ‘parse right timestamp’: In call to function ‘parse\_date’ at 5:19 an exception was thrown: Invalid format: “2019-04-05T19:08:08.513458-07:00” is malformed at “-07:00”

Basically this pointed to a format issue and everything in the timestamp needs to be accounted for in the parse cmd.

The following worked for the timestamps that rsyslog was forming.

> rule “parse right timestamp”  
> when  
> has\_field(“timestamp\_orig”)  
> then  
> let new\_date = parse\_date(to\_string($message.timestamp\_orig), “yyyy-MM-dd’T’HH:mm:ss.SSSSSSZ”);  
> set\_field(“timestamp”, new\_date);  
> end

 ![58%20PM](https://us1.discourse-cdn.com/flex016/uploads/graylog/original/2X/5/583457a9b9985e01b713686ae76639701e7cea96.png)

Hopefully this helps someone else.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/flex016/uploads/graylog/original/3X/c/7/c7c09c6b5099570133d6502b83f50ba4430de5b6.png) [@system](https://community.graylog.org/u/system)\
**Post date:** [April 20, 2019, 2:24am UTC](https://community.graylog.org/t/timestamp-replacement-issues-ingestion-timestamp-vs-log-timestamp/9842/4 "2019-04-20T02:24:59Z")

</div>

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.