I would like to get the time difference of two timestamps having "started" and "finished", they share the same correlationid

[ztony (tony zhang)]

Before you post: Your responses to these questions will help the community help you. Please complete this template if you’re asking a support question.
Don’t forget to select tags to help index your topic!

1. Describe your incident:
I would like to get the time difference of two timestamps having “started” and “finished”, they share the same correlationid

2. Describe your environment:

• OS Information:Linux

• Package Version: 3.3.15

• Service logs, configurations, and environment variables:

3. What steps have you already taken to try and solve the problem?

• epoch timestamp

I tried to use pipeline rules to get epoch format, below is the code snippet:

rule “Date”
when
has_field(“message”)
then

let new_date = parse_unix_milliseconds(to_string($message.timestamp)*1000,“YYYY-MM-DDTHH:mm:ss.sssZ”);
set_field(“epoch_timestamp”, new_date);
end

• calculate the time difference from query

I tried to use pipe and “eval” in the query, below is the code snippet:

[correlationid=“my-correlationid”]
| eval started = parse_datetime(“started”, “YYYY-MM-DDTHH:mm:ss.sssZ”)
| eval finished = parse_datetime(“finished”, “YYYY-MM-DDTHH:mm:ss.sssZ”)
| eval time_diff = finished - started

neither worked so far. any suggestions?
Thank you!
Tony

4. How can the community help?

Helpful Posting Tips: Tips for Posting Questions that Get Answers [Hold down CTRL and link on link to open tips documents in a separate tab]

[gsmith]

Hey @ztony
Sorry for the delay,
you looking for measurement of log latency? If so something like this example?

rule "add pipeline_processingtime"
when 
    has_field("input_time")
then
    let ingestion_time = to_date($message.input_time).millis;
    let current_time = to_date(now()).millis;
    let millis_diff = to_long(to_long(current_time) - to_long(ingestion_time));

    set_field("pipeline_processingtime", millis_diff);
end