Rule to compare 2 separate messages, calc timestamps diff and set new field

[joice0221 (Joice)]

Hi, I have a problem about creating a good rule that would allow me to search for 2 messages where one of them contains “Executing job” and the other contains “Finished job execution” and both have the same requestID. Then for these two messages I would like to calculate the timestamp difference and store it as a new field.

Below is something I’ve already created, but it doesn’t bring any positive results:

rule "Time diff"
when
    has_field("requestId") &&
    has_field("full_message") &&
    starts_with(to_string($message.full_message), "Executing job")
then
    let request_id = to_string($message.requestId);
    let timestamp1 = to_long($message.timestamp);
    
    let finished_message = has_field("requestId:" + request_id + " AND full_message:Finished*");
    
    let timestamp2 = to_long($message.last_message.timestamp);
    let time_diff = timestamp2 - timestamp1;
    set_field("time_difference_ms", time_diff);
end

I will be grateful for tips and good advice on how to write such a rule

[gsmith]

Hey @joice0221

I havent done that yet, But in my personal docs, I have log ingestion time this is a rough measurement of log latency.

rule "processingtime"
when 
    has_field("input_time")
then
    let ingestion_time = to_date($message.input_time).millis;
    let current_time = to_date(now()).millis;
    let millis_diff = to_long(to_long(current_time) - to_long(ingestion_time));

    set_field("pipeline_processingtime", millis_diff);
end