Graylog missing field in json embedded object

Hello,
I’m shipping logfiles in json format using logstash and its json filter.
Example (status is the important object)
{“status”: {“errorDetails”: “”, “errorMessage”: “”, “statusCode”: “SUCCESS”, “error”: false}, “logTag”: “SNOWFLAKE_QUERY”, “context”: {“datasetId”: “CSCOKE436PI_UAT”}, “executionMetrics”: {“execMillis”: 4888, “execType”: “Extract”, “execTarget”: “CS_OD_RPT”, “execTechnology”: “Snowflake”}, “logLevel”: “INFO”, “hostName”: “mue2rhepaggq001”, “dateTime”: “02/14/2020 07:26:32.695 UTC”, “logType”: “EVENT”, “version”: “362156”, “sourceApplication”: {“applicationName”: “Connect Extracts”, “buildVersion”: “CS_6.9.7.19”, “applicationComponent”: “FileGen”}, “messageId”: “4”, “duration”: {“totalTime”: 4888, “timeUnit”: “MILLISECONDS”}, “transactionId”: “4a691073-a63f-44cd-8067-eae2ea0a3b4d”, “dataVolumetrics”: “NA”, “event”: {“action”: “FileGen”, “type”: “Extract”, “target”: “Connect Extracts”}}

What I see in the logstash debug output:
“status” => {
“errorMessage” => “”,
“errorDetails” => “”,
“statusCode” => “SUCCESS”,
“error” => false
}

In Graylog I get status_statusCode only
What I expect is also status_error

Why don’t I get status_error ?

he @kai_poitschke

when you have logstash in between - how did you send that messages from logstash to graylog? What input did you use, how is logstash configured?

Hello,
the logstash version is 6.4.2
We ship via gelf

config looks like this ( removed stuff for clarity)

input {
  file {
    path => "/.../filegen_eventlog*.json"
    start_position => "beginning"
    type => "eventlog-extract-filegen"

    add_field => { "source_env" => "usprod" }

    close_older => 600
    max_open_files => 100
  }
}

filter {
    json {
      source => "message"
    }
    date {
      locale => "en"
      match => [ "dateTime" ,
                "ISO8601"
                ]
    }
  }
}
output {
  gelf {
    host => "....."
    port => "12201"
    protocol => "TCP"
  }
 # stdout { codec => "rubydebug" }
}

he @kai_poitschke

as you do the json processing in logstash and only send GELF as output you need to configure logstash that the nested json arrays are used. I do not know about the json parsing in logstash - but maybe someone else is able to give some light here.

Hello Jan,
logstash is sending it right, see my first posting. The status_statusCode field is present in Graylog (and elasticsearch), but status_error is not visible.

follow the trail - step by step. is that field is transfered from logstash to graylog? from graylog stored in elasticsearch?

Use the processing pipelines and the debug function to find the problem.

Hello @Jan,
please provide or point me to an example how to debug this in the pipeline. I’m not familiar in debugging GL.

Check documentation, I think it is very clear:
https://docs.graylog.org/en/3.2/pages/pipelines/functions.html#debug

  1. So use this snippet in your pipeline rule, or modify:

let debug_message = concat("Dropped message from ", to_string($message.source));
debug(debug_message);

  1. And after that check graylog log file to see debug output;
    sudo tail -f /var/log/graylog-server/server.log