Fields shown in JSON extractor preview missing in processed messages


(FadenB) #1

Hey,

we are having troubles with some field names.

We receive messages like the following via raw TCP input:

{"date":"2017-09-16","cs-method":"GET","proccessing_datetime":"2017-09-16T08:13:55.306Z","cs(Host)":"XXXXXXXXXXXXX.cloudfront.net","cs-uri-stem":"/monitoring.canary","cs-uri-query":"-","source":"logstash_cloudfront_processor","cs-protocol":"http","date_time":"2017-09-16 08:13:27","sc-status":"200","cs(User-Agent)":"Mozilla/5.0%2520(compatible;%2520monitis%2520-%2520premium%2520monitoring%2520service;%2520http://www.monitis.com)","cs(Referer)":"-","cs(Cookie)":"-","x-host-header":"linux.some.tld","cs-bytes":"214","x-edge-result-type":"Hit","x-forwarded-for":"-","ssl-protocol":"-","time-taken":"0.003","cs-protocol-version":"HTTP/1.1","x-edge-location":"LAX1","ssl-cipher":"-","x-edge-response-result-type":"Hit","sc-bytes":"601","x-edge-request-id":"mB3ZhsscyD52Ntn2nArolYbMuRNZrvRdAzq_S4YHptvVDYvJF7CwYQ==","time":"08:13:27","c-ip":"AAA.BBB.CCC.DD"}

We then apply the JSON extractor (default config) to it.
The “Try” button in the extractor config shows us exactly what we were expecting:


All the fields - including those starting with cs( - are shown.

The issue is that those fields are not visible once the message has been processed (image hosted on imgur as Graylog community does not allow more than 1 image for new users :confused: )
https://imgur.com/Q8QnIpw

Is this some Graylog restriction, wrong config or perhaps a bug?

Thanks!


(Jochen) #2

This currently is a restriction of Graylog.

Message keys have to adhere to the following regular expression:

Personally, I don’t see a reason for this nowadays (it was modeled after the GELF specification, but it shouldn’t be restricted to this), so feel free to file a feature request at https://github.com/Graylog2/graylog2-server/issues.


(system) #3

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.