JSON Extractor and Field Converter

Hi guys:

Using JSON log format in Apache and sending access logs to Graylog via FileBeats. In httpd.conf file:

LogFormat “{ “time”:”%{%Y-%m-%d}tT%{%T}t.%{msec_frac}tZ", “process”:"%D", “filename”:"%f", “src_ip”:"%a", “webserver”:"%V", “webserver_port”:"%p", “request”:"%U", “query”:"%q", “method”:"%m", “status”:"%>s", “userAgent”:"%{User-agent}i", “referer”:"%{Referer}i", “bytes_sent”:"%B", “response_time”:"%T"}, " combined

In Graylog’s Beats input a “simple” JSON extarctor does the trick and all good.

The little problem I’m seeing is that fields such as BYTES, or RESPONSE TIME are strored as string and therefore stats or graphs don’t apply.

Is there any way to “reformat/convert” fields with this setup? Could a pipeline be defined after the message gets processed by the extractor?

Thanks a lot


Yes, a pipeline can sort this out:

rule “WAF Access Logs Convert Fields”
// the following date format assumes there’s no time zone in the string
let new_bytes = to_long($message.bytes_sent);
set_field(“bytes”, new_bytes);

The new field “bytes” now stores the value as integer.


This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.