How to drop/truncate fields in JSON extractor?

Graylog Central (peer support)
1

Hi there,

I need advice on how to better configure Graylog for my use case. I’m sending logs from the Rails app in json format (using lograge gem), and then in Graylog I use JSON extractor to parse these into fields. This works quite well except two corner cases:

  1. There are many fields in these JSON messages that I don’t need. They quickly hit ElasticSearch limit of 1000 fields. How these could be dropped, preferably using some wildcard or regex?
  2. There are fields those lengths exceed 32kb (Base64 encoded files), thus they are not indexed. How these could be truncated to some sane length?
2

he @leshik

the json extractor does not give you the option. One option would be to reduce the noise from your application … but you could also delete fields in Graylog with the processing pipeline and the function remove_field( https://docs.graylog.org/en/3.2/pages/pipelines/functions.html#remove-field )

3

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.