I need advice on how to better configure Graylog for my use case. I’m sending logs from the Rails app in
json format (using
lograge gem), and then in Graylog I use JSON extractor to parse these into fields. This works quite well except two corner cases:
- There are many fields in these JSON messages that I don’t need. They quickly hit ElasticSearch limit of 1000 fields. How these could be dropped, preferably using some wildcard or regex?
- There are fields those lengths exceed 32kb (Base64 encoded files), thus they are not indexed. How these could be truncated to some sane length?