- Unprocessed messages is increasing, both of input buffer and output buffer are zero.
- Run
apt update && apt upgrade -y
, it include upgrade Graylog from 5.0.3 to 5.0.4 - After increased CPU(2 cores to 4 cores) and RAM(2GB to 4GB), I change JVM max/min heap size from 1G to 2G and processbuffer_processors from 2 to 4. But problem still exists.
- There are OTX time logs in Graylog server.log, so I comment out below pipeline rule:
let intel_otx = otx_lookup_ip(to_string($message.srcip)); set_field("srcip_OTX", intel_otx.otx_threat_indicated); set_field("srcip_OTX_ids", intel_otx.otx_threat_ids); set_field("srcip_OTX_names", intel_otx.otx_threat_names);
- Problem gone, both add normalRequest field if regex match and unprocessed messages.
- In order to prevent OTX block my query, I changed cache configuration to low down the OTX query.
I am not sure what the root cause it is.