AWS raw logs plugin: What fields should come out of the box with entity translation?

(Bronius Motekaitis) #1

Hi - Please help me understand what fields will always be expected for the AWS Logs plugin and whether other fields might be available in some cases out of the box.

We migrated from single iron rack host to multi-host on AWS cloud, and we switched from syslog message forwarding to AWS log forward over Kinesis using Graylog 2.4 and the “AWS Logs” plugin that ships with it. I see only the following fields in every message regardless of which log group produces the message:

  • aws_log_group
  • aws_log_stream
  • message
  • source
  • timestamp

This is with Entity Translation checked but without AWS keys configured in Graylog config. Is it normal that just these 4 fields and raw message are available out of the box? It is understandable if it is, in which case I have some pipeline processing to write…

What gives me reason to believe there might be more is the screenshot in plugin docs suggests a whole bunch more fields, but maybe this is just what comes with Flow Logs (which I am not using). Hoping for more fields, I continued with the suggested IAM user and Policy and configured Graylog with the new AWS keys. After this, my input stopped reporting messages (0 throughput). I removed AWS keys, and the Input picked right back up again.


(system) #2

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.