Reading logs from s3 directly

Graylog Add-ons
1

I have tried configuring https://github.com/sherzberg/graylog-plugin-s3, But it dosen’t seems to be working properly. I get error

Could not read messages from SNS. This is most likely a misconfiguration of the plugin. Going into sleep loop and retrying.
com.amazonaws.SdkClientException: The requested metadata is not found at http://169.254.169.254/latest/meta-data/iam/security-credentials/

Is there any alternative way so that I can read files in s3 bucket into graylog.
My usecase is I have some logs in s3 bucket pushed from CDN and ELB I want all that logs to be loaded into graylog.

2

Is your Graylog cluster running in AWS?

In general, I’d recommend creating an issue in the GitHub repository of this 3rd party plugin:

3

Is there an issue running graylog server running in aws instance?

4

No, but the plugin seems to expect Graylog to run in AWS (on EC2) in order to retrieve some security credentials. See the details of the error message you’ve posted.

5

Graylog server is running in aws, But I am not sure why this plugin is trying to access SNS as this plugin should take tasks from SQS and not from SNS.

6

For reference:
https://github.com/sherzberg/graylog-plugin-s3/issues/11

7

Thank You @jochen for help, got it working by adding iam role to my graylog server.

8

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.