Reading logs from s3 directly


(Tibin George) #1

I have tried configuring https://github.com/sherzberg/graylog-plugin-s3, But it dosen’t seems to be working properly. I get error

Could not read messages from SNS. This is most likely a misconfiguration of the plugin. Going into sleep loop and retrying.
com.amazonaws.SdkClientException: The requested metadata is not found at http://169.254.169.254/latest/meta-data/iam/security-credentials/

Is there any alternative way so that I can read files in s3 bucket into graylog.
My usecase is I have some logs in s3 bucket pushed from CDN and ELB I want all that logs to be loaded into graylog.


(Jochen) #2

Is your Graylog cluster running in AWS?

In general, I’d recommend creating an issue in the GitHub repository of this 3rd party plugin:


(Tibin George) #3

Is there an issue running graylog server running in aws instance?


(Jochen) #4

No, but the plugin seems to expect Graylog to run in AWS (on EC2) in order to retrieve some security credentials. See the details of the error message you’ve posted.


(Tibin George) #5

Graylog server is running in aws, But I am not sure why this plugin is trying to access SNS as this plugin should take tasks from SQS and not from SNS.


(Jochen) #6

For reference:


(Tibin George) #7

Thank You @jochen for help, got it working by adding iam role to my graylog server.


(system) #8

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.