Hi, Thanks for trying to help me!
From what I understand in this post you upgrading Graylog and I think you upgrade ES and MongoDb, Is this correct?
I restate/rephrase my goal, and the context of my question:
- I intended to upgrade graylog, but found the elasticsearch was on such an old version that it was impossible to upgrade.
- so instead of upgrading, I set up a new fresh instance, installing first mongodb and elasticsearch, and then graylog.
- I then tried to configure the new instance in ‘the same way’ as the old one. This is what I’ve been unable to achieve.
We would need the full log file
In fact, it contains only 5550 identical blocks, so I leave only the first and the last ones:
2022-06-08T03:55:46.610Z ERROR [CloudTrailSubscriber] Could not read messages from SQS. This is most likely a misconfiguration of the plugin. Going into sleep loop and retrying.
com.amazonaws.SdkClientException: Unable to load AWS credentials from any provider in the chain: [EnvironmentVariableCredentialsProvider: Unable to load AWS credentials from environment variables (AWS_ACCESS_KEY_ID (or AWS_ACCESS_KEY) and AWS_SECRET_KEY (or AWS_SECRET_ACCESS_KEY)), SystemPropertiesCredentialsProvider: Unable to load AWS credentials from Java system properties (aws.accessKeyId and aws.secretKey), WebIdentityTokenCredentialsProvider: You must specify a value for roleArn and roleSessionName, com.amazonaws.auth.profile.ProfileCredentialsProvider@3228fa59: profile file cannot be null, com.amazonaws.auth.EC2ContainerCredentialsProviderWrapper@3a9bf0b2: The requested metadata is not found at http://169.254.169.254/latest/meta-data/iam/security-credentials/]
at com.amazonaws.auth.AWSCredentialsProviderChain.getCredentials(AWSCredentialsProviderChain.java:136) ~[graylog-plugin-aws-4.3.1.jar:?]
at org.graylog.aws.auth.AWSAuthProvider.getCredentials(AWSAuthProvider.java:98) ~[graylog-plugin-aws-4.3.1.jar:?]
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.getCredentialsFromContext(AmazonHttpClient.java:1257) ~[graylog-plugin-aws-4.3.1.jar:?]
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.runBeforeRequestHandlers(AmazonHttpClient.java:833) ~[graylog-plugin-aws-4.3.1.jar:?]
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.doExecute(AmazonHttpClient.java:783) ~[graylog-plugin-aws-4.3.1.jar:?]
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeWithTimer(AmazonHttpClient.java:770) ~[graylog-plugin-aws-4.3.1.jar:?]
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.execute(AmazonHttpClient.java:744) ~[graylog-plugin-aws-4.3.1.jar:?]
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.access$500(AmazonHttpClient.java:704) ~[graylog-plugin-aws-4.3.1.jar:?]
at com.amazonaws.http.AmazonHttpClient$RequestExecutionBuilderImpl.execute(AmazonHttpClient.java:686) ~[graylog-plugin-aws-4.3.1.jar:?]
at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:550) ~[graylog-plugin-aws-4.3.1.jar:?]
at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:530) ~[graylog-plugin-aws-4.3.1.jar:?]
at com.amazonaws.services.sqs.AmazonSQSClient.doInvoke(AmazonSQSClient.java:2243) ~[graylog-plugin-aws-4.3.1.jar:?]
at com.amazonaws.services.sqs.AmazonSQSClient.invoke(AmazonSQSClient.java:2210) ~[graylog-plugin-aws-4.3.1.jar:?]
at com.amazonaws.services.sqs.AmazonSQSClient.invoke(AmazonSQSClient.java:2199) ~[graylog-plugin-aws-4.3.1.jar:?]
at com.amazonaws.services.sqs.AmazonSQSClient.executeReceiveMessage(AmazonSQSClient.java:1637) ~[graylog-plugin-aws-4.3.1.jar:?]
at com.amazonaws.services.sqs.AmazonSQSClient.receiveMessage(AmazonSQSClient.java:1607) ~[graylog-plugin-aws-4.3.1.jar:?]
at org.graylog.aws.inputs.cloudtrail.notifications.CloudtrailSQSClient.getNotifications(CloudtrailSQSClient.java:64) ~[graylog-plugin-aws-4.3.1.jar:?]
at org.graylog.aws.inputs.cloudtrail.CloudTrailSubscriber.run(CloudTrailSubscriber.java:105) [graylog-plugin-aws-4.3.1.jar:?]
...
2022-06-08T11:38:26.466Z ERROR [CloudTrailSubscriber] Could not read messages from SQS. This is most likely a misconfiguration of the plugin. Going into sleep loop and retrying.
com.amazonaws.SdkClientException: Unable to load AWS credentials from any provider in the chain: [EnvironmentVariableCredentialsProvider: Unable to load AWS credentials from environment variables (AWS_ACCESS_KEY_ID (or AWS_ACCESS_KEY) and AWS_SECRET_KEY (or AWS_SECRET_ACCESS_KEY)), SystemPropertiesCredentialsProvider: Unable to load AWS credentials from Java system properties (aws.accessKeyId and aws.secretKey), WebIdentityTokenCredentialsProvider: You must specify a value for roleArn and roleSessionName, com.amazonaws.auth.profile.ProfileCredentialsProvider@3228fa59: profile file cannot be null, com.amazonaws.auth.EC2ContainerCredentialsProviderWrapper@3a9bf0b2: The requested metadata is not found at http://169.254.169.254/latest/meta-data/iam/security-credentials/]
at com.amazonaws.auth.AWSCredentialsProviderChain.getCredentials(AWSCredentialsProviderChain.java:136) ~[graylog-plugin-aws-4.3.1.jar:?]
at org.graylog.aws.auth.AWSAuthProvider.getCredentials(AWSAuthProvider.java:98) ~[graylog-plugin-aws-4.3.1.jar:?]
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.getCredentialsFromContext(AmazonHttpClient.java:1257) ~[graylog-plugin-aws-4.3.1.jar:?]
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.runBeforeRequestHandlers(AmazonHttpClient.java:833) ~[graylog-plugin-aws-4.3.1.jar:?]
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.doExecute(AmazonHttpClient.java:783) ~[graylog-plugin-aws-4.3.1.jar:?]
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeWithTimer(AmazonHttpClient.java:770) ~[graylog-plugin-aws-4.3.1.jar:?]
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.execute(AmazonHttpClient.java:744) ~[graylog-plugin-aws-4.3.1.jar:?]
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.access$500(AmazonHttpClient.java:704) ~[graylog-plugin-aws-4.3.1.jar:?]
at com.amazonaws.http.AmazonHttpClient$RequestExecutionBuilderImpl.execute(AmazonHttpClient.java:686) ~[graylog-plugin-aws-4.3.1.jar:?]
at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:550) ~[graylog-plugin-aws-4.3.1.jar:?]
at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:530) ~[graylog-plugin-aws-4.3.1.jar:?]
at com.amazonaws.services.sqs.AmazonSQSClient.doInvoke(AmazonSQSClient.java:2243) ~[graylog-plugin-aws-4.3.1.jar:?]
at com.amazonaws.services.sqs.AmazonSQSClient.invoke(AmazonSQSClient.java:2210) ~[graylog-plugin-aws-4.3.1.jar:?]
at com.amazonaws.services.sqs.AmazonSQSClient.invoke(AmazonSQSClient.java:2199) ~[graylog-plugin-aws-4.3.1.jar:?]
at com.amazonaws.services.sqs.AmazonSQSClient.executeReceiveMessage(AmazonSQSClient.java:1637) ~[graylog-plugin-aws-4.3.1.jar:?]
at com.amazonaws.services.sqs.AmazonSQSClient.receiveMessage(AmazonSQSClient.java:1607) ~[graylog-plugin-aws-4.3.1.jar:?]
at org.graylog.aws.inputs.cloudtrail.notifications.CloudtrailSQSClient.getNotifications(CloudtrailSQSClient.java:64) ~[graylog-plugin-aws-4.3.1.jar:?]
at org.graylog.aws.inputs.cloudtrail.CloudTrailSubscriber.run(CloudTrailSubscriber.java:105) [graylog-plugin-aws-4.3.1.jar:?]
GL configuration
I removed all comments and empty lines, as well as the value of the two passwords:
is_master = true
node_id_file = /etc/graylog/server/node-id
password_secret = xxx
root_password_sha2 = xxx
bin_dir = /usr/share/graylog-server/bin
data_dir = /var/lib/graylog-server
plugin_dir = /usr/share/graylog-server/plugin
http_bind_address = 127.0.0.1:9000
enabled_index_rotation_strategies = count,size,time
rotation_strategy = count
elasticsearch_max_docs_per_index = 20000000
elasticsearch_max_number_of_indices = 20
retention_strategy = delete
elasticsearch_shards = 4
elasticsearch_replicas = 0
elasticsearch_index_prefix = graylog
allow_leading_wildcard_searches = false
allow_highlighting = false
elasticsearch_analyzer = standard
output_batch_size = 500
output_flush_interval = 1
output_fault_count_threshold = 5
output_fault_penalty_seconds = 30
processbuffer_processors = 5
outputbuffer_processors = 3
processor_wait_strategy = blocking
ring_size = 65536
inputbuffer_ring_size = 65536
inputbuffer_processors = 2
inputbuffer_wait_strategy = blocking
message_journal_enabled = true
message_journal_dir = /var/lib/graylog-server/journal
lb_recognition_period_seconds = 3
mongodb_uri = mongodb://localhost/graylog
mongodb_max_connections = 1000
mongodb_threads_allowed_to_block_multiplier = 5
proxied_requests_thread_pool_size = 32