Configuring graylog 4.2.9 from scratch, replicating a running 2.4.4 configuration

mgirod · May 31, 2022, 8:22am

1. Describe your incident:

Our running configuration is too old to upgrade – elasticsearch 2.4.6, mongodb 3.2.22
I decided to set up a brand new server (CentOS 8 on AWS), to collect data for 30 days, and to swap (forgetting the old data).
However, I’m not able to reproduce the setup.

2. Describe your environment:

OS Information: CentOS 8.5; SELinux: enabled / targeted / enforcing
Package Version: graylog 4.2.9
Service logs, configurations, and environment variables:
one local input configured GELF UDP, bind_address: 0.0.0.0

3. What steps have you already taken to try and solve the problem?
Installed, configured, started mongodb 4.2.20, elasticsearch 7.10, apache proxy

4. How can the community help?
What should I look for in the old running setup, and how to reproduce it in the new one?
Thanks

gsmith · May 31, 2022, 10:08pm

Hello && Welcome

Since GL 2.2 a lot of settings/configurations have changed.

You may want to look here

mgirod · June 2, 2022, 9:58am

Thanks for your reply, but I’m still stuck.
Let me rephrase my question. In my running 2-2 setup, in the Search view, I get (with 'search in the last 5 minutes, and no query at all yet):

## Search result
Found **17,959 messages** in 8 ms, searched in [1 index](http://graylog.alison.com/search#).
Results retrieved at 2022-06-02 09:50:18.

Before looking at these messages, I would like to understand where they come from, and how?
How should I find the answer to these questions?

I looked for an answer with:

# grep -s '[connection]' /var/log/graylog-server/server.log | egrep -c ':[0-9][0-9]*$'
134
# grep -s '[connection]' /var/log/graylog-server/server.log | perl -nle '$h{$1}++ if/(\S+:\d+)$/;END{print for sort{$a cmp$b}keys%h}'
localhost:27017
# head -1 /var/log/graylog-server/server.log | cut -d' ' -f1
2020-07-06T01:47:13.251Z

So… since July 2020, 134 connection openings were recorded in server.log, all to localhost:27017
But this is used between graylog and mongodb

mgirod · June 2, 2022, 2:37pm

BTW, on my new server (which I upgraded now to 4.3), I already have 137 such connections recorded since May 12!, and no messages.

I also added two ‘inputs’ on my new server (AWS and syslog/5140). I’ll check later if anything shows up.
But the problem is that on the old server, there’s only ‘Default GELF UDP input’, so that’s not the way it works there!?

gsmith · June 2, 2022, 9:49pm

Hello,

The logs shown doesn’t give a clear idea what the issue is.
My guess would be a configuration error and/or perhaps something to do with connections.

From what I understand in this post you upgrading Graylog and I think you upgrade ES and MongoDb, Is this correct?

Now your unable to see logs in the Global search for 5 minutes.

From what little information shown its hard to troubleshoot what this issue is.
We would need the full log file , GL configuration, knowing the status of ES and MongoDb service.
Remove personal info if posting & use the </> please)

By chance did you check Elasticsearch & MongoDb logs? If so what you see?

As of now I can only suggest checking the settings for Graylog to insure there the correct ones for Graylog-4.3… I’m not sure if you seen the new documentation on changes and the reason I stated this is because the post above mentioned is from this statement below.

Normally when a delay on messages is shown this could be a result of Date/Time configuration. If no messages are shown but you see them using tcpdump on Graylog, I then would check security service like Iptables/Selinux and permissions on files and folders also insure Elasticsearch is running without issues. Another issue could be when using TCP/TLS connections.

mgirod · June 8, 2022, 9:20am

Hi, Thanks for trying to help me!

From what I understand in this post you upgrading Graylog and I think you upgrade ES and MongoDb, Is this correct?

I restate/rephrase my goal, and the context of my question:

I intended to upgrade graylog, but found the elasticsearch was on such an old version that it was impossible to upgrade.
so instead of upgrading, I set up a new fresh instance, installing first mongodb and elasticsearch, and then graylog.
I then tried to configure the new instance in ‘the same way’ as the old one. This is what I’ve been unable to achieve.

We would need the full log file

In fact, it contains only 5550 identical blocks, so I leave only the first and the last ones:

2022-06-08T03:55:46.610Z ERROR [CloudTrailSubscriber] Could not read messages from SQS. This is most likely a misconfiguration of the plugin. Going into sleep loop and retrying.
com.amazonaws.SdkClientException: Unable to load AWS credentials from any provider in the chain: [EnvironmentVariableCredentialsProvider: Unable to load AWS credentials from environment variables (AWS_ACCESS_KEY_ID (or AWS_ACCESS_KEY) and AWS_SECRET_KEY (or AWS_SECRET_ACCESS_KEY)), SystemPropertiesCredentialsProvider: Unable to load AWS credentials from Java system properties (aws.accessKeyId and aws.secretKey), WebIdentityTokenCredentialsProvider: You must specify a value for roleArn and roleSessionName, com.amazonaws.auth.profile.ProfileCredentialsProvider@3228fa59: profile file cannot be null, com.amazonaws.auth.EC2ContainerCredentialsProviderWrapper@3a9bf0b2: The requested metadata is not found at http://169.254.169.254/latest/meta-data/iam/security-credentials/]
	at com.amazonaws.auth.AWSCredentialsProviderChain.getCredentials(AWSCredentialsProviderChain.java:136) ~[graylog-plugin-aws-4.3.1.jar:?]
	at org.graylog.aws.auth.AWSAuthProvider.getCredentials(AWSAuthProvider.java:98) ~[graylog-plugin-aws-4.3.1.jar:?]
	at com.amazonaws.http.AmazonHttpClient$RequestExecutor.getCredentialsFromContext(AmazonHttpClient.java:1257) ~[graylog-plugin-aws-4.3.1.jar:?]
	at com.amazonaws.http.AmazonHttpClient$RequestExecutor.runBeforeRequestHandlers(AmazonHttpClient.java:833) ~[graylog-plugin-aws-4.3.1.jar:?]
	at com.amazonaws.http.AmazonHttpClient$RequestExecutor.doExecute(AmazonHttpClient.java:783) ~[graylog-plugin-aws-4.3.1.jar:?]
	at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeWithTimer(AmazonHttpClient.java:770) ~[graylog-plugin-aws-4.3.1.jar:?]
	at com.amazonaws.http.AmazonHttpClient$RequestExecutor.execute(AmazonHttpClient.java:744) ~[graylog-plugin-aws-4.3.1.jar:?]
	at com.amazonaws.http.AmazonHttpClient$RequestExecutor.access$500(AmazonHttpClient.java:704) ~[graylog-plugin-aws-4.3.1.jar:?]
	at com.amazonaws.http.AmazonHttpClient$RequestExecutionBuilderImpl.execute(AmazonHttpClient.java:686) ~[graylog-plugin-aws-4.3.1.jar:?]
	at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:550) ~[graylog-plugin-aws-4.3.1.jar:?]
	at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:530) ~[graylog-plugin-aws-4.3.1.jar:?]
	at com.amazonaws.services.sqs.AmazonSQSClient.doInvoke(AmazonSQSClient.java:2243) ~[graylog-plugin-aws-4.3.1.jar:?]
	at com.amazonaws.services.sqs.AmazonSQSClient.invoke(AmazonSQSClient.java:2210) ~[graylog-plugin-aws-4.3.1.jar:?]
	at com.amazonaws.services.sqs.AmazonSQSClient.invoke(AmazonSQSClient.java:2199) ~[graylog-plugin-aws-4.3.1.jar:?]
	at com.amazonaws.services.sqs.AmazonSQSClient.executeReceiveMessage(AmazonSQSClient.java:1637) ~[graylog-plugin-aws-4.3.1.jar:?]
	at com.amazonaws.services.sqs.AmazonSQSClient.receiveMessage(AmazonSQSClient.java:1607) ~[graylog-plugin-aws-4.3.1.jar:?]
	at org.graylog.aws.inputs.cloudtrail.notifications.CloudtrailSQSClient.getNotifications(CloudtrailSQSClient.java:64) ~[graylog-plugin-aws-4.3.1.jar:?]
	at org.graylog.aws.inputs.cloudtrail.CloudTrailSubscriber.run(CloudTrailSubscriber.java:105) [graylog-plugin-aws-4.3.1.jar:?]
...
2022-06-08T11:38:26.466Z ERROR [CloudTrailSubscriber] Could not read messages from SQS. This is most likely a misconfiguration of the plugin. Going into sleep loop and retrying.
com.amazonaws.SdkClientException: Unable to load AWS credentials from any provider in the chain: [EnvironmentVariableCredentialsProvider: Unable to load AWS credentials from environment variables (AWS_ACCESS_KEY_ID (or AWS_ACCESS_KEY) and AWS_SECRET_KEY (or AWS_SECRET_ACCESS_KEY)), SystemPropertiesCredentialsProvider: Unable to load AWS credentials from Java system properties (aws.accessKeyId and aws.secretKey), WebIdentityTokenCredentialsProvider: You must specify a value for roleArn and roleSessionName, com.amazonaws.auth.profile.ProfileCredentialsProvider@3228fa59: profile file cannot be null, com.amazonaws.auth.EC2ContainerCredentialsProviderWrapper@3a9bf0b2: The requested metadata is not found at http://169.254.169.254/latest/meta-data/iam/security-credentials/]
	at com.amazonaws.auth.AWSCredentialsProviderChain.getCredentials(AWSCredentialsProviderChain.java:136) ~[graylog-plugin-aws-4.3.1.jar:?]
	at org.graylog.aws.auth.AWSAuthProvider.getCredentials(AWSAuthProvider.java:98) ~[graylog-plugin-aws-4.3.1.jar:?]
	at com.amazonaws.http.AmazonHttpClient$RequestExecutor.getCredentialsFromContext(AmazonHttpClient.java:1257) ~[graylog-plugin-aws-4.3.1.jar:?]
	at com.amazonaws.http.AmazonHttpClient$RequestExecutor.runBeforeRequestHandlers(AmazonHttpClient.java:833) ~[graylog-plugin-aws-4.3.1.jar:?]
	at com.amazonaws.http.AmazonHttpClient$RequestExecutor.doExecute(AmazonHttpClient.java:783) ~[graylog-plugin-aws-4.3.1.jar:?]
	at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeWithTimer(AmazonHttpClient.java:770) ~[graylog-plugin-aws-4.3.1.jar:?]
	at com.amazonaws.http.AmazonHttpClient$RequestExecutor.execute(AmazonHttpClient.java:744) ~[graylog-plugin-aws-4.3.1.jar:?]
	at com.amazonaws.http.AmazonHttpClient$RequestExecutor.access$500(AmazonHttpClient.java:704) ~[graylog-plugin-aws-4.3.1.jar:?]
	at com.amazonaws.http.AmazonHttpClient$RequestExecutionBuilderImpl.execute(AmazonHttpClient.java:686) ~[graylog-plugin-aws-4.3.1.jar:?]
	at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:550) ~[graylog-plugin-aws-4.3.1.jar:?]
	at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:530) ~[graylog-plugin-aws-4.3.1.jar:?]
	at com.amazonaws.services.sqs.AmazonSQSClient.doInvoke(AmazonSQSClient.java:2243) ~[graylog-plugin-aws-4.3.1.jar:?]
	at com.amazonaws.services.sqs.AmazonSQSClient.invoke(AmazonSQSClient.java:2210) ~[graylog-plugin-aws-4.3.1.jar:?]
	at com.amazonaws.services.sqs.AmazonSQSClient.invoke(AmazonSQSClient.java:2199) ~[graylog-plugin-aws-4.3.1.jar:?]
	at com.amazonaws.services.sqs.AmazonSQSClient.executeReceiveMessage(AmazonSQSClient.java:1637) ~[graylog-plugin-aws-4.3.1.jar:?]
	at com.amazonaws.services.sqs.AmazonSQSClient.receiveMessage(AmazonSQSClient.java:1607) ~[graylog-plugin-aws-4.3.1.jar:?]
	at org.graylog.aws.inputs.cloudtrail.notifications.CloudtrailSQSClient.getNotifications(CloudtrailSQSClient.java:64) ~[graylog-plugin-aws-4.3.1.jar:?]
	at org.graylog.aws.inputs.cloudtrail.CloudTrailSubscriber.run(CloudTrailSubscriber.java:105) [graylog-plugin-aws-4.3.1.jar:?]

GL configuration

I removed all comments and empty lines, as well as the value of the two passwords:

is_master = true
node_id_file = /etc/graylog/server/node-id
password_secret = xxx
root_password_sha2 = xxx
bin_dir = /usr/share/graylog-server/bin
data_dir = /var/lib/graylog-server
plugin_dir = /usr/share/graylog-server/plugin
http_bind_address = 127.0.0.1:9000
enabled_index_rotation_strategies = count,size,time
rotation_strategy = count
elasticsearch_max_docs_per_index = 20000000
elasticsearch_max_number_of_indices = 20
retention_strategy = delete
elasticsearch_shards = 4
elasticsearch_replicas = 0
elasticsearch_index_prefix = graylog
allow_leading_wildcard_searches = false
allow_highlighting = false
elasticsearch_analyzer = standard
output_batch_size = 500
output_flush_interval = 1
output_fault_count_threshold = 5
output_fault_penalty_seconds = 30
processbuffer_processors = 5
outputbuffer_processors = 3
processor_wait_strategy = blocking
ring_size = 65536
inputbuffer_ring_size = 65536
inputbuffer_processors = 2
inputbuffer_wait_strategy = blocking
message_journal_enabled = true
message_journal_dir = /var/lib/graylog-server/journal
lb_recognition_period_seconds = 3
mongodb_uri = mongodb://localhost/graylog
mongodb_max_connections = 1000
mongodb_threads_allowed_to_block_multiplier = 5
proxied_requests_thread_pool_size = 32

mgirod · June 8, 2022, 11:58am

Did I manage to delete my reply???
No… it was just hidden as spam…
So… I continue here.

status of ES and MongoDb service

[root@ip-172-31-34-89 ~]# systemctl status elasticsearch
● elasticsearch.service - Elasticsearch
   Loaded: loaded (/usr/lib/systemd/system/elasticsearch.service; enabled; vendor preset: disabled)
   Active: active (running) since Fri 2022-05-20 15:38:21 UTC; 2 weeks 4 days ago
     Docs: https://www.elastic.co
 Main PID: 1092 (java)
    Tasks: 56 (limit: 194419)
   Memory: 1.4G
   CGroup: /system.slice/elasticsearch.service
           └─1092 /usr/share/elasticsearch/jdk/bin/java -Xshare:auto -Des.networkaddress.cache.ttl=60 -Des.networkaddress.cache.negative.ttl=10 -XX:+AlwaysPreTouch -Xss1m -Djava.awt.headless=true -Dfile.encoding=UTF-8 -Djna.nosys=true -XX:-OmitStackTraceInFastThrow -XX:+ShowCodeDetailsInExceptionMessages -Dio.netty.noUnsafe=true -Dio.netty.noKeySetOptimization=true -Dio.netty.recycler.maxCapacityPerThread=0 -Dio.netty.allocator.numDirectArenas=0 -Dlog4j.shutdownHookEnabled=false -Dlog4j2.disable.jmx=true -Djava.locale.providers=SPI,COMPAT -Xms1g -Xmx1g -XX:+UseG1GC -XX:G1ReservePercent=25 -XX:InitiatingHeapOccupancyPercent=30 -Djava.io.tmpdir=/tmp/elasticsearch-5998333925082969722 -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=/var/lib/elasticsearch -XX:ErrorFile=/var/log/elasticsearch/hs_err_pid%p.log -Xlog:gc*,gc+age=trace,safepoint:file=/var/log/elasticsearch/gc.log:utctime,pid,tags:filecount=32,filesize=64m -XX:MaxDirectMemorySize=536870912 -Des.path.home=/usr/share/elasticsearch -Des.path.conf=/etc/elasticsearch -Des.distribution.flavor=oss -Des.distribution.type=rpm -Des.bundled_jdk=true -cp /usr/share/elasticsearch/lib/* org.elasticsearch.bootstrap.Elasticsearch -p /var/run/elasticsearch/elasticsearch.pid --quiet

May 20 15:38:02 ip-172-31-34-89.ec2.internal systemd[1]: Starting Elasticsearch...
May 20 15:38:21 ip-172-31-34-89.ec2.internal systemd[1]: Started Elasticsearch.
[root@ip-172-31-34-89 ~]# systemctl status mongod
● mongod.service - MongoDB Database Server
   Loaded: loaded (/usr/lib/systemd/system/mongod.service; enabled; vendor preset: disabled)
   Active: active (running) since Fri 2022-05-20 15:38:03 UTC; 2 weeks 4 days ago
     Docs: https://docs.mongodb.org/manual
 Main PID: 1037 (mongod)
   Memory: 546.6M
   CGroup: /system.slice/mongod.service
           └─1037 /usr/bin/mongod -f /etc/mongod.conf

May 20 15:38:00 ip-172-31-34-89.ec2.internal systemd[1]: Starting MongoDB Database Server...
May 20 15:38:01 ip-172-31-34-89.ec2.internal mongod[862]: about to fork child process, waiting until server is ready for connections.
May 20 15:38:01 ip-172-31-34-89.ec2.internal mongod[862]: forked process: 1037
May 20 15:38:03 ip-172-31-34-89.ec2.internal mongod[862]: child process started successfully, parent exiting

Elasticsearch & MongoDb logs

[root@ip-172-31-34-89 ~]# ll /var/log/elasticsearch/
total 3880
-rw-r--r--. 1 elasticsearch elasticsearch 2599219 Jun  8 12:03 gc.log
-rw-r--r--. 1 elasticsearch elasticsearch    2041 May 10 09:05 gc.log.00
-rw-r--r--. 1 elasticsearch elasticsearch    4776 May 10 09:05 gc.log.01
-rw-r--r--. 1 elasticsearch elasticsearch    2041 May 10 09:11 gc.log.02
-rw-r--r--. 1 elasticsearch elasticsearch   26154 May 10 09:11 gc.log.03
-rw-r--r--. 1 elasticsearch elasticsearch    2065 May 12 10:45 gc.log.04
-rw-r--r--. 1 elasticsearch elasticsearch   26915 May 12 10:45 gc.log.05
-rw-r--r--. 1 elasticsearch elasticsearch    2065 May 12 10:45 gc.log.06
-rw-r--r--. 1 elasticsearch elasticsearch   22265 May 12 10:45 gc.log.07
-rw-r--r--. 1 elasticsearch elasticsearch    2065 May 12 10:52 gc.log.08
-rw-r--r--. 1 elasticsearch elasticsearch 1206625 May 20 15:37 gc.log.09
-rw-r--r--. 1 elasticsearch elasticsearch    2017 May 20 15:38 gc.log.10
-rw-r--r--. 1 elasticsearch elasticsearch    2708 May 12 10:45 graylog-2022-05-10-1.json.gz
-rw-r--r--. 1 elasticsearch elasticsearch    2588 May 12 10:45 graylog-2022-05-10-1.log.gz
-rw-r--r--. 1 elasticsearch elasticsearch    3477 May 13 00:00 graylog-2022-05-12-1.json.gz
-rw-r--r--. 1 elasticsearch elasticsearch    3273 May 13 00:00 graylog-2022-05-12-1.log.gz
-rw-r--r--. 1 elasticsearch elasticsearch     428 May 18 15:58 graylog-2022-05-13-1.json.gz
-rw-r--r--. 1 elasticsearch elasticsearch     301 May 18 15:58 graylog-2022-05-13-1.log.gz
-rw-r--r--. 1 elasticsearch elasticsearch     492 May 20 15:37 graylog-2022-05-18-1.json.gz
-rw-r--r--. 1 elasticsearch elasticsearch     362 May 20 15:37 graylog-2022-05-18-1.log.gz
-rw-r--r--. 1 elasticsearch elasticsearch    2328 Jun  1 00:00 graylog-2022-05-20-1.json.gz
-rw-r--r--. 1 elasticsearch elasticsearch    2207 Jun  1 00:00 graylog-2022-05-20-1.log.gz
-rw-r--r--. 1 elasticsearch elasticsearch       0 May 10 09:05 graylog_deprecation.json
-rw-r--r--. 1 elasticsearch elasticsearch       0 May 10 09:05 graylog_deprecation.log
-rw-r--r--. 1 elasticsearch elasticsearch       0 May 10 09:05 graylog_index_indexing_slowlog.json
-rw-r--r--. 1 elasticsearch elasticsearch       0 May 10 09:05 graylog_index_indexing_slowlog.log
-rw-r--r--. 1 elasticsearch elasticsearch       0 May 10 09:05 graylog_index_search_slowlog.json
-rw-r--r--. 1 elasticsearch elasticsearch       0 May 10 09:05 graylog_index_search_slowlog.log
-rw-r--r--. 1 elasticsearch elasticsearch    1217 Jun  1 00:00 graylog.log
-rw-r--r--. 1 elasticsearch elasticsearch    2387 Jun  1 00:00 graylog_server.json
[root@ip-172-31-34-89 ~]# tail /var/log/elasticsearch/gc.log
[2022-06-08T12:03:22.827+0000][1092][gc,phases   ] GC(1003)   Other: 0.4ms
[2022-06-08T12:03:22.828+0000][1092][gc,heap     ] GC(1003) Eden regions: 613->0(613)
[2022-06-08T12:03:22.828+0000][1092][gc,heap     ] GC(1003) Survivor regions: 1->1(77)
[2022-06-08T12:03:22.828+0000][1092][gc,heap     ] GC(1003) Old regions: 52->52
[2022-06-08T12:03:22.828+0000][1092][gc,heap     ] GC(1003) Archive regions: 2->2
[2022-06-08T12:03:22.828+0000][1092][gc,heap     ] GC(1003) Humongous regions: 7->7
[2022-06-08T12:03:22.828+0000][1092][gc,metaspace] GC(1003) Metaspace: 72704K(74660K)->72704K(74660K) NonClass: 63763K(65112K)->63763K(65112K) Class: 8941K(9548K)->8941K(9548K)
[2022-06-08T12:03:22.828+0000][1092][gc          ] GC(1003) Pause Young (Normal) (G1 Evacuation Pause) 672M->59M(1024M) 2.876ms
[2022-06-08T12:03:22.828+0000][1092][gc,cpu      ] GC(1003) User=0.01s Sys=0.01s Real=0.00s
[2022-06-08T12:03:22.828+0000][1092][safepoint   ] Safepoint "G1CollectForAllocation", Time since last: 1362013335441 ns, Reaching safepoint: 130481 ns, At safepoint: 2950304 ns, Total: 3080785 ns
[root@ip-172-31-34-89 ~]# ll /var/log/mongodb/
total 228
-rw-r-----. 1 mongod mongod 231990 Jun  2 15:00 mongod.log
[root@ip-172-31-34-89 ~]# tail /var/log/mongodb/mongod.log
2022-06-02T11:31:10.858+0000 I  NETWORK  [conn26] received client metadata from 127.0.0.1:32988 conn26: { driver: { name: "mongo-java-driver|legacy", version: "3.12.1" }, os: { type: "Linux", name: "Linux", architecture: "amd64", version: "4.18.0-348.23.1.el8_5.x86_64" }, platform: "Java/Red Hat, Inc./1.8.0_312-b07" }
2022-06-02T11:31:10.860+0000 I  NETWORK  [conn27] received client metadata from 127.0.0.1:32990 conn27: { driver: { name: "mongo-java-driver|legacy", version: "3.12.1" }, os: { type: "Linux", name: "Linux", architecture: "amd64", version: "4.18.0-348.23.1.el8_5.x86_64" }, platform: "Java/Red Hat, Inc./1.8.0_312-b07" }
2022-06-02T11:31:10.865+0000 I  NETWORK  [listener] connection accepted from 127.0.0.1:32992 #28 (6 connections now open)
2022-06-02T11:31:10.865+0000 I  NETWORK  [listener] connection accepted from 127.0.0.1:32994 #29 (7 connections now open)
2022-06-02T11:31:10.865+0000 I  NETWORK  [conn28] received client metadata from 127.0.0.1:32992 conn28: { driver: { name: "mongo-java-driver|legacy", version: "3.12.1" }, os: { type: "Linux", name: "Linux", architecture: "amd64", version: "4.18.0-348.23.1.el8_5.x86_64" }, platform: "Java/Red Hat, Inc./1.8.0_312-b07" }
2022-06-02T11:31:10.865+0000 I  NETWORK  [listener] connection accepted from 127.0.0.1:32996 #30 (8 connections now open)
2022-06-02T11:31:10.867+0000 I  NETWORK  [conn30] received client metadata from 127.0.0.1:32996 conn30: { driver: { name: "mongo-java-driver|legacy", version: "3.12.1" }, os: { type: "Linux", name: "Linux", architecture: "amd64", version: "4.18.0-348.23.1.el8_5.x86_64" }, platform: "Java/Red Hat, Inc./1.8.0_312-b07" }
2022-06-02T11:31:10.869+0000 I  NETWORK  [conn29] received client metadata from 127.0.0.1:32994 conn29: { driver: { name: "mongo-java-driver|legacy", version: "3.12.1" }, os: { type: "Linux", name: "Linux", architecture: "amd64", version: "4.18.0-348.23.1.el8_5.x86_64" }, platform: "Java/Red Hat, Inc./1.8.0_312-b07" }
2022-06-02T15:00:24.606+0000 I  NETWORK  [listener] connection accepted from 127.0.0.1:35836 #31 (9 connections now open)
2022-06-02T15:00:24.607+0000 I  NETWORK  [conn31] received client metadata from 127.0.0.1:35836 conn31: { driver: { name: "mongo-java-driver|legacy", version: "3.12.1" }, os: { type: "Linux", name: "Linux", architecture: "amd64", version: "4.18.0-348.23.1.el8_5.x86_64" }, platform: "Java/Red Hat, Inc./1.8.0_312-b07" }

gsmith · June 8, 2022, 10:28pm

Hello @mgirod

Thanks for the add info but the logs you posted are either incorrect and/or incomplete.

Garbage collection logs.

tail /var/log/elasticsearch/gc.log

So I’m not sure, you have not posted anything substantial to troubleshoot this issue that I can see. Unsure what you have check or tested nor any reply on my the post above.

Most if not all GL members show what they did, either a screenshots, executing command or configurations made. This helps.

Here are some documentations you may want to look over

mgirod · June 9, 2022, 8:10am

Thanks for the add info but the logs you posted are either incorrect and/or incomplete

Sorry for that… The beginning of my answer was ‘temporarily hidden’, and I expected it would appear again, but it did not…

Our automated spam filter, Akismet, has temporarily hidden your post in Configuring graylog 4.2.9 from scratch, replicating a running 2.4.4 configuration for review.
A staff member will review your post soon, and it should appear shortly.

Anyway, time runs short for me: my last day as devops in Alison is tomorrow, so I’ll leave this baby to whomever replaces me. Thanks for your help!

system · June 23, 2022, 8:10am

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.

Topic		Replies	Views
Configuration Issue Graylog Central (peer support)	11	3757	April 7, 2021
Issue with 2.23 to 2.30 Update (Tarball) Graylog Central (peer support)	7	596	August 23, 2017
Upgrading Graylog Graylog Central (peer support)	7	903	November 17, 2017
Virtualizing my Graylog Server for Testing Purposes Graylog Central (peer support)	4	729	May 28, 2019
CentOS 8 and Graylog -UNOFFICIAL- Graylog Central (peer support)	1	379	May 1, 2020

Configuring graylog 4.2.9 from scratch, replicating a running 2.4.4 configuration

Related topics