“Show Receiver messages”, the Loading never stop

Hello everyone.

I have a classical problem, when I do “Show Receiver messages”, the Loading never stop.

Let me explain my context,

I have 4 Virtual Machine:

Graylog : 192.168.159.163  (Graylog 2.4.6)
MongoDB :192.168.159.165 (db version v2.6.12)
Elasticsearch : 192.168.159.159 ("lucene_version" : "5.5.4" , "number" : "2.4.6")
Rsyslog : 192.168.159.166

This is my configurations:

Graylog :

[root@graylogv2 ~]# netstat -tlnp
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address           Foreign Address         State       PID/Program name
tcp        0      0 127.0.0.1:25            0.0.0.0:*               LISTEN      1071/master
tcp        0      0 0.0.0.0:1025            0.0.0.0:*               LISTEN      1373/java
tcp        0      0 192.168.159.163:9000    0.0.0.0:*               LISTEN      1373/java
tcp        0      0 0.0.0.0:22              0.0.0.0:*               LISTEN      899/sshd

serveur.conf

is_master = true
node_id_file = /etc/graylog/server/node-id
elasticsearch_max_docs_per_index = 20000000
password_secret = 6HEHVhdIYNEzao1xDSLGW456789V8qj5cU3mBDPdcRp2YJNMdfCJgWQgOFzTqNZFtMJjcyWTxVmDmBVXPZMoY1mwny9
root_password_sha2 = bb4cd01aa7c719c31234567115e95970af73d3a2c0d6202ceaf3d1183928b
plugin_dir = /usr/share/graylog-server/plugin
rest_listen_uri = http://192.168.159.163:9000/api/
#rest_listen_uri = http://127.0.0.1:9000/api/
web_listen_uri = http://192.168.159.163:9000/
#web_listen_uri = http://127.0.0.1:9000/
rotation_strategy = count
elasticsearch_hosts = http://192.168.159.159:9200
elasticsearch_max_number_of_indices = 20
retention_strategy = delete
elasticsearch_shards = 4
elasticsearch_replicas = 0
elasticsearch_index_prefix = graylog
allow_leading_wildcard_searches = false
allow_highlighting = false
elasticsearch_analyzer = standard
output_batch_size = 500
output_flush_interval = 1
output_fault_count_threshold = 5
output_fault_penalty_seconds = 30
processbuffer_processors = 2
outputbuffer_processors = 2
processor_wait_strategy = blocking
ring_size = 65536
inputbuffer_ring_size = 65536
inputbuffer_processors = 2
inputbuffer_wait_strategy = blocking
message_journal_enabled = true
message_journal_dir = /var/lib/graylog-server/journal
lb_recognition_period_seconds = 3
mongodb_uri = mongodb://192.168.159.165:27017/graylog
mongodb_max_connections = 1000
mongodb_threads_allowed_to_block_multiplier = 5
content_packs_dir = /usr/share/graylog-server/contentpacks
content_packs_auto_load = grok-patterns.json
proxied_requests_thread_pool_size = 32
root_timezone = Europe/Paris

Mongodb

vi /etc/mongod.conf
bind_ip = 192.168.159.165

Netstat:

[root@mongov2 ~]# netstat -tlnp
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address           Foreign Address         State       PID/Program name
tcp        0      0 127.0.0.1:25            0.0.0.0:*               LISTEN      1137/master     
tcp        0      0 192.168.159.165:27017   0.0.0.0:*               LISTEN      22457/mongod  

ElasticSearch

vi /etc/elasticsearch/elasticsearch.yml

cluster.name: graylog
network.host: ["127.0.0.1","192.168.159.159"]

netstat

[root@elastic ~]# netstat -tlnp
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address           Foreign Address         State       PID/Program name
tcp        0      0 127.0.0.1:25            0.0.0.0:*               LISTEN      1149/master
tcp        0      0 0.0.0.0:22              0.0.0.0:*               LISTEN      1020/sshd
tcp6       0      0 ::1:25                  :::*                    LISTEN      1149/master
tcp6       0      0 192.168.159.159:9200    :::*                    LISTEN      1026/java
tcp6       0      0 127.0.0.1:9200          :::*                    LISTEN      1026/java
tcp6       0      0 192.168.159.159:9300    :::*                    LISTEN      1026/java
tcp6       0      0 127.0.0.1:9300          :::*                    LISTEN      1026/java
tcp6       0      0 :::22                   :::*                    LISTEN      1020/sshd

Rsyslog

On rsyslog.conf

# Provides TCP syslog reception
$ModLoad imtcp
$InputTCPServerRun 1025

$template RemoteLogsTesting,"/var/log/remotehosts/%HOSTNAME%/%$now%.log"
if $fromhost-ip != '127.0.0.1' then -?RemoteLogsTesting
& stop

*.* @@192.168.159.163:1025

My all log coming on rsyslog

[root@rsyslog remotehosts]# tree
.
├── elastic
│   ├── 2018-10-01.log
│   ├── 2018-10-02.log
│   ├── 2018-10-03.log
│   └── 2018-10-04.log
├── graylogv2
│   ├── 2018-10-01.log
│   ├── 2018-10-02.log
│   ├── 2018-10-03.log
│   └── 2018-10-04.log
└── mongov2
    ├── 2018-10-01.log
    ├── 2018-10-02.log
    ├── 2018-10-03.log
    └── 2018-10-04.log

This is my imput:

I supose i do a mistake or I forgot somthing !

Thank you for reading.

Edit : I just try to disable IPv6 on all Vms.

And upgrade Mongodb:

[root@mongoV2 ~]# mongod --version
db version v4.0.3
git version: 7ea530946fa7880364d88c8d8b6026bbc9ffa48c
OpenSSL version: OpenSSL 1.0.1e-fips 11 Feb 2013
allocator: tcmalloc
modules: none
build environment:
    distmod: rhel70
    distarch: x86_64
    target_arch: x86_64 

The problem is the same.

Thank you.

Hello,

When I do

[root@graylogv2 ~]# tcpdump -i ens33 host 192.168.159.166 and tcp port 1025

I have :

tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on ens33, link-type EN10MB (Ethernet), capture size 262144 bytes
10:36:16.095306 IP elastic.38956 > rsyslog.blackjack: Flags [P.], seq 1521063633:1521063831, ack 834329731, win 229, options [nop,nop,TS val 4845135 ecr 4259073], length 198
10:36:16.095344 IP rsyslog.blackjack > elastic.38956: Flags [.], ack 198, win 463, options [nop,nop,TS val 4434358 ecr 4845135], length 0
10:36:16.098380 IP elastic.38956 > rsyslog.blackjack: Flags [P.], seq 198:291, ack 1, win 229, options [nop,nop,TS val 4845138 ecr 4434358], length 93
10:36:16.098468 IP rsyslog.blackjack > elastic.38956: Flags [.], ack 291, win 463, options [nop,nop,TS val 4434362 ecr 4845138], length 0
10:36:16.100600 IP elastic.38956 > rsyslog.blackjack: Flags [P.], seq 291:354, ack 1, win 229, options [nop,nop,TS val 4845140 ecr 4434362], length 63
10:36:16.100690 IP rsyslog.blackjack > elastic.38956: Flags [.], ack 354, win 463, options [nop,nop,TS val 4434364 ecr 4845140], length 0
10:36:16.101724 IP elastic.38956 > rsyslog.blackjack: Flags [P.], seq 354:425, ack 1, win 229, options [nop,nop,TS val 4845141 ecr 4434364], length 71
10:36:16.101830 IP rsyslog.blackjack > elastic.38956: Flags [.], ack 425, win 463, options [nop,nop,TS val 4434365 ecr 4845141], length 0
10:36:16.102949 IP elastic.38956 > rsyslog.blackjack: Flags [P.], seq 425:491, ack 1, win 229, options [nop,nop,TS val 4845142 ecr 4434365], length 66

So the connexion is good I think

Thank you

on the right of the input, you have metrics displayed. Did you see any stats counting?

1 Like

Hello Jan,
Thank you for your time.

I supose I have not metrics.

I dont understant why because the tcpdump on port 1025 works.

For information, i have upgrade Elasticseach to 5.6 from 2.x. The problem is the same.

Thank you.

I try other configuration, the “Total” move a bit:

But the “Loading” status stay the same :frowning:

Thank you.

Hello,

I solve the probleme.

It come from Elasticsearch, I have beging with version 2, and upgrade for version 5.

I do the same configuration from scratch, but I gegin by the version 5, and now that works.
Normaly I could deleted and recreate node, but now all is ok.

Thank you for your advisement.

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.