Upgrade to 3.2 server don't show logs from sidecars and inputs

Graylog Central (peer support)
1

Hello,

After upgrade to version 3.2 from 3.1.x server don’t shows logs from sidecars and inputs. If i try go to inputs and click on “Show received messages” or go to sidecars and click on “show messages” i have only blank page with “Loading…” and no happen nothing. Below is my steps for upgrade and platform info.

Platform: Ubuntu 18.04.4 on ESXi 6.7
Deploy: from OVA image

I done steps from links:
1.#update-ova-to-latest-version
2.#operating-package-upgrade-deb-apt

and I took the steps:

$ wget https://packages.graylog2.org/repo/packages/graylog-3.2-repository_latest.deb
$ sudo dpkg -i graylog-3.2-repository_latest.deb
$ sudo apt-get update
$ sudo apt-get install graylog-server

After this i took the steps from:
https://docs.graylog.org/en/3.2/pages/upgrade/graylog-3.2.html

and before start services run out command:

curl -s localhost:9200/_cat/aliases/*_deflector?h=index

but this command returns nothing, so i don’t do rest of commands. After above steps i start services (graylog-server and elasticsearch) and i was check status of services (started) and logs collecting (problem).

Where can be problem?

Thank you for any help!

2

Hello,

Can anyone help with this?

3

It is only for POC use, not for live system. Install a fresh OVA.

How would you like to connect a stopped service?
The curl try to connect the elasticsearch. And you should get a error message.

4

Thanks macko003 for answer,

OVA is for POC - i know but i needed quick implementation. If the current problem will be resolved I have time to carry out the implementation from package.

How would you like to connect a stopped service?
The curl try to connect the elasticsearch. And you should get a error message.

Why stopped services? I checked with systemctl status graylog-server elasticsearch after upgrade whether services are active and running and after that i was check logs collecting.

Output of:

curl -XGET 'http://127.0.0.1:9200/_cluster/health?pretty=true'

give me status green without errors:

{
  "cluster_name" : "graylog",
  "status" : "green",
  "timed_out" : false,
  "number_of_nodes" : 1,
  "number_of_data_nodes" : 1,
  "active_primary_shards" : 72,
  "active_shards" : 72,
  "relocating_shards" : 0,
  "initializing_shards" : 0,
  "unassigned_shards" : 0,
  "delayed_unassigned_shards" : 0,
  "number_of_pending_tasks" : 0,
  "number_of_in_flight_fetch" : 0,
  "task_max_waiting_in_queue_millis" : 0,
  "active_shards_percent_as_number" : 100.0
}
5

My private opinion, a clear install is faster then import the ova, set the timezone, admin password, etc…

You wrote

This is why I thought. And you wrote you got nothing.

If you don’t get back the deflectors, please check the community, sometimes it happened under the update.

It can also cause your loading message problem.
As far as I know you should have error message about that in the server.log.

6

Hi macko003,

Sorry for delay with respond.

But deflectors should working fine referring to server.log:

2020-03-20T17:57:58.655+01:00 INFO  [MongoIndexSet] Cycling from <graylog_5> to <graylog_6>.
2020-03-20T17:57:58.655+01:00 INFO  [MongoIndexSet] Creating target index <graylog_6>.
2020-03-20T17:57:58.679+01:00 INFO  [Indices] Successfully created index template graylog-internal
2020-03-20T17:57:58.852+01:00 INFO  [MongoIndexSet] Waiting for allocation of index <graylog_6>.
2020-03-20T17:57:58.854+01:00 INFO  [MongoIndexSet] Index <graylog_6> has been successfully allocated.
2020-03-20T17:57:58.854+01:00 INFO  [MongoIndexSet] Pointing index alias <graylog_deflector> to new index <graylog_6>.
2020-03-20T17:57:58.891+01:00 INFO  [SystemJobManager] Submitted SystemJob <f3c523b0-6acb-11ea-bb1c-000c298d17da> [org.graylog2.indexer.indices.jobs.SetInde$
2020-03-20T17:57:58.891+01:00 INFO  [MongoIndexSet] Successfully pointed index alias <graylog_deflector> to index <graylog_6>.
2020-03-20T17:58:28.913+01:00 INFO  [SetIndexReadOnlyJob] Flushing old index <graylog_5>.
2020-03-20T17:58:29.535+01:00 INFO  [SetIndexReadOnlyJob] Setting old index <graylog_5> to read-only.
2020-03-20T17:58:29.569+01:00 INFO  [SystemJobManager] Submitted SystemJob <060e3b10-6acc-11ea-bb1c-000c298d17da> [org.graylog2.indexer.indices.jobs.Optimiz$
2020-03-20T17:58:29.593+01:00 INFO  [CreateNewSingleIndexRangeJob] Calculating ranges for index graylog_5.
2020-03-20T17:58:29.596+01:00 INFO  [OptimizeIndexJob] Optimizing index <graylog_5>.
2020-03-20T17:58:32.960+01:00 INFO  [MongoIndexRangeService] Calculated range of [graylog_5] in [3365ms].
2020-03-20T17:58:32.961+01:00 INFO  [CreateNewSingleIndexRangeJob] Created ranges for index graylog_5.

And i don’t have any error with deflectors but i have warn with beats:

2020-03-12T14:35:05.998+01:00 WARN  [UdpTransport] receiveBufferSize (SO_RCVBUF) for input RawUDPInput{title=global_beats_for_networking, type=org.graylog2.inputs.raw.udp.RawUDPInput, nodeId=d51371f2-6491-4327-967a-085f382c757f} (channel [id: 0x10cf64ed, L:/0:0:0:0:0:0:0:0%0:11001]) should be 262144 but is 425984.
7

Hello,

Update:

I again took steps from:
https://docs.graylog.org/en/3.2/pages/installation/operating_system_packages.html#operating-package-upgrade-deb-apt

including:

sudo apt-get install graylog-server graylog-enterprise-plugins graylog-integrations-plugins graylog-enterprise-integrations-plugins

and

for index in `curl -s localhost:9200/_cat/aliases/*_deflector?h=index`; do curl -s -X PUT --data '{"properties":{"gl2_accounted_message_size":{"type": "long"}}}' -H Content-Type:application/json localhost:9200/$index/_mapping/message ; done

After above steps and restart services i’m able to see logs from sidecars and inputs.

Thank You!

8

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.