Upgrade 3.0.1 to 3.1.3 on Centos7

Arethusa · November 13, 2019, 9:42am

Hello,

I do the upgrade like that:

# wget https://packages.graylog2.org/repo/el/stableServer/3.1/x86_64/graylog-server-3.1.3-1.noarch.rpm
# rpm -Uvh graylog-server-3.1.3-1.noarch.rpm
# systemctl restart graylog-server
# reboot (cause the port are not ok)

The version is ok:

# rpm -qi graylog-server
Name        : graylog-server
Version     : 3.1.3

The service is Running but with errors:

[root@graylog server]# systemctl status graylog-server
● graylog-server.service - Graylog server
   Loaded: loaded (/usr/lib/systemd/system/graylog-server.service; enabled; vendor preset: disabled)
   Active: active (running) since mer. 2019-11-13 09:31:31 CET; 1h 0min ago
     Docs: http://docs.graylog.org/
 Main PID: 4587 (graylog-server)
   CGroup: /system.slice/graylog-server.service
           ├─4587 /bin/sh /usr/share/graylog-server/bin/graylog-server
           └─4601 /usr/bin/java -Xms1g -Xmx1g -XX:NewRatio=1 -server -XX:+ResizeTLAB -XX:+UseConcMarkSweepGC -XX:+CMSConcurrentMTEnabled -XX:+CMSClassUnloadingEnabled -XX:-OmitSta...

nov. 13 09:31:41 graylog graylog-server[4587]: at org.graylog2.bootstrap.Main.main(Main.java:50)
nov. 13 09:31:41 graylog graylog-server[4587]: 16 errors
nov. 13 09:31:41 graylog graylog-server[4587]: at com.google.inject.internal.Errors.throwCreationExceptionIfErrorsExist(Errors.java:543)
nov. 13 09:31:41 graylog graylog-server[4587]: at com.google.inject.internal.InternalInjectorCreator.injectDynamically(InternalInjectorCreator.java:186)
nov. 13 09:31:41 graylog graylog-server[4587]: at com.google.inject.internal.InternalInjectorCreator.build(InternalInjectorCreator.java:109)
nov. 13 09:31:41 graylog graylog-server[4587]: at com.google.inject.Guice.createInjector(Guice.java:87)
nov. 13 09:31:41 graylog graylog-server[4587]: at org.graylog2.shared.bindings.GuiceInjectorHolder.createInjector(GuiceInjectorHolder.java:34)
nov. 13 09:31:41 graylog graylog-server[4587]: at org.graylog2.bootstrap.CmdLineTool.setupInjector(CmdLineTool.java:379)
nov. 13 09:31:41 graylog graylog-server[4587]: at org.graylog2.bootstrap.CmdLineTool.run(CmdLineTool.java:194)
nov. 13 09:31:41 graylog graylog-server[4587]: at org.graylog2.bootstrap.Main.main(Main.java:50)

And when I do

# netstat -tlnp

The port 9000 is not here

This is my configuration:

is_master = true
node_id_file = /etc/graylog/server/node-id
elasticsearch_max_docs_per_index = 20000000
password_secret = xx
root_password_sha2 = xx

http_bind_address = 192.168.1.206:9000

rotation_strategy = count
elasticsearch_hosts = http://192.168.1.218:9200
elasticsearch_max_number_of_indices = 20
retention_strategy = delete
elasticsearch_shards = 4
elasticsearch_replicas = 0
elasticsearch_index_prefix = graylog
allow_leading_wildcard_searches = false
allow_highlighting = false
elasticsearch_analyzer = standard
output_batch_size = 500
output_flush_interval = 1
output_fault_count_threshold = 5
output_fault_penalty_seconds = 30
processbuffer_processors = 2
outputbuffer_processors = 2
processor_wait_strategy = blocking
ring_size = 65536
inputbuffer_ring_size = 65536
inputbuffer_processors = 2
inputbuffer_wait_strategy = blocking
message_journal_enabled = true
message_journal_dir = /var/lib/graylog-server/journal
lb_recognition_period_seconds = 3
mongodb_uri = mongodb://192.168.1.207:27017/graylog
mongodb_max_connections = 1000
mongodb_threads_allowed_to_block_multiplier = 5
content_packs_dir = /usr/share/graylog-server/contentpacks
content_packs_auto_load = grok-patterns.json
proxied_requests_thread_pool_size = 32
root_timezone = Europe/Paris

Normally the configuration between 3.0 and 3.1 is the same, I read documentation, but I probably forgot something.

Thank you.

jan · November 13, 2019, 3:13pm

and what is the content of your graylog server.log?

Arethusa · November 13, 2019, 3:21pm

Hello,

2019-11-13T15:42:07.875+01:00 ERROR [CmdLineTool] Guice error (more detail on log level debug): Error injecting constructor, java.lang.IllegalArgumentException: Duplicate permission found. Permission "Permission{permission=view:create, description=Create new view}" already exists!

and

2019-11-13T16:17:41.537+01:00 ERROR [Permissions] Error adding permissions for plugin: org.graylog.plugins.enterprise.search.rest.EnterpriseSearchRestPermissions

Thank you.

jan · November 13, 2019, 3:25pm

sorry only with this snippets debugging is not possible.

Stop Graylog (if running)
move the current server.log to a different location (or rename it)
start Graylog again

Post the output here that someone might get an idea what the problem is.

Arethusa · November 13, 2019, 3:27pm

Ok, I will do it.

Maybe the probleme is from plugin enterprise 3.0.2

[root@graylog plugin]# ls
graylog-plugin-aws-3.1.3.jar  graylog-plugin-collector-3.1.3.jar  graylog-plugin-enterprise-3.0.2.jar  graylog-plugin-threatintel-3.1.3.jar  LICENSE-ENTERPRISE

jan · November 13, 2019, 3:28pm

I guess, yes. Plugins also need to be upgrade … like written in the docs.

Arethusa · November 13, 2019, 3:30pm

Log :

2019-11-13T16:28:26.437+01:00 INFO  [CmdLineTool] Loaded plugin: AWS plugins 3.1.3 [org.graylog.aws.AWSPlugin]
2019-11-13T16:28:26.440+01:00 INFO  [CmdLineTool] Loaded plugin: Collector 3.1.3 [org.graylog.plugins.collector.CollectorPlugin]
2019-11-13T16:28:26.441+01:00 INFO  [CmdLineTool] Loaded plugin: Graylog Enterprise 3.0.2 [org.graylog.plugins.enterprise.EnterprisePlugin]
2019-11-13T16:28:26.441+01:00 INFO  [CmdLineTool] Loaded plugin: Threat Intelligence Plugin 3.1.3 [org.graylog.plugins.threatintel.ThreatIntelPlugin]
2019-11-13T16:28:26.771+01:00 INFO  [CmdLineTool] Running with JVM arguments: -Xms1g -Xmx1g -XX:NewRatio=1 -XX:+ResizeTLAB -XX:+UseConcMarkSweepGC -XX:+CMSConcurrentMTEnabled -XX:+CMSClassUnloadingEnabled -XX:-OmitStackTraceInFastThrow -XX:+UseParNewGC -Dlog4j.configurationFile=file:///etc/graylog/server/log4j2.xml -Djava.library.path=/usr/share/graylog-server/lib/sigar -Dgraylog2.installation_source=rpm
2019-11-13T16:28:27.026+01:00 INFO  [Version] HV000001: Hibernate Validator 5.1.3.Final
2019-11-13T16:28:30.144+01:00 INFO  [InputBufferImpl] Message journal is enabled.
2019-11-13T16:28:30.170+01:00 INFO  [NodeId] Node ID: 0bb7c06a-122c-47b6-b4aa-0a1b18d8d825
2019-11-13T16:28:30.385+01:00 INFO  [LogManager] Loading logs.
2019-11-13T16:28:30.418+01:00 WARN  [Log] Found a corrupted index file, /var/lib/graylog-server/journal/messagejournal-0/00000000000033603452.index, deleting and rebuilding index...
2019-11-13T16:28:30.485+01:00 INFO  [LogManager] Logs loading complete.
2019-11-13T16:28:30.489+01:00 INFO  [KafkaJournal] Initialized Kafka based journal at /var/lib/graylog-server/journal
2019-11-13T16:28:30.510+01:00 INFO  [cluster] Cluster created with settings {hosts=[192.168.1.207:27017], mode=SINGLE, requiredClusterType=UNKNOWN, serverSelectionTimeout='30000 ms', maxWaitQueueSize=5000}
2019-11-13T16:28:30.552+01:00 INFO  [cluster] Cluster description not yet available. Waiting for 30000 ms before timing out
2019-11-13T16:28:30.723+01:00 INFO  [connection] Opened connection [connectionId{localValue:1, serverValue:10221}] to 192.168.1.207:27017
2019-11-13T16:28:30.735+01:00 INFO  [cluster] Monitor thread successfully connected to server with description ServerDescription{address=192.168.1.207:27017, type=STANDALONE, state=CONNECTED, ok=true, version=ServerVersion{versionList=[4, 0, 5]}, minWireVersion=0, maxWireVersion=7, maxDocumentSize=16777216, logicalSessionTimeoutMinutes=30, roundTripTimeNanos=9965503}
2019-11-13T16:28:30.763+01:00 INFO  [connection] Opened connection [connectionId{localValue:2, serverValue:10222}] to 192.168.1.207:27017
2019-11-13T16:28:31.188+01:00 INFO  [InputBufferImpl] Initialized InputBufferImpl with ring size <65536> and wait strategy <BlockingWaitStrategy>, running 2 parallel message handlers.
2019-11-13T16:28:31.562+01:00 INFO  [AbstractJestClient] Setting server pool to a list of 1 servers: [http://192.168.1.218:9200]
2019-11-13T16:28:31.563+01:00 INFO  [JestClientFactory] Using multi thread/connection supporting pooling connection manager
2019-11-13T16:28:31.693+01:00 INFO  [JestClientFactory] Using custom ObjectMapper instance
2019-11-13T16:28:31.693+01:00 INFO  [JestClientFactory] Node Discovery disabled...
2019-11-13T16:28:31.694+01:00 INFO  [JestClientFactory] Idle connection reaping disabled...
2019-11-13T16:28:32.198+01:00 WARN  [LicenseChecker] License violation - Detected irregular traffic records
2019-11-13T16:28:32.437+01:00 INFO  [ProcessBuffer] Initialized ProcessBuffer with ring size <65536> and wait strategy <BlockingWaitStrategy>.
2019-11-13T16:28:32.956+01:00 WARN  [GeoIpResolverEngine] GeoIP database file does not exist: /etc/graylog/server/GeoLite2-City.mmdb
2019-11-13T16:28:32.980+01:00 INFO  [OutputBuffer] Initialized OutputBuffer with ring size <65536> and wait strategy <BlockingWaitStrategy>.
2019-11-13T16:28:33.094+01:00 WARN  [GeoIpResolverEngine] GeoIP database file does not exist: /etc/graylog/server/GeoLite2-City.mmdb
2019-11-13T16:28:33.101+01:00 INFO  [connection] Opened connection [connectionId{localValue:3, serverValue:10223}] to 192.168.1.207:27017
2019-11-13T16:28:33.120+01:00 ERROR [Permissions] Error adding permissions for plugin: org.graylog.plugins.enterprise.search.rest.EnterpriseSearchRestPermissions
2019-11-13T16:28:33.174+01:00 ERROR [Permissions] Error adding permissions for plugin: org.graylog.plugins.enterprise.search.rest.EnterpriseSearchRestPermissions
2019-11-13T16:28:33.175+01:00 ERROR [Permissions] Error adding permissions for plugin: org.graylog.plugins.enterprise.search.rest.EnterpriseSearchRestPermissions
2019-11-13T16:28:33.187+01:00 ERROR [Permissions] Error adding permissions for plugin: org.graylog.plugins.enterprise.search.rest.EnterpriseSearchRestPermissions
2019-11-13T16:28:33.192+01:00 ERROR [Permissions] Error adding permissions for plugin: org.graylog.plugins.enterprise.search.rest.EnterpriseSearchRestPermissions
2019-11-13T16:28:33.200+01:00 ERROR [Permissions] Error adding permissions for plugin: org.graylog.plugins.enterprise.search.rest.EnterpriseSearchRestPermissions
2019-11-13T16:28:33.201+01:00 ERROR [Permissions] Error adding permissions for plugin: org.graylog.plugins.enterprise.search.rest.EnterpriseSearchRestPermissions
2019-11-13T16:28:33.202+01:00 ERROR [Permissions] Error adding permissions for plugin: org.graylog.plugins.enterprise.search.rest.EnterpriseSearchRestPermissions
2019-11-13T16:28:33.393+01:00 ERROR [Permissions] Error adding permissions for plugin: org.graylog.plugins.enterprise.search.rest.EnterpriseSearchRestPermissions
2019-11-13T16:28:33.393+01:00 ERROR [Permissions] Error adding permissions for plugin: org.graylog.plugins.enterprise.search.rest.EnterpriseSearchRestPermissions
2019-11-13T16:28:33.423+01:00 ERROR [Permissions] Error adding permissions for plugin: org.graylog.plugins.enterprise.search.rest.EnterpriseSearchRestPermissions
2019-11-13T16:28:33.457+01:00 ERROR [Permissions] Error adding permissions for plugin: org.graylog.plugins.enterprise.search.rest.EnterpriseSearchRestPermissions
2019-11-13T16:28:33.458+01:00 ERROR [Permissions] Error adding permissions for plugin: org.graylog.plugins.enterprise.search.rest.EnterpriseSearchRestPermissions
2019-11-13T16:28:33.802+01:00 ERROR [Permissions] Error adding permissions for plugin: org.graylog.plugins.enterprise.search.rest.EnterpriseSearchRestPermissions
2019-11-13T16:28:33.803+01:00 ERROR [Permissions] Error adding permissions for plugin: org.graylog.plugins.enterprise.search.rest.EnterpriseSearchRestPermissions

jan · November 13, 2019, 3:34pm

update the enterprise plugin to the same version as the Graylog server and it will work.

Arethusa · November 13, 2019, 3:43pm

Yes, I try:

[root@graylog plugin]# rpm -Uvh https://packages.graylog2.org/repo/packages/graylog-3.1-repository_latest.rpm
Récupération de https://packages.graylog2.org/repo/packages/graylog-3.1-repository_latest.rpm
Préparation...                       ################################# [100%]
Mise à jour / installation...
   1:graylog-3.1-repository-1-1       ################################# [ 50%]
Nettoyage/suppression...
   2:graylog-3.0-repository-1-6       ################################# [100%]

Then,

[root@graylog plugin]# yum install graylog-enterprise-plugins
Modules complémentaires chargés : fastestmirror
Loading mirror speeds from cached hostfile
 * base: centos.mirror.fr.planethoster.net
 * epel: mirror.imt-systems.com
 * extras: centos.mirror.fr.planethoster.net
 * updates: centos.mirror.fr.planethoster.net
Le paquet graylog-enterprise-plugins-3.0.2-1.noarch est déjà installé dans sa dernière version

That say “The graylog-enterprise-plugins-3.0.2-1.noarch package is already installed in its latest version”

Arethusa · November 13, 2019, 4:06pm

But if I move graylog-plugin-enterprise-3.0.2.jar and LICENSE-ENTERPRISE, graylog works.
So I just have to find “graylog-plugin-enterprise-3.1.3.jar”

Thank you.

jan · November 13, 2019, 4:47pm

I guess you have just downloaded the tar and placed that file in the plugin folder - and did not use the operation system package.

Please check the docs: https://docs.graylog.org/en/3.1/pages/enterprise/setup.html#rpm

I would recommend that you install the following packages and the Graylog delivered plugins will be updated when you upgrade Graylog:

yum install graylog-server graylog-enterprise-plugins graylog-integrations-plugins graylog-enterprise-integrations-plugins

Arethusa · November 14, 2019, 8:36am

Hello,

Thank you for advisement.

Usually I don’t have internet on my Graylog server, so the manipulations are not exactly the same.

Maybe we have a mistake on your link (or my interpretation is bad!)

On this link for 3.1

https://docs.graylog.org/en/3.1/pages/enterprise/setup.html#rpm

Under Tarball, the link is for 3.0.2

https://downloads.graylog.org/releases/graylog-enterprise/graylog-enterprise-plugins-3.0.2.tgz

I think we should have this one because we are on 3.1 page.

https://downloads.graylog.org/releases/graylog-enterprise/graylog-enterprise-plugins-3.1.3.tgz

Thank you for help and advisement.

jan · November 14, 2019, 10:09am

He @Arethusa

thank you for this point - I did not see this. Actually this is a bug in the documentation … I’ll fix that now.

Someitadmin · November 22, 2019, 4:35am

Next time do yum update. It will update graylog for you.

system · December 6, 2019, 4:35am

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.

Topic		Replies	Views
Problem by Update 3.0 -> 3.1 Graylog Central (peer support)	2	1214	September 9, 2019
Upgrading from 2.2 -> 2.3 Graylog Central (peer support)	3	681	October 10, 2017
Graylog update on OVA image Graylog Central (peer support)	3	2315	October 1, 2019
Graylog 4.3 Problem installing Graylog Central (peer support)	5	799	June 14, 2022
Graylog installation throwing 503 Graylog Central (peer support)	3	781	January 12, 2018

Upgrade 3.0.1 to 3.1.3 on Centos7

Related Topics