Upgrade 3.0.1 to 3.1.3 on Centos7

Arethusa · November 13, 2019, 9:42am

Hello,

I do the upgrade like that:

# wget https://packages.graylog2.org/repo/el/stableServer/3.1/x86_64/graylog-server-3.1.3-1.noarch.rpm
# rpm -Uvh graylog-server-3.1.3-1.noarch.rpm
# systemctl restart graylog-server
# reboot (cause the port are not ok)

The version is ok:

# rpm -qi graylog-server
Name        : graylog-server
Version     : 3.1.3

The service is Running but with errors:

[root@graylog server]# systemctl status graylog-server
● graylog-server.service - Graylog server
   Loaded: loaded (/usr/lib/systemd/system/graylog-server.service; enabled; vendor preset: disabled)
   Active: active (running) since mer. 2019-11-13 09:31:31 CET; 1h 0min ago
     Docs: http://docs.graylog.org/
 Main PID: 4587 (graylog-server)
   CGroup: /system.slice/graylog-server.service
           ├─4587 /bin/sh /usr/share/graylog-server/bin/graylog-server
           └─4601 /usr/bin/java -Xms1g -Xmx1g -XX:NewRatio=1 -server -XX:+ResizeTLAB -XX:+UseConcMarkSweepGC -XX:+CMSConcurrentMTEnabled -XX:+CMSClassUnloadingEnabled -XX:-OmitSta...

nov. 13 09:31:41 graylog graylog-server[4587]: at org.graylog2.bootstrap.Main.main(Main.java:50)
nov. 13 09:31:41 graylog graylog-server[4587]: 16 errors
nov. 13 09:31:41 graylog graylog-server[4587]: at com.google.inject.internal.Errors.throwCreationExceptionIfErrorsExist(Errors.java:543)
nov. 13 09:31:41 graylog graylog-server[4587]: at com.google.inject.internal.InternalInjectorCreator.injectDynamically(InternalInjectorCreator.java:186)
nov. 13 09:31:41 graylog graylog-server[4587]: at com.google.inject.internal.InternalInjectorCreator.build(InternalInjectorCreator.java:109)
nov. 13 09:31:41 graylog graylog-server[4587]: at com.google.inject.Guice.createInjector(Guice.java:87)
nov. 13 09:31:41 graylog graylog-server[4587]: at org.graylog2.shared.bindings.GuiceInjectorHolder.createInjector(GuiceInjectorHolder.java:34)
nov. 13 09:31:41 graylog graylog-server[4587]: at org.graylog2.bootstrap.CmdLineTool.setupInjector(CmdLineTool.java:379)
nov. 13 09:31:41 graylog graylog-server[4587]: at org.graylog2.bootstrap.CmdLineTool.run(CmdLineTool.java:194)
nov. 13 09:31:41 graylog graylog-server[4587]: at org.graylog2.bootstrap.Main.main(Main.java:50)

And when I do

# netstat -tlnp

The port 9000 is not here

This is my configuration:

is_master = true
node_id_file = /etc/graylog/server/node-id
elasticsearch_max_docs_per_index = 20000000
password_secret = xx
root_password_sha2 = xx

http_bind_address = 192.168.1.206:9000

rotation_strategy = count
elasticsearch_hosts = http://192.168.1.218:9200
elasticsearch_max_number_of_indices = 20
retention_strategy = delete
elasticsearch_shards = 4
elasticsearch_replicas = 0
elasticsearch_index_prefix = graylog
allow_leading_wildcard_searches = false
allow_highlighting = false
elasticsearch_analyzer = standard
output_batch_size = 500
output_flush_interval = 1
output_fault_count_threshold = 5
output_fault_penalty_seconds = 30
processbuffer_processors = 2
outputbuffer_processors = 2
processor_wait_strategy = blocking
ring_size = 65536
inputbuffer_ring_size = 65536
inputbuffer_processors = 2
inputbuffer_wait_strategy = blocking
message_journal_enabled = true
message_journal_dir = /var/lib/graylog-server/journal
lb_recognition_period_seconds = 3
mongodb_uri = mongodb://192.168.1.207:27017/graylog
mongodb_max_connections = 1000
mongodb_threads_allowed_to_block_multiplier = 5
content_packs_dir = /usr/share/graylog-server/contentpacks
content_packs_auto_load = grok-patterns.json
proxied_requests_thread_pool_size = 32
root_timezone = Europe/Paris

Normally the configuration between 3.0 and 3.1 is the same, I read documentation, but I probably forgot something.

Thank you.

jan · November 13, 2019, 3:13pm

and what is the content of your graylog server.log?

Arethusa · November 13, 2019, 3:21pm

Hello,

2019-11-13T15:42:07.875+01:00 ERROR [CmdLineTool] Guice error (more detail on log level debug): Error injecting constructor, java.lang.IllegalArgumentException: Duplicate permission found. Permission "Permission{permission=view:create, description=Create new view}" already exists!

and

2019-11-13T16:17:41.537+01:00 ERROR [Permissions] Error adding permissions for plugin: org.graylog.plugins.enterprise.search.rest.EnterpriseSearchRestPermissions

Thank you.

jan · November 13, 2019, 3:25pm

sorry only with this snippets debugging is not possible.

Stop Graylog (if running)
move the current server.log to a different location (or rename it)
start Graylog again

Post the output here that someone might get an idea what the problem is.

Arethusa · November 13, 2019, 3:27pm

Ok, I will do it.

Maybe the probleme is from plugin enterprise 3.0.2

[root@graylog plugin]# ls
graylog-plugin-aws-3.1.3.jar  graylog-plugin-collector-3.1.3.jar  graylog-plugin-enterprise-3.0.2.jar  graylog-plugin-threatintel-3.1.3.jar  LICENSE-ENTERPRISE

jan · November 13, 2019, 3:28pm

I guess, yes. Plugins also need to be upgrade … like written in the docs.

Arethusa · November 13, 2019, 3:30pm

Log :

2019-11-13T16:28:26.437+01:00 INFO  [CmdLineTool] Loaded plugin: AWS plugins 3.1.3 [org.graylog.aws.AWSPlugin]
2019-11-13T16:28:26.440+01:00 INFO  [CmdLineTool] Loaded plugin: Collector 3.1.3 [org.graylog.plugins.collector.CollectorPlugin]
2019-11-13T16:28:26.441+01:00 INFO  [CmdLineTool] Loaded plugin: Graylog Enterprise 3.0.2 [org.graylog.plugins.enterprise.EnterprisePlugin]
2019-11-13T16:28:26.441+01:00 INFO  [CmdLineTool] Loaded plugin: Threat Intelligence Plugin 3.1.3 [org.graylog.plugins.threatintel.ThreatIntelPlugin]
2019-11-13T16:28:26.771+01:00 INFO  [CmdLineTool] Running with JVM arguments: -Xms1g -Xmx1g -XX:NewRatio=1 -XX:+ResizeTLAB -XX:+UseConcMarkSweepGC -XX:+CMSConcurrentMTEnabled -XX:+CMSClassUnloadingEnabled -XX:-OmitStackTraceInFastThrow -XX:+UseParNewGC -Dlog4j.configurationFile=file:///etc/graylog/server/log4j2.xml -Djava.library.path=/usr/share/graylog-server/lib/sigar -Dgraylog2.installation_source=rpm
2019-11-13T16:28:27.026+01:00 INFO  [Version] HV000001: Hibernate Validator 5.1.3.Final
2019-11-13T16:28:30.144+01:00 INFO  [InputBufferImpl] Message journal is enabled.
2019-11-13T16:28:30.170+01:00 INFO  [NodeId] Node ID: 0bb7c06a-122c-47b6-b4aa-0a1b18d8d825
2019-11-13T16:28:30.385+01:00 INFO  [LogManager] Loading logs.
2019-11-13T16:28:30.418+01:00 WARN  [Log] Found a corrupted index file, /var/lib/graylog-server/journal/messagejournal-0/00000000000033603452.index, deleting and rebuilding index...
2019-11-13T16:28:30.485+01:00 INFO  [LogManager] Logs loading complete.
2019-11-13T16:28:30.489+01:00 INFO  [KafkaJournal] Initialized Kafka based journal at /var/lib/graylog-server/journal
2019-11-13T16:28:30.510+01:00 INFO  [cluster] Cluster created with settings {hosts=[192.168.1.207:27017], mode=SINGLE, requiredClusterType=UNKNOWN, serverSelectionTimeout='30000 ms', maxWaitQueueSize=5000}
2019-11-13T16:28:30.552+01:00 INFO  [cluster] Cluster description not yet available. Waiting for 30000 ms before timing out
2019-11-13T16:28:30.723+01:00 INFO  [connection] Opened connection [connectionId{localValue:1, serverValue:10221}] to 192.168.1.207:27017
2019-11-13T16:28:30.735+01:00 INFO  [cluster] Monitor thread successfully connected to server with description ServerDescription{address=192.168.1.207:27017, type=STANDALONE, state=CONNECTED, ok=true, version=ServerVersion{versionList=[4, 0, 5]}, minWireVersion=0, maxWireVersion=7, maxDocumentSize=16777216, logicalSessionTimeoutMinutes=30, roundTripTimeNanos=9965503}
2019-11-13T16:28:30.763+01:00 INFO  [connection] Opened connection [connectionId{localValue:2, serverValue:10222}] to 192.168.1.207:27017
2019-11-13T16:28:31.188+01:00 INFO  [InputBufferImpl] Initialized InputBufferImpl with ring size <65536> and wait strategy <BlockingWaitStrategy>, running 2 parallel message handlers.
2019-11-13T16:28:31.562+01:00 INFO  [AbstractJestClient] Setting server pool to a list of 1 servers: [http://192.168.1.218:9200]
2019-11-13T16:28:31.563+01:00 INFO  [JestClientFactory] Using multi thread/connection supporting pooling connection manager
2019-11-13T16:28:31.693+01:00 INFO  [JestClientFactory] Using custom ObjectMapper instance
2019-11-13T16:28:31.693+01:00 INFO  [JestClientFactory] Node Discovery disabled...
2019-11-13T16:28:31.694+01:00 INFO  [JestClientFactory] Idle connection reaping disabled...
2019-11-13T16:28:32.198+01:00 WARN  [LicenseChecker] License violation - Detected irregular traffic records
2019-11-13T16:28:32.437+01:00 INFO  [ProcessBuffer] Initialized ProcessBuffer with ring size <65536> and wait strategy <BlockingWaitStrategy>.
2019-11-13T16:28:32.956+01:00 WARN  [GeoIpResolverEngine] GeoIP database file does not exist: /etc/graylog/server/GeoLite2-City.mmdb
2019-11-13T16:28:32.980+01:00 INFO  [OutputBuffer] Initialized OutputBuffer with ring size <65536> and wait strategy <BlockingWaitStrategy>.
2019-11-13T16:28:33.094+01:00 WARN  [GeoIpResolverEngine] GeoIP database file does not exist: /etc/graylog/server/GeoLite2-City.mmdb
2019-11-13T16:28:33.101+01:00 INFO  [connection] Opened connection [connectionId{localValue:3, serverValue:10223}] to 192.168.1.207:27017
2019-11-13T16:28:33.120+01:00 ERROR [Permissions] Error adding permissions for plugin: org.graylog.plugins.enterprise.search.rest.EnterpriseSearchRestPermissions
2019-11-13T16:28:33.174+01:00 ERROR [Permissions] Error adding permissions for plugin: org.graylog.plugins.enterprise.search.rest.EnterpriseSearchRestPermissions
2019-11-13T16:28:33.175+01:00 ERROR [Permissions] Error adding permissions for plugin: org.graylog.plugins.enterprise.search.rest.EnterpriseSearchRestPermissions
2019-11-13T16:28:33.187+01:00 ERROR [Permissions] Error adding permissions for plugin: org.graylog.plugins.enterprise.search.rest.EnterpriseSearchRestPermissions
2019-11-13T16:28:33.192+01:00 ERROR [Permissions] Error adding permissions for plugin: org.graylog.plugins.enterprise.search.rest.EnterpriseSearchRestPermissions
2019-11-13T16:28:33.200+01:00 ERROR [Permissions] Error adding permissions for plugin: org.graylog.plugins.enterprise.search.rest.EnterpriseSearchRestPermissions
2019-11-13T16:28:33.201+01:00 ERROR [Permissions] Error adding permissions for plugin: org.graylog.plugins.enterprise.search.rest.EnterpriseSearchRestPermissions
2019-11-13T16:28:33.202+01:00 ERROR [Permissions] Error adding permissions for plugin: org.graylog.plugins.enterprise.search.rest.EnterpriseSearchRestPermissions
2019-11-13T16:28:33.393+01:00 ERROR [Permissions] Error adding permissions for plugin: org.graylog.plugins.enterprise.search.rest.EnterpriseSearchRestPermissions
2019-11-13T16:28:33.393+01:00 ERROR [Permissions] Error adding permissions for plugin: org.graylog.plugins.enterprise.search.rest.EnterpriseSearchRestPermissions
2019-11-13T16:28:33.423+01:00 ERROR [Permissions] Error adding permissions for plugin: org.graylog.plugins.enterprise.search.rest.EnterpriseSearchRestPermissions
2019-11-13T16:28:33.457+01:00 ERROR [Permissions] Error adding permissions for plugin: org.graylog.plugins.enterprise.search.rest.EnterpriseSearchRestPermissions
2019-11-13T16:28:33.458+01:00 ERROR [Permissions] Error adding permissions for plugin: org.graylog.plugins.enterprise.search.rest.EnterpriseSearchRestPermissions
2019-11-13T16:28:33.802+01:00 ERROR [Permissions] Error adding permissions for plugin: org.graylog.plugins.enterprise.search.rest.EnterpriseSearchRestPermissions
2019-11-13T16:28:33.803+01:00 ERROR [Permissions] Error adding permissions for plugin: org.graylog.plugins.enterprise.search.rest.EnterpriseSearchRestPermissions

jan · November 13, 2019, 3:34pm

update the enterprise plugin to the same version as the Graylog server and it will work.

Arethusa · November 13, 2019, 3:43pm

Yes, I try:

[root@graylog plugin]# rpm -Uvh https://packages.graylog2.org/repo/packages/graylog-3.1-repository_latest.rpm
Récupération de https://packages.graylog2.org/repo/packages/graylog-3.1-repository_latest.rpm
Préparation...                       ################################# [100%]
Mise à jour / installation...
   1:graylog-3.1-repository-1-1       ################################# [ 50%]
Nettoyage/suppression...
   2:graylog-3.0-repository-1-6       ################################# [100%]

Then,

[root@graylog plugin]# yum install graylog-enterprise-plugins
Modules complémentaires chargés : fastestmirror
Loading mirror speeds from cached hostfile
 * base: centos.mirror.fr.planethoster.net
 * epel: mirror.imt-systems.com
 * extras: centos.mirror.fr.planethoster.net
 * updates: centos.mirror.fr.planethoster.net
Le paquet graylog-enterprise-plugins-3.0.2-1.noarch est déjà installé dans sa dernière version

That say “The graylog-enterprise-plugins-3.0.2-1.noarch package is already installed in its latest version”

Arethusa · November 13, 2019, 4:06pm

But if I move graylog-plugin-enterprise-3.0.2.jar and LICENSE-ENTERPRISE, graylog works.
So I just have to find “graylog-plugin-enterprise-3.1.3.jar”

Thank you.

jan · November 13, 2019, 4:47pm

I guess you have just downloaded the tar and placed that file in the plugin folder - and did not use the operation system package.

Please check the docs: https://docs.graylog.org/en/3.1/pages/enterprise/setup.html#rpm

I would recommend that you install the following packages and the Graylog delivered plugins will be updated when you upgrade Graylog:

yum install graylog-server graylog-enterprise-plugins graylog-integrations-plugins graylog-enterprise-integrations-plugins

Arethusa · November 14, 2019, 8:36am

Hello,

Thank you for advisement.

Usually I don’t have internet on my Graylog server, so the manipulations are not exactly the same.

Maybe we have a mistake on your link (or my interpretation is bad!)

On this link for 3.1

https://docs.graylog.org/en/3.1/pages/enterprise/setup.html#rpm

Under Tarball, the link is for 3.0.2

https://downloads.graylog.org/releases/graylog-enterprise/graylog-enterprise-plugins-3.0.2.tgz

I think we should have this one because we are on 3.1 page.

https://downloads.graylog.org/releases/graylog-enterprise/graylog-enterprise-plugins-3.1.3.tgz

Thank you for help and advisement.

jan · November 14, 2019, 10:09am

He @Arethusa

thank you for this point - I did not see this. Actually this is a bug in the documentation … I’ll fix that now.

Someitadmin · November 22, 2019, 4:35am

Next time do yum update. It will update graylog for you.

system · December 6, 2019, 4:35am

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.

Topic		Replies	Views
Upgrade from 2.3.2 to 3.1.3 provides exceptions Graylog Central (peer support)	5	895	January 17, 2020
Unable to Upgrade from 3.0.2-1 to Graylog 3.1 via yum Graylog Central (peer support)	2	1217	September 4, 2019
Problem by Update 3.0 -> 3.1 Graylog Central (peer support)	2	1246	September 9, 2019
Upgrading from 2.2 -> 2.3 Graylog Central (peer support)	3	714	October 10, 2017
Graylog Upgrade from 2.3.1 to 2.3.2 Graylog Central (peer support)	5	842	November 6, 2017

Upgrade 3.0.1 to 3.1.3 on Centos7

Related topics