Graylog web interface problem!

xked · February 4, 2019, 2:13am

Hi. I’ve a problem. Elasticsearch, mongodb is working. But graylog service has problem.

/etc/elasticsearch/elasticsearch.yml

cluster.name: graylog
network.host: 192.168.1.222
http.port: 9200

[root@localhost]# curl -XGET ‘http://192.168.1.222:9200/_cluster/health?pretty=true’

{
“cluster_name” : “graylog”,
“status” : “green”,
“timed_out” : false,
“number_of_nodes” : 1,
“number_of_data_nodes” : 1,
“active_primary_shards” : 0,
“active_shards” : 0,
“relocating_shards” : 0,
“initializing_shards” : 0,
“unassigned_shards” : 0,
“delayed_unassigned_shards” : 0,
“number_of_pending_tasks” : 0,
“number_of_in_flight_fetch” : 0,
“task_max_waiting_in_queue_millis” : 0,
“active_shards_percent_as_number” : 100.0
}

/etc/graylog/server/server.conf

password_secret = <my_pass_secret>
root_password_sha2 = <my_pass_sha2>
rest_listen_uri = http://192.168.1.222:12900/
web_listen_uri = http://192.168.1.222:9000/

[root@localhost]# systemctl status graylog-server.service
● graylog-server.service - Graylog server
Loaded: loaded (/usr/lib/systemd/system/graylog-server.service; enabled; vendor preset: disabled)
Active: activating (auto-restart) (Result: exit-code) since Pzt 2019-02-04 05:05:46 +03; 3s ago

Process: 20798 ExecStart=/usr/share/graylog-server/bin/graylog-server (code=exited, status=1/FAILURE)
Main PID: 20798 (code=exited, status=1/FAILURE)

Şub 04 05:05:46 localhost.localdomain systemd[1]: graylog-server.service: main process exited, code=exited, status=1/FAILURE
Şub 04 05:05:46 localhost.localdomain systemd[1]: Unit graylog-server.service entered failed state.
Şub 04 05:05:46 localhost.localdomainsystemd[1]: graylog-server.service failed.

Logs: /var/log/graylog-server - last lines -

2019-02-05T00:09:51.663+03:00 INFO [CmdLineTool] Loaded plugin: AWS plugins 2.5.1 [org.graylog.aws.plugin.AWSPlugin]
2019-02-05T00:09:51.665+03:00 INFO [CmdLineTool] Loaded plugin: Elastic Beats Input 2.5.1 [org.graylog.plugins.beats.BeatsInputPlugin]
2019-02-05T00:09:51.666+03:00 INFO [CmdLineTool] Loaded plugin: CEF Input 2.5.1 [org.graylog.plugins.cef.CEFInputPlugin]
2019-02-05T00:09:51.667+03:00 INFO [CmdLineTool] Loaded plugin: Collector 2.5.1 [org.graylog.plugins.collector.CollectorPlugin]
2019-02-05T00:09:51.668+03:00 INFO [CmdLineTool] Loaded plugin: Enterprise Integration Plugin 2.5.1 [org.graylog.plugins.enterprise_integration.EnterpriseIntegrationPlugin]
2019-02-05T00:09:51.669+03:00 INFO [CmdLineTool] Loaded plugin: MapWidgetPlugin 2.5.1 [org.graylog.plugins.map.MapWidgetPlugin]
2019-02-05T00:09:51.670+03:00 INFO [CmdLineTool] Loaded plugin: NetFlow Plugin 2.5.1 [org.graylog.plugins.netflow.NetFlowPlugin]
2019-02-05T00:09:51.676+03:00 INFO [CmdLineTool] Loaded plugin: Pipeline Processor Plugin 2.5.1 [org.graylog.plugins.pipelineprocessor.ProcessorPlugin]
2019-02-05T00:09:51.677+03:00 INFO [CmdLineTool] Loaded plugin: Threat Intelligence Plugin 2.5.1 [org.graylog.plugins.threatintel.ThreatIntelPlugin]
2019-02-05T00:09:52.008+03:00 INFO [CmdLineTool] Running with JVM arguments: -Xms1g -Xmx1g -XX:NewRatio=1 -XX:+ResizeTLAB -XX:+UseConcMarkSweepGC -XX:+CMSConcurrentMTEnabled -XX:+CMSClassUnloadingEnabled -XX:+UseParNewGC -XX:-OmitStackTraceInFastThrow -Dlog4j.configurationFile=file:///etc/graylog/server/log4j2.xml -Djava.library.path=/usr/share/graylog-server/lib/sigar -Dgraylog2.installation_source=rpm
2019-02-05T00:09:52.218+03:00 INFO [Version] HV000001: Hibernate Validator 5.1.3.Final
2019-02-05T00:09:55.218+03:00 INFO [InputBufferImpl] Message journal is enabled.
2019-02-05T00:09:55.246+03:00 INFO [NodeId] Node ID: a80a69d0-0d07-434d-aef1-3bd81e8afc37
2019-02-05T00:09:55.547+03:00 INFO [LogManager] Loading logs.
2019-02-05T00:09:55.595+03:00 WARN [Log] Found a corrupted index file, /var/lib/graylog-server/journal/messagejournal-0/00000000000000000000.index, deleting and rebuilding index…
2019-02-05T00:09:55.649+03:00 INFO [LogManager] Logs loading complete.
2019-02-05T00:09:55.649+03:00 INFO [KafkaJournal] Initialized Kafka based journal at /var/lib/graylog-server/journal
2019-02-05T00:09:55.681+03:00 INFO [InputBufferImpl] Initialized InputBufferImpl with ring size <65536> and wait strategy , running 2 parallel message handlers.
2019-02-05T00:09:55.717+03:00 INFO [cluster] Cluster created with settings {hosts=[localhost:27017], mode=SINGLE, requiredClusterType=UNKNOWN, serverSelectionTimeout=‘30000 ms’, maxWaitQueueSize=5000}
2019-02-05T00:09:55.795+03:00 INFO [cluster] No server chosen by ReadPreferenceServerSelector{readPreference=primary} from cluster description ClusterDescription{type=UNKNOWN, connectionMode=SINGLE, serverDescriptions=[ServerDescription{address=localhost:27017, type=UNKNOWN, state=CONNECTING}]}. Waiting for 30000 ms before timing out
2019-02-05T00:09:55.813+03:00 INFO [connection] Opened connection [connectionId{localValue:1, serverValue:22}] to localhost:27017
2019-02-05T00:09:55.817+03:00 INFO [cluster] Monitor thread successfully connected to server with description ServerDescription{address=localhost:27017, type=STANDALONE, state=CONNECTED, ok=true, version=ServerVersion{versionList=[3, 6, 10]}, minWireVersion=0, maxWireVersion=6, maxDocumentSize=16777216, roundTripTimeNanos=867808}
2019-02-05T00:09:55.831+03:00 INFO [connection] Opened connection [connectionId{localValue:2, serverValue:23}] to localhost:27017
2019-02-05T00:09:56.348+03:00 INFO [AbstractJestClient] Setting server pool to a list of 1 servers: [http://192.168.1.222:92000]
2019-02-05T00:09:56.349+03:00 INFO [JestClientFactory] Using multi thread/connection supporting pooling connection manager
2019-02-05T00:09:56.454+03:00 INFO [JestClientFactory] Using custom ObjectMapper instance
2019-02-05T00:09:56.454+03:00 INFO [JestClientFactory] Node Discovery disabled…
2019-02-05T00:09:56.455+03:00 INFO [JestClientFactory] Idle connection reaping disabled…
2019-02-05T00:09:56.837+03:00 INFO [ProcessBuffer] Initialized ProcessBuffer with ring size <65536> and wait strategy .
2019-02-05T00:09:58.950+03:00 INFO [OutputBuffer] Initialized OutputBuffer with ring size <65536> and wait strategy .
2019-02-05T00:09:58.999+03:00 INFO [ProcessBuffer] Initialized ProcessBuffer with ring size <65536> and wait strategy .
2019-02-05T00:09:59.227+03:00 INFO [ProcessBuffer] Initialized ProcessBuffer with ring size <65536> and wait strategy .
2019-02-05T00:09:59.455+03:00 INFO [ProcessBuffer] Initialized ProcessBuffer with ring size <65536> and wait strategy .
2019-02-05T00:09:59.631+03:00 INFO [ProcessBuffer] Initialized ProcessBuffer with ring size <65536> and wait strategy .
2019-02-05T00:09:59.805+03:00 INFO [ProcessBuffer] Initialized ProcessBuffer with ring size <65536> and wait strategy .
2019-02-05T00:10:00.045+03:00 INFO [ProcessBuffer] Initialized ProcessBuffer with ring size <65536> and wait strategy .
2019-02-05T00:10:00.217+03:00 INFO [ProcessBuffer] Initialized ProcessBuffer with ring size <65536> and wait strategy .
2019-02-05T00:10:00.377+03:00 INFO [ProcessBuffer] Initialized ProcessBuffer with ring size <65536> and wait strategy .
2019-02-05T00:10:00.735+03:00 INFO [ProcessBuffer] Initialized ProcessBuffer with ring size <65536> and wait strategy .
2019-02-05T00:10:00.748+03:00 INFO [connection] Opened connection [connectionId{localValue:3, serverValue:24}] to localhost:27017
2019-02-05T00:10:00.973+03:00 INFO [ProcessBuffer] Initialized ProcessBuffer with ring size <65536> and wait strategy .
2019-02-05T00:10:01.101+03:00 INFO [ProcessBuffer] Initialized ProcessBuffer with ring size <65536> and wait strategy .
2019-02-05T00:10:01.314+03:00 ERROR [CmdLineTool] Guice error (more detail on log level debug): Error injecting constructor, java.lang.RuntimeException: Unable to fetch module from resource: ByteArrayResource[bytes=[80, 75, 3, 4, 20, 0, 8, 8, 8, 0, …], encoding=null]
2019-02-05T00:10:01.315+03:00 ERROR [CmdLineTool] Guice error (more detail on log level debug): Error injecting constructor, java.lang.RuntimeException: Unable to fetch module from resource: ByteArrayResource[bytes=[80, 75, 3, 4, 20, 0, 8, 8, 8, 0, …], encoding=null]
2019-02-05T00:10:01.315+03:00 ERROR [CmdLineTool] Guice error (more detail on log level debug): Error injecting constructor, java.lang.RuntimeException: Unable to fetch module from resource: ByteArrayResource[bytes=[80, 75, 3, 4, 20, 0, 8, 8, 8, 0, …], encoding=null]
2019-02-05T00:10:01.315+03:00 ERROR [CmdLineTool] Guice error (more detail on log level debug): Error injecting constructor, java.lang.RuntimeException: Unable to fetch module from resource: ByteArrayResource[bytes=[80, 75, 3, 4, 20, 0, 8, 8, 8, 0, …], encoding=null]
2019-02-05T00:10:01.316+03:00 ERROR [CmdLineTool] Guice error (more detail on log level debug): Error injecting constructor, java.lang.RuntimeException: Unable to fetch module from resource: ByteArrayResource[bytes=[80, 75, 3, 4, 20, 0, 8, 8, 8, 0, …], encoding=null]
2019-02-05T00:10:01.316+03:00 ERROR [CmdLineTool] Guice error (more detail on log level debug): Error injecting constructor, java.lang.RuntimeException: Unable to fetch module from resource: ByteArrayResource[bytes=[80, 75, 3, 4, 20, 0, 8, 8, 8, 0, …], encoding=null]
2019-02-05T00:10:01.316+03:00 ERROR [CmdLineTool] Guice error (more detail on log level debug): Error injecting constructor, java.lang.RuntimeException: Unable to fetch module from resource: ByteArrayResource[bytes=[80, 75, 3, 4, 20, 0, 8, 8, 8, 0, …], encoding=null]
2019-02-05T00:10:01.316+03:00 ERROR [CmdLineTool] Guice error (more detail on log level debug): Error injecting constructor, java.lang.RuntimeException: Unable to fetch module from resource: ByteArrayResource[bytes=[80, 75, 3, 4, 20, 0, 8, 8, 8, 0, …], encoding=null]
2019-02-05T00:10:01.317+03:00 ERROR [CmdLineTool] Guice error (more detail on log level debug): Error injecting constructor, java.lang.RuntimeException: Unable to fetch module from resource: ByteArrayResource[bytes=[80, 75, 3, 4, 20, 0, 8, 8, 8, 0, …], encoding=null]
2019-02-05T00:10:01.317+03:00 ERROR [CmdLineTool] Guice error (more detail on log level debug): Error injecting constructor, java.lang.RuntimeException: Unable to fetch module from resource: ByteArrayResource[bytes=[80, 75, 3, 4, 20, 0, 8, 8, 8, 0, …], encoding=null]
2019-02-05T00:10:01.317+03:00 ERROR [CmdLineTool] Guice error (more detail on log level debug): Error injecting constructor, java.lang.RuntimeException: Unable to fetch module from resource: ByteArrayResource[bytes=[80, 75, 3, 4, 20, 0, 8, 8, 8, 0, …], encoding=null]
2019-02-05T00:10:01.317+03:00 ERROR [CmdLineTool] Guice error (more detail on log level debug): Error injecting constructor, java.lang.RuntimeException: Unable to fetch module from resource: ByteArrayResource[bytes=[80, 75, 3, 4, 20, 0, 8, 8, 8, 0, …], encoding=null]
2019-02-05T00:10:01.318+03:00 ERROR [CmdLineTool] Guice error (more detail on log level debug): Error injecting constructor, java.lang.RuntimeException: Unable to fetch module from resource: ByteArrayResource[bytes=[80, 75, 3, 4, 20, 0, 8, 8, 8, 0, …], encoding=null]
2019-02-05T00:10:01.318+03:00 ERROR [CmdLineTool] Guice error (more detail on log level debug): Error injecting constructor, java.lang.RuntimeException: Unable to fetch module from resource: ByteArrayResource[bytes=[80, 75, 3, 4, 20, 0, 8, 8, 8, 0, …], encoding=null]

jan · February 4, 2019, 6:37am

do not use a aboned version of Graylog - current stable Version is 2.5 and your 2.3 will not get any updates (not even security updates)
check the JAVA Version you have installed on the server (JAVA 8 is the only supported)

xked · February 4, 2019, 11:30am

@jan Thanks for your reply. Again i installed graylog 2.5 . But same error.

Information :

java version   "1.8.0_191"
elasticsearch versi  :  "version" : {
       "number" :            " 6.6.0"
       "lucene_version" : "7.6.0"

Source I’m using >>> http://docs.graylog.org/en/2.5/pages/installation/os/centos.html.

macko003 · February 4, 2019, 12:09pm

It can’t cause your errors, but it will another
check netstat -ntlp the elasticsearch’s IP.
In ES config you bind for private IP, but in the log GL try to work with loopback IP.
If you have one node only, maybe change the ES config back to LO IP.

Also after you fixed your problems as @jan wrote

xked · February 4, 2019, 12:29pm

@macko003

netstat -ntlp output:

Proto Recv-Q Send-Q Local Address           Foreign Address         State       PID/Program name    
tcp        0      0 127.0.0.1:27017         0.0.0.0:*               LISTEN      7511/mongod         
tcp        0      0 0.0.0.0:111             0.0.0.0:*               LISTEN      1/systemd           
tcp        0      0 192.168.122.1:53        0.0.0.0:*               LISTEN      7692/dnsmasq        
tcp        0      0 0.0.0.0:22              0.0.0.0:*               LISTEN      7121/sshd           
tcp        0      0 127.0.0.1:631           0.0.0.0:*               LISTEN      7119/cupsd          
tcp        0      0 127.0.0.1:25            0.0.0.0:*               LISTEN      7427/master         
tcp6       0      0 :::111                  :::*                    LISTEN      1/systemd           
tcp6       0      0 192.168.1.222:9200      :::*                    LISTEN      20523/java          
tcp6       0      0 192.168.1.222:9300      :::*                    LISTEN      20523/java          
tcp6       0      0 :::22                   :::*                    LISTEN      7121/sshd           
tcp6       0      0 ::1:631                 :::*                    LISTEN      7119/cupsd          
tcp6       0      0 ::1:25                  :::*                    LISTEN      7427/master

xked · February 4, 2019, 8:03pm

[root@localhost ~]# systemctl status elasticsearch.service`

> ● elasticsearch.service - Elasticsearch
>        Loaded: loaded (/usr/lib/systemd/system/elasticsearch.service; disabled; vendor preset: disabled)
>        Active: active (running) since Pzt 2019-02-04 22:57:20 +03; 3min 58s ago
>          Docs: http://www.elastic.co
>      Main PID: 22362 (java)
>         Tasks: 40
>        CGroup: /system.slice/elasticsearch.service
>                └─22362 /bin/java -Xms1g -Xmx1g -XX:+UseConcMarkSweepGC -XX:CMSInitiatingOccupancyFraction=75 -XX:+UseCMSInitiatingOccupancyOnly -Des.networkaddress.cache.ttl=60 -Des.networkad...
> 
>     Şub 04 22:57:26 localhost.localdomain elasticsearch[22362]: at org.elasticsearch.common.logging.LogConfigurator.configure(LogConfigurator.java:127)
>     Şub 04 22:57:26 localhost.localdomain elasticsearch[22362]: at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:302)
>     Şub 04 22:57:26 localhost.localdomain elasticsearch[22362]: at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:159)
>     Şub 04 22:57:26 localhost.localdomain elasticsearch[22362]: at org.elasticsearch.bootstrap.Elasticsearch.execute(Elasticsearch.java:150)
>     Şub 04 22:57:26 localhost.localdomain elasticsearch[22362]: at org.elasticsearch.cli.EnvironmentAwareCommand.execute(EnvironmentAwareCommand.java:86)
>     Şub 04 22:57:26 localhost.localdomain elasticsearch[22362]: at org.elasticsearch.cli.Command.mainWithoutErrorHandling(Command.java:124)
>     Şub 04 22:57:26 localhost.localdomain elasticsearch[22362]: at org.elasticsearch.cli.Command.main(Command.java:90)
>     Şub 04 22:57:26 localhost.localdomain elasticsearch[22362]: at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:116)
>     Şub 04 22:57:26 localhost.localdomain elasticsearch[22362]: at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:93)
>     Şub 04 22:57:26 localhost.localdomain elasticsearch[22362]: 2019-02-04 22:57:26,614 main ERROR Null object returned for Delete in DefaultRolloverStrategy.

[root@localhost ~]# systemctl status graylog-server.service

> ● graylog-server.service - Graylog server
>    Loaded: loaded (/usr/lib/systemd/system/graylog-server.service; enabled; vendor preset: disabled)
>    Active: active (running) since Pzt 2019-02-04 22:57:33 +03; 5min ago
>      Docs: http://docs.graylog.org/
>  Main PID: 22575 (graylog-server)
>     Tasks: 63
>    CGroup: /system.slice/graylog-server.service
>            ├─22575 /bin/sh /usr/share/graylog-server/bin/graylog-server
>            └─22576 /usr/bin/java -Xms1g -Xmx1g -XX:NewRatio=1 -server -XX:+ResizeTLAB -XX:+UseConcMarkSweepGC -XX:+CMSConcurrentMTEnabled -XX:+CMSClassUnloadingEnabled -XX:+UseParNewGC -X...
> 
> Şub 04 22:57:57 localhost.localdomain graylog-server[22575]: at org.graylog2.bootstrap.Main.main(Main.java:44)
> Şub 04 22:57:57 localhost.localdomain graylog-server[22575]: Caused by: java.lang.RuntimeException: XSD validation failed against the new schema (cvc-enumeration-valid: Value 'ıdentıty' …).
> Şub 04 22:57:57 localhost.localdomain graylog-server[22575]: at org.drools.compiler.kproject.models.KieModuleModelImpl$KieModuleValidator.validate(KieModuleModelImpl.java:309)
> Şub 04 22:57:57 localhost.localdomain graylog-server[22575]: at org.drools.compiler.kproject.models.KieModuleModelImpl$KieModuleValidator.validate(KieModuleModelImpl.java:274)
> Şub 04 22:57:57 localhost.localdomain graylog-server[22575]: at org.drools.compiler.kproject.models.KieModuleModelImpl$KieModuleValidator.access$100(KieModuleModelImpl.java:249)
> Şub 04 22:57:57 localhost.localdomain graylog-server[22575]: at org.drools.compiler.kproject.models.KieModuleModelImpl$kModuleMarshaller.fromXML(KieModuleModelImpl.java:196)
> Şub 04 22:57:57 localhost.localdomain graylog-server[22575]: at org.drools.compiler.kproject.models.KieModuleModelImpl.fromXML(KieModuleModelImpl.java:146)
> Şub 04 22:57:57 localhost.localdomain graylog-server[22575]: at org.drools.compiler.kie.builder.impl.KieRepositoryImpl.getKieModule(KieRepositoryImpl.java:243)
> Şub 04 22:57:57 localhost.localdomain graylog-server[22575]: ... 119 more
> Şub 04 22:57:57 localhost.localdomain graylog-server[22575]: Caused by: org.xml.sax.SAXParseException; lineNumber: 3; columnNumber: 194; cvc-enumeration-valid: Value 'ıdentıty'…enumeration.
> Hint: Some lines were ellipsized, use -l to show in full.

macko003 · February 4, 2019, 8:36pm

check your graylog’s XML config’s third lines…

eg.

xked · February 4, 2019, 8:47pm

Yes i checked but same. I didn’t change anything in there before.

Same problem as in there but not working for me.

macko003 · February 4, 2019, 9:00pm

any do you see any connection between you issue and the linked one?
did you try what @jan wrote?

xked · February 4, 2019, 9:06pm

Yes. I tried . Versions Graylog 2.5 Java 1.8.0_191.

I changed last log in /var/log/graylog-server/server.log in first post.

macko003 · February 5, 2019, 8:14am

I have no more idea.
You can try to remove the package, and reinstall it from an official repo.

xked · February 5, 2019, 2:41pm

Problem is foreign language. I installed Centos ENG and it worked.

Thanks for support.

system · February 19, 2019, 2:42pm

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.

Topic		Replies	Views
Unable to connect to Graylog Web Interface Graylog Central (peer support)	3	11297	November 16, 2018
Graylog Web Interface Issue Graylog Central (peer support)	8	212	April 11, 2024
New setup for graylog the web interface is not working Graylog Central (peer support)	2	3220	June 21, 2017
Can't connect to graylog service Graylog Central (peer support)	7	3492	November 8, 2018
Graylog and Elasticsearch work but I can't reach web interface Graylog Central (peer support)	4	3873	September 27, 2017

Graylog web interface problem!

Related topics