Graylog input showing no incoming messages


#1

elastic version 5.5.2
graylog version 2.3.1
mongodb version 2.6.10

the problem
i’m trying to get messages from a php application on a remote server into graylog using gelf over udp. tcpdump shows traffic coming in when i send below test message. however the inputs in graylog show nothing incoming. i redacted some of the ip addresses.

what occurs to me already is the strange date on the logs of elastic. elastic does not seem to log anything new when restarting the server. however graylog reports that the elasticsearch cluster is green. (edit: the date problem has been fixed, the cluster log messages somehow weren’t writeable by the user the process runs under. i updated the logs with the correct version)

i was hoping if anyone could help me out troubleshoot this issue. thank you for reading.

send message command
echo -n '{ "version": "1.1", "host": "example.org", "short_message": "A short message", "level": 5, "_some_info": "foo" }' | nc -w0 -u bbb.bbb.bbb.bbb 12201

tcpdump output

sudo tcpdump -i ens3 udp port 12201
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on ens3, link-type EN10MB (Ethernet), capture size 262144 bytes
12:48:30.909255 IP aaa.aaa.aaa.aaa > bbb.bbb.bbb.bbb.12201: UDP, length 112

graylog input configuration
bind_address: bbb.bbb.bbb.bbb
decompress_size_limit: 8388608
override_source:
port: 12201
recv_buffer_size: 262144

elasticsearch graylog cluster log

[2017-09-05 13:34:26,084][INFO ][node                     ] [Vavavoom] version[2.4.4], pid[4517], build[fcbb46d/2017-01-03T11:33:16Z]
[2017-09-05 13:34:26,085][INFO ][node                     ] [Vavavoom] initializing ...
[2017-09-05 13:34:26,465][INFO ][plugins                  ] [Vavavoom] modules [reindex, lang-expression, lang-groovy], plugins [], sites []
[2017-09-05 13:34:26,481][INFO ][env                      ] [Vavavoom] using [1] data paths, mounts [[/ (/dev/vda1)]], net usable_space [107.3gb], net total_space [143.5gb], spins? [possibly], types [ext4]
[2017-09-05 13:34:26,481][INFO ][env                      ] [Vavavoom] heap size [495.3mb], compressed ordinary object pointers [true]
[2017-09-05 13:34:28,181][INFO ][node                     ] [Vavavoom] initialized
[2017-09-05 13:34:28,181][INFO ][node                     ] [Vavavoom] starting ...
[2017-09-05 13:34:28,236][INFO ][transport                ] [Vavavoom] publish_address {127.0.0.1:9300}, bound_addresses {[::1]:9300}, {127.0.0.1:9300}
[2017-09-05 13:34:28,240][INFO ][discovery                ] [Vavavoom] graylog/UuXnej4bQFmxq0yFml8qSQ
[2017-09-05 13:34:31,276][INFO ][cluster.service          ] [Vavavoom] new_master {Vavavoom}{UuXnej4bQFmxq0yFml8qSQ}{127.0.0.1}{127.0.0.1:9300}, added {{graylog-dbcea0b8-d5cb-46a3-bdeb-b689bfbcd75c}{Evx8OsmkSqamRwLVDeTxMg}{127.0.0.1}{127.0.0.1:9350}{client=true, data=false, master=false},}, reason: zen-disco-join(elected_as_master, [0] joins received)
[2017-09-05 13:34:31,324][INFO ][http                     ] [Vavavoom] publish_address {127.0.0.1:9200}, bound_addresses {[::1]:9200}, {127.0.0.1:9200}
[2017-09-05 13:34:31,324][INFO ][node                     ] [Vavavoom] started
[2017-09-05 13:34:31,398][INFO ][gateway                  ] [Vavavoom] recovered [1] indices into cluster_state
[2017-09-05 13:34:31,824][INFO ][cluster.routing.allocation] [Vavavoom] Cluster health status changed from [RED] to [GREEN] (reason: [shards started [[graylog_0][0], [graylog_0][1], [graylog_0][1], [graylog_0][0]] ...]).

graylog logs

2017-09-05T12:55:25.880+02:00 INFO  [CmdLineTool] Loaded plugin: Elastic Beats Input 2.2.3 [org.graylog.plugins.beats.BeatsInputPlugin]
2017-09-05T12:55:25.882+02:00 INFO  [CmdLineTool] Loaded plugin: Collector 2.2.3 [org.graylog.plugins.collector.CollectorPlugin]
2017-09-05T12:55:25.882+02:00 INFO  [CmdLineTool] Loaded plugin: Enterprise Integration Plugin 2.2.3 [org.graylog.plugins.enterprise_integration.EnterpriseIntegrationPlugin]
2017-09-05T12:55:25.883+02:00 INFO  [CmdLineTool] Loaded plugin: MapWidgetPlugin 2.2.3 [org.graylog.plugins.map.MapWidgetPlugin]
2017-09-05T12:55:25.891+02:00 INFO  [CmdLineTool] Loaded plugin: Pipeline Processor Plugin 2.2.3 [org.graylog.plugins.pipelineprocessor.ProcessorPlugin]
2017-09-05T12:55:25.891+02:00 INFO  [CmdLineTool] Loaded plugin: Anonymous Usage Statistics 2.2.3 [org.graylog.plugins.usagestatistics.UsageStatsPlugin]
2017-09-05T12:55:26.192+02:00 INFO  [CmdLineTool] Running with JVM arguments: -Xms256m -Xmx256m -XX:NewRatio=1 -XX:+ResizeTLAB -XX:+UseConcMarkSweepGC -XX:+CMSConcurrentMTEnabled -XX:+CMSClassUnloadingEnabled -XX:+UseParNewGC -XX:-OmitStackTraceInFastThrow -Djava.net.preferIPv4Stack=true -Dlog4j.configurationFile=file:///etc/graylog/server/log4j2.xml -Djava.library.path=/usr/share/graylog-server/lib/sigar -Dgraylog2.installation_source=deb
2017-09-05T12:55:26.425+02:00 INFO  [Version] HV000001: Hibernate Validator null
2017-09-05T12:55:28.697+02:00 INFO  [InputBufferImpl] Message journal is enabled.
2017-09-05T12:55:28.717+02:00 INFO  [NodeId] Node ID: dbcea0b8-d5cb-46a3-bdeb-b689bfbcd75c
2017-09-05T12:55:28.896+02:00 INFO  [LogManager] Loading logs.
2017-09-05T12:55:28.938+02:00 INFO  [LogManager] Logs loading complete.
2017-09-05T12:55:28.938+02:00 INFO  [KafkaJournal] Initialized Kafka based journal at /var/lib/graylog-server/journal
2017-09-05T12:55:28.961+02:00 INFO  [InputBufferImpl] Initialized InputBufferImpl with ring size <65536> and wait strategy <BlockingWaitStrategy>, running 2 parallel message handlers.
2017-09-05T12:55:28.977+02:00 INFO  [cluster] Cluster created with settings {hosts=[localhost:27017], mode=SINGLE, requiredClusterType=UNKNOWN, serverSelectionTimeout='30000 ms', maxWaitQueueSize=5000}
2017-09-05T12:55:29.022+02:00 INFO  [cluster] No server chosen by ReadPreferenceServerSelector{readPreference=primary} from cluster description ClusterDescription{type=UNKNOWN, connectionMode=SINGLE, serverDescriptions=[ServerDescription{address=localhost:27017, type=UNKNOWN, state=CONNECTING}]}. Waiting for 30000 ms before timing out
2017-09-05T12:55:29.041+02:00 INFO  [connection] Opened connection [connectionId{localValue:1, serverValue:11}] to localhost:27017
2017-09-05T12:55:29.043+02:00 INFO  [cluster] Monitor thread successfully connected to server with description ServerDescription{address=localhost:27017, type=STANDALONE, state=CONNECTED, ok=true, version=ServerVersion{versionList=[2, 6, 10]}, minWireVersion=0, maxWireVersion=2, maxDocumentSize=16777216, roundTripTimeNanos=537531}
2017-09-05T12:55:29.048+02:00 INFO  [connection] Opened connection [connectionId{localValue:2, serverValue:12}] to localhost:27017
2017-09-05T12:55:29.347+02:00 INFO  [node] [graylog-dbcea0b8-d5cb-46a3-bdeb-b689bfbcd75c] version[2.4.4], pid[4208], build[fcbb46d/2017-01-03T11:33:16Z]
2017-09-05T12:55:29.347+02:00 INFO  [node] [graylog-dbcea0b8-d5cb-46a3-bdeb-b689bfbcd75c] initializing ...
2017-09-05T12:55:29.352+02:00 INFO  [plugins] [graylog-dbcea0b8-d5cb-46a3-bdeb-b689bfbcd75c] modules [], plugins [graylog-monitor], sites []
2017-09-05T12:55:30.753+02:00 INFO  [node] [graylog-dbcea0b8-d5cb-46a3-bdeb-b689bfbcd75c] initialized
2017-09-05T12:55:30.849+02:00 INFO  [ProcessBuffer] Initialized ProcessBuffer with ring size <65536> and wait strategy <BlockingWaitStrategy>.
2017-09-05T12:55:32.243+02:00 INFO  [RulesEngineProvider] No static rules file loaded.
2017-09-05T12:55:32.509+02:00 WARN  [GeoIpResolverEngine] GeoIP database file does not exist: /etc/graylog/server/GeoLite2-City.mmdb
2017-09-05T12:55:32.513+02:00 INFO  [OutputBuffer] Initialized OutputBuffer with ring size <65536> and wait strategy <BlockingWaitStrategy>.
2017-09-05T12:55:32.567+02:00 WARN  [GeoIpResolverEngine] GeoIP database file does not exist: /etc/graylog/server/GeoLite2-City.mmdb
2017-09-05T12:55:32.637+02:00 WARN  [GeoIpResolverEngine] GeoIP database file does not exist: /etc/graylog/server/GeoLite2-City.mmdb
2017-09-05T12:55:32.700+02:00 WARN  [GeoIpResolverEngine] GeoIP database file does not exist: /etc/graylog/server/GeoLite2-City.mmdb
2017-09-05T12:55:32.762+02:00 WARN  [GeoIpResolverEngine] GeoIP database file does not exist: /etc/graylog/server/GeoLite2-City.mmdb
2017-09-05T12:55:33.311+02:00 INFO  [ServerBootstrap] Graylog server 2.2.3+7adc951 starting up
2017-09-05T12:55:33.311+02:00 INFO  [ServerBootstrap] JRE: Oracle Corporation 1.8.0_131 on Linux 4.4.0-78-generic
2017-09-05T12:55:33.311+02:00 INFO  [ServerBootstrap] Deployment: deb
2017-09-05T12:55:33.311+02:00 INFO  [ServerBootstrap] OS: Ubuntu 16.04.2 LTS (xenial)
2017-09-05T12:55:33.312+02:00 INFO  [ServerBootstrap] Arch: amd64
2017-09-05T12:55:33.324+02:00 WARN  [DeadEventLoggingListener] Received unhandled event of type <org.graylog2.plugin.lifecycles.Lifecycle> from event bus <AsyncEventBus{graylog-eventbus}>
2017-09-05T12:55:33.343+02:00 INFO  [PeriodicalsService] Starting 26 periodicals ...
2017-09-05T12:55:33.346+02:00 INFO  [Periodicals] Starting [org.graylog2.periodical.ThroughputCalculator] periodical in [0s], polling every [1s].
2017-09-05T12:55:33.345+02:00 INFO  [node] [graylog-dbcea0b8-d5cb-46a3-bdeb-b689bfbcd75c] starting ...
2017-09-05T12:55:33.379+02:00 INFO  [Periodicals] Starting [org.graylog2.periodical.AlertScannerThread] periodical in [10s], polling every [60s].
2017-09-05T12:55:33.381+02:00 INFO  [Periodicals] Starting [org.graylog2.periodical.BatchedElasticSearchOutputFlushThread] periodical in [0s], polling every [1s].
2017-09-05T12:55:33.381+02:00 INFO  [Periodicals] Starting [org.graylog2.periodical.ClusterHealthCheckThread] periodical in [120s], polling every [20s].
2017-09-05T12:55:33.392+02:00 INFO  [Periodicals] Starting [org.graylog2.periodical.ContentPackLoaderPeriodical] periodical, running forever.
2017-09-05T12:55:33.400+02:00 INFO  [Periodicals] Starting [org.graylog2.periodical.GarbageCollectionWarningThread] periodical, running forever.
2017-09-05T12:55:33.401+02:00 INFO  [Periodicals] Starting [org.graylog2.periodical.IndexerClusterCheckerThread] periodical in [0s], polling every [30s].
2017-09-05T12:55:33.402+02:00 INFO  [Periodicals] Starting [org.graylog2.periodical.IndexRetentionThread] periodical in [0s], polling every [300s].
2017-09-05T12:55:33.402+02:00 INFO  [Periodicals] Starting [org.graylog2.periodical.IndexRotationThread] periodical in [0s], polling every [10s].
2017-09-05T12:55:33.404+02:00 INFO  [IndexRetentionThread] Elasticsearch cluster not available, skipping index retention checks.
2017-09-05T12:55:33.408+02:00 INFO  [Periodicals] Starting [org.graylog2.periodical.NodePingThread] periodical in [0s], polling every [1s].
2017-09-05T12:55:33.417+02:00 INFO  [connection] Opened connection [connectionId{localValue:3, serverValue:13}] to localhost:27017
2017-09-05T12:55:33.422+02:00 INFO  [Periodicals] Starting [org.graylog2.periodical.VersionCheckThread] periodical in [300s], polling every [1800s].
2017-09-05T12:55:33.423+02:00 INFO  [Periodicals] Starting [org.graylog2.periodical.ThrottleStateUpdaterThread] periodical in [1s], polling every [1s].
2017-09-05T12:55:33.423+02:00 INFO  [Periodicals] Starting [org.graylog2.events.ClusterEventPeriodical] periodical in [0s], polling every [1s].
2017-09-05T12:55:33.425+02:00 INFO  [Periodicals] Starting [org.graylog2.events.ClusterEventCleanupPeriodical] periodical in [0s], polling every [86400s].
2017-09-05T12:55:33.425+02:00 INFO  [Periodicals] Starting [org.graylog2.periodical.ClusterIdGeneratorPeriodical] periodical, running forever.
2017-09-05T12:55:33.426+02:00 INFO  [Periodicals] Starting [org.graylog2.periodical.IndexRangesMigrationPeriodical] periodical, running forever.
2017-09-05T12:55:33.427+02:00 INFO  [Periodicals] Starting [org.graylog2.periodical.IndexRangesCleanupPeriodical] periodical in [15s], polling every [3600s].
2017-09-05T12:55:33.433+02:00 INFO  [connection] Opened connection [connectionId{localValue:5, serverValue:15}] to localhost:27017
2017-09-05T12:55:33.437+02:00 INFO  [connection] Opened connection [connectionId{localValue:4, serverValue:14}] to localhost:27017
2017-09-05T12:55:33.440+02:00 INFO  [connection] Opened connection [connectionId{localValue:7, serverValue:17}] to localhost:27017
2017-09-05T12:55:33.444+02:00 INFO  [connection] Opened connection [connectionId{localValue:6, serverValue:16}] to localhost:27017
2017-09-05T12:55:33.444+02:00 INFO  [connection] Opened connection [connectionId{localValue:8, serverValue:18}] to localhost:27017
2017-09-05T12:55:33.463+02:00 INFO  [PeriodicalsService] Not starting [org.graylog2.periodical.UserPermissionMigrationPeriodical] periodical. Not configured to run on this node.
2017-09-05T12:55:33.464+02:00 INFO  [Periodicals] Starting [org.graylog2.periodical.AlarmCallbacksMigrationPeriodical] periodical, running forever.
2017-09-05T12:55:33.466+02:00 INFO  [Periodicals] Starting [org.graylog2.periodical.ConfigurationManagementPeriodical] periodical, running forever.
2017-09-05T12:55:33.473+02:00 INFO  [Periodicals] Starting [org.graylog2.periodical.LdapGroupMappingMigration] periodical, running forever.
2017-09-05T12:55:33.477+02:00 INFO  [Periodicals] Starting [org.graylog2.periodical.IndexFailuresPeriodical] periodical, running forever.
2017-09-05T12:55:33.483+02:00 INFO  [Periodicals] Starting [org.graylog.plugins.usagestatistics.UsageStatsNodePeriodical] periodical in [300s], polling every [21600s].
2017-09-05T12:55:33.485+02:00 INFO  [Periodicals] Starting [org.graylog.plugins.usagestatistics.UsageStatsClusterPeriodical] periodical in [300s], polling every [21600s].
2017-09-05T12:55:33.504+02:00 INFO  [Periodicals] Starting [org.graylog.plugins.pipelineprocessor.periodical.LegacyDefaultStreamMigration] periodical, running forever.
2017-09-05T12:55:33.505+02:00 INFO  [Periodicals] Starting [org.graylog.plugins.collector.periodical.PurgeExpiredCollectorsThread] periodical in [0s], polling every [3600s].
2017-09-05T12:55:33.517+02:00 INFO  [IndexerClusterCheckerThread] Indexer not fully initialized yet. Skipping periodic cluster check.
2017-09-05T12:55:33.538+02:00 INFO  [LegacyDefaultStreamMigration] Legacy default stream has no connections, no migration needed.
2017-09-05T12:55:33.624+02:00 INFO  [V20161130141500_DefaultStreamRecalcIndexRanges] Cluster not connected yet, delaying migration until it is reachable.
2017-09-05T12:55:33.647+02:00 INFO  [transport] [graylog-dbcea0b8-d5cb-46a3-bdeb-b689bfbcd75c] publish_address {127.0.0.1:9350}, bound_addresses {127.0.0.1:9350}
2017-09-05T12:55:33.652+02:00 INFO  [discovery] [graylog-dbcea0b8-d5cb-46a3-bdeb-b689bfbcd75c] graylog/Evx8OsmkSqamRwLVDeTxMg
2017-09-05T12:55:33.817+02:00 INFO  [JerseyService] Enabling CORS for HTTP endpoint
2017-09-05T12:55:36.654+02:00 WARN  [discovery] [graylog-dbcea0b8-d5cb-46a3-bdeb-b689bfbcd75c] waited for 3s and no initial state was set by the discovery
2017-09-05T12:55:36.654+02:00 INFO  [node] [graylog-dbcea0b8-d5cb-46a3-bdeb-b689bfbcd75c] started
2017-09-05T12:55:36.738+02:00 INFO  [service] [graylog-dbcea0b8-d5cb-46a3-bdeb-b689bfbcd75c] detected_master {Galaxy Master}{CVOsZg3GR0qaHVvQ611W0Q}{127.0.0.1}{127.0.0.1:9300}, added {{Galaxy Master}{CVOsZg3GR0qaHVvQ611W0Q}{127.0.0.1}{127.0.0.1:9300},}, reason: zen-disco-receive(from master [{Galaxy Master}{CVOsZg3GR0qaHVvQ611W0Q}{127.0.0.1}{127.0.0.1:9300}])
2017-09-05T12:55:44.367+02:00 INFO  [NetworkListener] Started listener bound to [xxx.xxx.xxx.xxx:9000]
2017-09-05T12:55:44.368+02:00 INFO  [HttpServer] [HttpServer] Started.
2017-09-05T12:55:44.368+02:00 INFO  [JerseyService] Started REST API at <http://xxx.xxx.xxx.xxx:9000/api/>
2017-09-05T12:55:44.368+02:00 INFO  [JerseyService] Started Web Interface at <http://xxx.xxx.xxx.xxx:9000/>
2017-09-05T12:55:44.370+02:00 INFO  [ServerBootstrap] Services started, startup times in ms: {OutputSetupService [RUNNING]=42, BufferSynchronizerService [RUNNING]=71, KafkaJournal [RUNNING]=71, InputSetupService [RUNNING]=123, JournalReader [RUNNING]=150, StreamCacheService [RUNNING]=167, PeriodicalsService [RUNNING]=194, ConfigurationEtagService [RUNNING]=195, IndexerSetupService [RUNNING]=3415, JerseyService [RUNNING]=11031}
2017-09-05T12:55:44.369+02:00 INFO  [ServiceManagerListener] Services are healthy
2017-09-05T12:55:44.375+02:00 INFO  [ServerBootstrap] Graylog server up and running.
2017-09-05T12:55:44.376+02:00 INFO  [InputSetupService] Triggering launching persisted inputs, node transitioned from Uninitialized [LB:DEAD] to Running [LB:ALIVE]
2017-09-05T12:55:44.402+02:00 INFO  [InputStateListener] Input [GELF UDP/59ae7741a956ef635cea7eb8] is now STARTING
2017-09-05T12:55:44.404+02:00 INFO  [InputStateListener] Input [Raw/Plaintext UDP/59ae7c54a956ef635cea841e] is now STARTING
2017-09-05T12:55:44.437+02:00 WARN  [NettyTransport] receiveBufferSize (SO_RCVBUF) for input RawUDPInput{title=test, type=org.graylog2.inputs.raw.udp.RawUDPInput, nodeId=null} should be 262144 but is 212992.
2017-09-05T12:55:44.437+02:00 WARN  [NettyTransport] receiveBufferSize (SO_RCVBUF) for input GELFUDPInput{title=appliance-gelf-udp, type=org.graylog2.inputs.gelf.udp.GELFUDPInput, nodeId=null} should be 262144 but is 212992.
2017-09-05T12:55:44.439+02:00 INFO  [InputStateListener] Input [Raw/Plaintext UDP/59ae7c54a956ef635cea841e] is now RUNNING
2017-09-05T12:55:44.440+02:00 INFO  [InputStateListener] Input [GELF UDP/59ae7741a956ef635cea7eb8] is now RUNNING
2017-09-05T12:56:30.108+02:00 WARN  [ProxiedResource] Unable to call http://xxx.xxx.xxx.xxx:9000/api/system/jobs on node <dbcea0b8-d5cb-46a3-bdeb-b689bfbcd75c>
java.net.SocketTimeoutException: Read timed out
	at java.net.SocketInputStream.socketRead0(Native Method) ~[?:1.8.0_131]
	at java.net.SocketInputStream.socketRead(SocketInputStream.java:116) ~[?:1.8.0_131]
	at java.net.SocketInputStream.read(SocketInputStream.java:171) ~[?:1.8.0_131]
	at java.net.SocketInputStream.read(SocketInputStream.java:141) ~[?:1.8.0_131]
	at okio.Okio$2.read(Okio.java:138) ~[graylog.jar:?]
	at okio.AsyncTimeout$2.read(AsyncTimeout.java:236) ~[graylog.jar:?]
	at okio.RealBufferedSource.indexOf(RealBufferedSource.java:325) ~[graylog.jar:?]
	at okio.RealBufferedSource.indexOf(RealBufferedSource.java:314) ~[graylog.jar:?]
	at okio.RealBufferedSource.readUtf8LineStrict(RealBufferedSource.java:210) ~[graylog.jar:?]
	at okhttp3.internal.http1.Http1Codec.readResponse(Http1Codec.java:191) ~[graylog.jar:?]
	at okhttp3.internal.http1.Http1Codec.readResponseHeaders(Http1Codec.java:132) ~[graylog.jar:?]
	at okhttp3.internal.http.CallServerInterceptor.intercept(CallServerInterceptor.java:54) ~[graylog.jar:?]
	at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:92) ~[graylog.jar:?]
	at okhttp3.internal.connection.ConnectInterceptor.intercept(ConnectInterceptor.java:45) ~[graylog.jar:?]
	at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:92) ~[graylog.jar:?]
	at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:67) ~[graylog.jar:?]
	at okhttp3.internal.cache.CacheInterceptor.intercept(CacheInterceptor.java:93) ~[graylog.jar:?]
	at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:92) ~[graylog.jar:?]
	at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:67) ~[graylog.jar:?]
	at okhttp3.internal.http.BridgeInterceptor.intercept(BridgeInterceptor.java:93) ~[graylog.jar:?]
	at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:92) ~[graylog.jar:?]
	at okhttp3.internal.http.RetryAndFollowUpInterceptor.intercept(RetryAndFollowUpInterceptor.java:120) ~[graylog.jar:?]
	at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:92) ~[graylog.jar:?]
	at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:67) ~[graylog.jar:?]
	at org.graylog2.rest.RemoteInterfaceProvider.lambda$get$0(RemoteInterfaceProvider.java:59) ~[graylog.jar:?]
	at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:92) ~[graylog.jar:?]
	at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:67) ~[graylog.jar:?]
	at okhttp3.RealCall.getResponseWithInterceptorChain(RealCall.java:179) ~[graylog.jar:?]
	at okhttp3.RealCall.execute(RealCall.java:63) ~[graylog.jar:?]
	at retrofit2.OkHttpCall.execute(OkHttpCall.java:174) ~[graylog.jar:?]
	at org.graylog2.shared.rest.resources.ProxiedResource.lambda$null$0(ProxiedResource.java:76) ~[graylog.jar:?]
	at java.util.concurrent.FutureTask.run(FutureTask.java:266) [?:1.8.0_131]
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:1.8.0_131]
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0_131]
	at java.lang.Thread.run(Thread.java:748) [?:1.8.0_131]
2017-09-05T13:34:25.465+02:00 INFO  [zen] [graylog-dbcea0b8-d5cb-46a3-bdeb-b689bfbcd75c] master_left [{Galaxy Master}{CVOsZg3GR0qaHVvQ611W0Q}{127.0.0.1}{127.0.0.1:9300}], reason [transport disconnected]
2017-09-05T13:34:25.465+02:00 WARN  [zen] [graylog-dbcea0b8-d5cb-46a3-bdeb-b689bfbcd75c] master left (reason = transport disconnected), current nodes: {{graylog-dbcea0b8-d5cb-46a3-bdeb-b689bfbcd75c}{Evx8OsmkSqamRwLVDeTxMg}{127.0.0.1}{127.0.0.1:9350}{client=true, data=false, master=false},}
2017-09-05T13:34:25.468+02:00 INFO  [service] [graylog-dbcea0b8-d5cb-46a3-bdeb-b689bfbcd75c] removed {{Galaxy Master}{CVOsZg3GR0qaHVvQ611W0Q}{127.0.0.1}{127.0.0.1:9300},}, reason: zen-disco-master_failed ({Galaxy Master}{CVOsZg3GR0qaHVvQ611W0Q}{127.0.0.1}{127.0.0.1:9300})
2017-09-05T13:34:25.469+02:00 WARN  [ClusterStateMonitor] No Elasticsearch data nodes in cluster, cluster is completely offline.
2017-09-05T13:34:31.282+02:00 INFO  [service] [graylog-dbcea0b8-d5cb-46a3-bdeb-b689bfbcd75c] detected_master {Vavavoom}{UuXnej4bQFmxq0yFml8qSQ}{127.0.0.1}{127.0.0.1:9300}, added {{Vavavoom}{UuXnej4bQFmxq0yFml8qSQ}{127.0.0.1}{127.0.0.1:9300},}, reason: zen-disco-receive(from master [{Vavavoom}{UuXnej4bQFmxq0yFml8qSQ}{127.0.0.1}{127.0.0.1:9300}])

(Jochen) #2

This looks like you’ve misconfigured the rest_listen_uri or rest_transport_uri settings in the Graylog configuration.

Make sure that the Graylog node is able to communicate with the URI given in these settings (and in the quoted error message).


#3

Thanks for the reply! I had set up the rest_listen_uri to the public IP of the machine running graylog. I have changed it now to http://0.0.0.0:9000/api but now I get the following error:

Error message
Bad request
Original Request
GET http://10.0.0.100:9000/api/system/sessions
Status code
undefined
Full error message
Error: Request has been terminated
Possible causes: the network is offline, Origin is not allowed by Access-Control-Allow-Origin, the page is being unloaded, etc.

The machine has 2 IP addresses, a private and public network one. I guess setting it to 0.0.0.0 defaults it to the private network adapter rather than the public one.


(Jochen) #4

You can configure the URI of the Graylog REST API which the web interface should connect to by using the web_endpoint_uri configuration setting.


#5

Does that exception occur when the webinterface can’t reach the REST API? Because as far as I can tell in the developer tools of my browser all REST calls are working fine. Every now and again a time out occurs but it does not seem consistent.


(system) #6

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.