Hello, i’m running Graylog 2.3.1 on Ubuntu 16.04 with ElasticSearch 5. When Graylog is getting started there are nothing in the logs about Elasticsearch and i can get messages but whenever i try to see received messages “Loading” takes forever. No errors or warnings in both Graylog and Elasticsearch logs. In the Graylog ui, cluster health indicator is Green. I don’t know what to do. Tried restarts several time to no avail. What could be the problem?
Thanks in advance.
What’s the configuration of Graylog and Elasticsearch on your system?
What’s in the logs of your Elasticsearch and Graylog nodes?
http://docs.graylog.org/en/2.3/pages/configuration/file_location.html
[2017-08-10T13:23:49,208][INFO ][o.e.n.Node ] [node-0] initializing ...
[2017-08-10T13:23:49,311][INFO ][o.e.e.NodeEnvironment ] [node-0] using [1] data paths, mounts [[/ (/dev/md2)]], net usable_space [1.3tb], net total_space [1.7tb], spins? [possibly], types [ext4]
[2017-08-10T13:23:49,311][INFO ][o.e.e.NodeEnvironment ] [node-0] heap size [1.9gb], compressed ordinary object pointers [true]
[2017-08-10T13:23:49,324][INFO ][o.e.n.Node ] [node-0] node name [node-0], node ID [alS-vJsBQbCcpeN5pDBIUQ]
[2017-08-10T13:23:49,324][INFO ][o.e.n.Node ] [node-0] version[5.5.1], pid[23835], build[19c13d0/2017-07-18T20:44:24.823Z], OS[Linux/4.4.0-31-generic/amd64], JVM[Oracle Corporation/OpenJDK 64-Bit Server VM/1.8.0_131/25.131-b11]
[2017-08-10T13:23:49,324][INFO ][o.e.n.Node ] [node-0] JVM arguments [-Xms2g, -Xmx2g, -XX:+UseConcMarkSweepGC, -XX:CMSInitiatingOccupancyFraction=75, -XX:+UseCMSInitiatingOccupancyOnly, -XX:+AlwaysPreTouch, -Xss1m, -Djava.awt.headless=true, -Dfile.encoding=UTF-8, -Djna.nosys=true, -Djdk.io.permissionsUseCanonicalPath=true, -Dio.netty.noUnsafe=true, -Dio.netty.noKeySetOptimization=true, -Dio.netty.recycler.maxCapacityPerThread=0, -Dlog4j.shutdownHookEnabled=false, -Dlog4j2.disable.jmx=true, -Dlog4j.skipJansi=true, -XX:+HeapDumpOnOutOfMemoryError, -Des.path.home=/usr/share/elasticsearch]
[2017-08-10T13:23:49,748][INFO ][o.e.p.PluginsService ] [node-0] loaded module [aggs-matrix-stats]
[2017-08-10T13:23:49,748][INFO ][o.e.p.PluginsService ] [node-0] loaded module [ingest-common]
[2017-08-10T13:23:49,748][INFO ][o.e.p.PluginsService ] [node-0] loaded module [lang-expression]
[2017-08-10T13:23:49,748][INFO ][o.e.p.PluginsService ] [node-0] loaded module [lang-groovy]
[2017-08-10T13:23:49,748][INFO ][o.e.p.PluginsService ] [node-0] loaded module [lang-mustache]
[2017-08-10T13:23:49,748][INFO ][o.e.p.PluginsService ] [node-0] loaded module [lang-painless]
[2017-08-10T13:23:49,748][INFO ][o.e.p.PluginsService ] [node-0] loaded module [parent-join]
[2017-08-10T13:23:49,749][INFO ][o.e.p.PluginsService ] [node-0] loaded module [percolator]
[2017-08-10T13:23:49,749][INFO ][o.e.p.PluginsService ] [node-0] loaded module [reindex]
[2017-08-10T13:23:49,749][INFO ][o.e.p.PluginsService ] [node-0] loaded module [transport-netty3]
[2017-08-10T13:23:49,749][INFO ][o.e.p.PluginsService ] [node-0] loaded module [transport-netty4]
[2017-08-10T13:23:49,749][INFO ][o.e.p.PluginsService ] [node-0] no plugins loaded
[2017-08-10T13:23:50,579][INFO ][o.e.d.DiscoveryModule ] [node-0] using discovery type [zen]
[2017-08-10T13:23:51,044][INFO ][o.e.n.Node ] [node-0] initialized
[2017-08-10T13:23:51,044][INFO ][o.e.n.Node ] [node-0] starting ...
[2017-08-10T13:23:51,122][INFO ][o.e.t.TransportService ] [node-0] publish_address {xxx:9300}, bound_addresses {[::]:9300}
[2017-08-10T13:23:51,128][INFO ][o.e.b.BootstrapChecks ] [node-0] bound or publishing to a non-loopback or non-link-local address, enforcing bootstrap checks
[2017-08-10T13:23:54,154][INFO ][o.e.c.s.ClusterService ] [node-0] new_master {node-0}{alS-vJsBQbCcpeN5pDBIUQ}{y9OTMlLJTmSSkQ9EggZrXA}{XXX}{XXX:9300}, reason: zen-disco-elected-as-master ([0] nodes joined)
[2017-08-10T13:23:54,180][INFO ][o.e.h.n.Netty4HttpServerTransport] [node-0] publish_address {XXX:9200}, bound_addresses {[::]:9200}
[2017-08-10T13:23:54,180][INFO ][o.e.n.Node ] [node-0] started
[2017-08-10T13:23:54,717][INFO ][o.e.g.GatewayService ] [node-0] recovered [5] indices into cluster_state
[2017-08-10T13:23:57,716][INFO ][o.e.c.r.a.AllocationService] [node-0] Cluster health status changed from [RED] to [YELLOW] (reason: [shards started [[.kibana][0]] ...]).
2017-08-10 08:38:13,503 INFO : org.graylog2.bootstrap.CmdLineTool - Loaded plugin: Elastic Beats Input 2.3.0 [org.graylog.plugins.beats.BeatsInputPlugin]
2017-08-10 08:38:13,505 INFO : org.graylog2.bootstrap.CmdLineTool - Loaded plugin: Collector 2.3.0 [org.graylog.plugins.collector.CollectorPlugin]
2017-08-10 08:38:13,505 INFO : org.graylog2.bootstrap.CmdLineTool - Loaded plugin: Enterprise Integration Plugin 2.3.0 [org.graylog.plugins.enterprise_integration.EnterpriseIntegrationPlugin]
2017-08-10 08:38:13,505 INFO : org.graylog2.bootstrap.CmdLineTool - Loaded plugin: MapWidgetPlugin 2.3.0 [org.graylog.plugins.map.MapWidgetPlugin]
2017-08-10 08:38:13,511 INFO : org.graylog2.bootstrap.CmdLineTool - Loaded plugin: Pipeline Processor Plugin 2.3.0 [org.graylog.plugins.pipelineprocessor.ProcessorPlugin]
2017-08-10 08:38:13,511 INFO : org.graylog2.bootstrap.CmdLineTool - Loaded plugin: Anonymous Usage Statistics 2.3.0 [org.graylog.plugins.usagestatistics.UsageStatsPlugin]
2017-08-10 08:38:13,632 INFO : org.graylog2.bootstrap.CmdLineTool - Running with JVM arguments: -Xms1g -Xmx2g -XX:NewRatio=1 -XX:MaxMetaspaceSize=256m -XX:+ResizeTLAB -XX:+UseConcMarkSweepGC -XX:+CMSConcurrentMTEnabled -XX:+CMSClassUnloadingEnabled -XX:+UseParNewGC -XX:-OmitStackTraceInFastThrow -Dlog4j.configurationFile=/usr/share/graylog/data/config/log4j2.xml -Djava.library.path=/usr/share/graylog/lib/sigar/ -Dgraylog2.installation_source=docker
2017-08-10 08:38:13,753 INFO : org.hibernate.validator.internal.util.Version - HV000001: Hibernate Validator null
2017-08-10 08:38:14,951 INFO : org.graylog2.shared.buffers.InputBufferImpl - Message journal is enabled.
2017-08-10 08:38:14,963 INFO : org.graylog2.plugin.system.NodeId - Node ID: f51d48d9-bc89-453e-84ec-549122ff5e50
2017-08-10 08:38:15,079 INFO : kafka.log.LogManager - Loading logs.
2017-08-10 08:38:15,115 INFO : kafka.log.LogManager - Logs loading complete.
2017-08-10 08:38:15,115 INFO : org.graylog2.shared.journal.KafkaJournal - Initialized Kafka based journal at /usr/share/graylog/data/journal
2017-08-10 08:38:15,124 INFO : org.graylog2.shared.buffers.InputBufferImpl - Initialized InputBufferImpl with ring size <65536> and wait strategy <BlockingWaitStrategy>, running 2 parallel message handlers.
2017-08-10 08:38:15,134 INFO : org.mongodb.driver.cluster - Cluster created with settings {hosts=[XXX:27018], mode=SINGLE, requiredClusterType=UNKNOWN, serverSelectionTimeout='30000 ms', maxWaitQueueSize=500}
2017-08-10 08:38:15,160 INFO : org.mongodb.driver.cluster - No server chosen by ReadPreferenceServerSelector{readPreference=primary} from cluster description ClusterDescription{type=UNKNOWN, connectionMode=SINGLE, serverDescriptions=[ServerDescription{address=XXX:27018, type=UNKNOWN, state=CONNECTING}]}. Waiting for 30000 ms before timing out
2017-08-10 08:38:15,179 INFO : org.mongodb.driver.connection - Opened connection [connectionId{localValue:1, serverValue:17400}] to XXX:27018
2017-08-10 08:38:15,182 INFO : org.mongodb.driver.cluster - Monitor thread successfully connected to server with description ServerDescription{address=XXX:27018, type=STANDALONE, state=CONNECTED, ok=true, version=ServerVersion{versionList=[3, 2, 8]}, minWireVersion=0, maxWireVersion=4, maxDocumentSize=16777216, roundTripTimeNanos=1837603}
2017-08-10 08:38:15,190 INFO : org.mongodb.driver.connection - Opened connection [connectionId{localValue:2, serverValue:17401}] to XXX:27018
2017-08-10 08:38:15,399 INFO : io.searchbox.client.AbstractJestClient - Setting server pool to a list of 1 servers: [http://XXXXX:9200]
2017-08-10 08:38:15,400 INFO : io.searchbox.client.JestClientFactory - Using multi thread/connection supporting pooling connection manager
2017-08-10 08:38:15,450 INFO : io.searchbox.client.JestClientFactory - Using custom ObjectMapper instance
2017-08-10 08:38:15,450 INFO : io.searchbox.client.JestClientFactory - Node Discovery disabled...
2017-08-10 08:38:15,450 INFO : io.searchbox.client.JestClientFactory - Idle connection reaping disabled...
2017-08-10 08:38:15,702 INFO : org.graylog2.shared.buffers.ProcessBuffer - Initialized ProcessBuffer with ring size <65536> and wait strategy <BlockingWaitStrategy>.
2017-08-10 08:38:16,953 INFO : org.graylog2.bindings.providers.RulesEngineProvider - No static rules file loaded.
2017-08-10 08:38:17,110 WARN : org.graylog.plugins.map.geoip.GeoIpResolverEngine - GeoIP database file does not exist: /etc/graylog/server/GeoLite2-City.mmdb
2017-08-10 08:38:17,118 INFO : org.graylog2.buffers.OutputBuffer - Initialized OutputBuffer with ring size <65536> and wait strategy <BlockingWaitStrategy>.
2017-08-10 08:38:17,165 WARN : org.graylog.plugins.map.geoip.GeoIpResolverEngine - GeoIP database file does not exist: /etc/graylog/server/GeoLite2-City.mmdb
2017-08-10 08:38:17,219 WARN : org.graylog.plugins.map.geoip.GeoIpResolverEngine - GeoIP database file does not exist: /etc/graylog/server/GeoLite2-City.mmdb
2017-08-10 08:38:17,269 WARN : org.graylog.plugins.map.geoip.GeoIpResolverEngine - GeoIP database file does not exist: /etc/graylog/server/GeoLite2-City.mmdb
2017-08-10 08:38:17,321 WARN : org.graylog.plugins.map.geoip.GeoIpResolverEngine - GeoIP database file does not exist: /etc/graylog/server/GeoLite2-City.mmdb
2017-08-10 08:38:17,613 INFO : org.graylog2.bootstrap.ServerBootstrap - Graylog server 2.3.0+81f8228 starting up
2017-08-10 08:38:17,613 INFO : org.graylog2.bootstrap.ServerBootstrap - JRE: Oracle Corporation 1.8.0_72-internal on Linux 4.4.0-31-generic
2017-08-10 08:38:17,613 INFO : org.graylog2.bootstrap.ServerBootstrap - Deployment: docker
2017-08-10 08:38:17,613 INFO : org.graylog2.bootstrap.ServerBootstrap - OS: Debian GNU/Linux 8 (jessie) (debian)
2017-08-10 08:38:17,613 INFO : org.graylog2.bootstrap.ServerBootstrap - Arch: amd64
2017-08-10 08:38:17,615 WARN : org.graylog2.shared.events.DeadEventLoggingListener - Received unhandled event of type <org.graylog2.plugin.lifecycles.Lifecycle> from event bus <AsyncEventBus{graylog-eventbus}>
2017-08-10 08:38:17,636 INFO : org.graylog2.shared.initializers.PeriodicalsService - Starting 26 periodicals ...
2017-08-10 08:38:17,636 INFO : org.graylog2.periodical.Periodicals - Starting [org.graylog2.periodical.ThroughputCalculator] periodical in [0s], polling every [1s].
2017-08-10 08:38:17,637 INFO : org.graylog2.periodical.Periodicals - Starting [org.graylog2.periodical.AlertScannerThread] periodical in [10s], polling every [60s].
2017-08-10 08:38:17,637 INFO : org.graylog2.periodical.Periodicals - Starting [org.graylog2.periodical.BatchedElasticSearchOutputFlushThread] periodical in [0s], polling every [1s].
2017-08-10 08:38:17,638 INFO : org.graylog2.periodical.Periodicals - Starting [org.graylog2.periodical.ClusterHealthCheckThread] periodical in [120s], polling every [20s].
2017-08-10 08:38:17,639 INFO : org.graylog2.periodical.Periodicals - Starting [org.graylog2.periodical.ContentPackLoaderPeriodical] periodical, running forever.
2017-08-10 08:38:17,641 INFO : org.graylog2.periodical.Periodicals - Starting [org.graylog2.periodical.GarbageCollectionWarningThread] periodical, running forever.
2017-08-10 08:38:17,641 INFO : org.graylog2.periodical.Periodicals - Starting [org.graylog2.periodical.IndexerClusterCheckerThread] periodical in [0s], polling every [30s].
2017-08-10 08:38:17,641 INFO : org.graylog2.periodical.Periodicals - Starting [org.graylog2.periodical.IndexRetentionThread] periodical in [0s], polling every [300s].
2017-08-10 08:38:17,642 INFO : org.graylog2.periodical.Periodicals - Starting [org.graylog2.periodical.IndexRotationThread] periodical in [0s], polling every [10s].
2017-08-10 08:38:17,642 INFO : org.graylog2.periodical.Periodicals - Starting [org.graylog2.periodical.NodePingThread] periodical in [0s], polling every [1s].
2017-08-10 08:38:17,642 INFO : org.graylog2.periodical.Periodicals - Starting [org.graylog2.periodical.VersionCheckThread] periodical in [300s], polling every [1800s].
2017-08-10 08:38:17,642 INFO : org.graylog2.periodical.Periodicals - Starting [org.graylog2.periodical.ThrottleStateUpdaterThread] periodical in [1s], polling every [1s].
2017-08-10 08:38:17,643 INFO : org.graylog2.periodical.Periodicals - Starting [org.graylog2.events.ClusterEventPeriodical] periodical in [0s], polling every [1s].
2017-08-10 08:38:17,643 INFO : org.graylog2.periodical.Periodicals - Starting [org.graylog2.events.ClusterEventCleanupPeriodical] periodical in [0s], polling every [86400s].
2017-08-10 08:38:17,643 INFO : org.graylog2.periodical.Periodicals - Starting [org.graylog2.periodical.ClusterIdGeneratorPeriodical] periodical, running forever.
2017-08-10 08:38:17,643 INFO : org.graylog2.periodical.Periodicals - Starting [org.graylog2.periodical.IndexRangesMigrationPeriodical] periodical, running forever.
2017-08-10 08:38:17,645 INFO : org.graylog2.periodical.Periodicals - Starting [org.graylog2.periodical.IndexRangesCleanupPeriodical] periodical in [15s], polling every [3600s].
2017-08-10 08:38:17,647 INFO : org.mongodb.driver.connection - Opened connection [connectionId{localValue:4, serverValue:17403}] to XXX:27018
2017-08-10 08:38:17,648 INFO : org.mongodb.driver.connection - Opened connection [connectionId{localValue:3, serverValue:17402}] to XXX:27018
2017-08-10 08:38:17,649 INFO : org.mongodb.driver.connection - Opened connection [connectionId{localValue:5, serverValue:17404}] to XXX:27018
2017-08-10 08:38:17,655 INFO : org.mongodb.driver.connection - Opened connection [connectionId{localValue:9, serverValue:17408}] to XXX:27018
2017-08-10 08:38:17,658 INFO : org.mongodb.driver.connection - Opened connection [connectionId{localValue:11, serverValue:17410}] to XXX:27018
2017-08-10 08:38:17,659 INFO : org.mongodb.driver.connection - Opened connection [connectionId{localValue:6, serverValue:17405}] to XXX:27018
2017-08-10 08:38:17,659 INFO : org.mongodb.driver.connection - Opened connection [connectionId{localValue:7, serverValue:17406}] to XXX:27018
2017-08-10 08:38:17,661 INFO : org.mongodb.driver.connection - Opened connection [connectionId{localValue:10, serverValue:17409}] to XXX:27018
2017-08-10 08:38:17,662 INFO : org.mongodb.driver.connection - Opened connection [connectionId{localValue:8, serverValue:17407}] to XXX:27018
2017-08-10 08:38:17,678 INFO : org.graylog2.shared.initializers.PeriodicalsService - Not starting [org.graylog2.periodical.UserPermissionMigrationPeriodical] periodical. Not configured to run on this node.
2017-08-10 08:38:17,678 INFO : org.graylog2.periodical.Periodicals - Starting [org.graylog2.periodical.AlarmCallbacksMigrationPeriodical] periodical, running forever.
2017-08-10 08:38:17,678 INFO : org.graylog2.periodical.Periodicals - Starting [org.graylog2.periodical.ConfigurationManagementPeriodical] periodical, running forever.
2017-08-10 08:38:17,684 INFO : org.graylog2.periodical.Periodicals - Starting [org.graylog2.periodical.LdapGroupMappingMigration] periodical, running forever.
2017-08-10 08:38:17,686 INFO : org.graylog2.periodical.Periodicals - Starting [org.graylog2.periodical.IndexFailuresPeriodical] periodical, running forever.
2017-08-10 08:38:17,686 INFO : org.graylog2.periodical.Periodicals - Starting [org.graylog.plugins.usagestatistics.UsageStatsNodePeriodical] periodical in [300s], polling every [21600s].
2017-08-10 08:38:17,688 INFO : org.graylog2.periodical.Periodicals - Starting [org.graylog.plugins.usagestatistics.UsageStatsClusterPeriodical] periodical in [300s], polling every [21600s].
2017-08-10 08:38:17,731 INFO : org.graylog2.periodical.Periodicals - Starting [org.graylog.plugins.pipelineprocessor.periodical.LegacyDefaultStreamMigration] periodical, running forever.
2017-08-10 08:38:17,733 INFO : org.graylog2.periodical.Periodicals - Starting [org.graylog.plugins.collector.periodical.PurgeExpiredCollectorsThread] periodical in [0s], polling every [3600s].
2017-08-10 08:38:17,740 INFO : org.graylog.plugins.pipelineprocessor.periodical.LegacyDefaultStreamMigration - Legacy default stream has no connections, no migration needed.
2017-08-10 08:38:17,961 INFO : org.graylog2.shared.initializers.JerseyService - Enabling CORS for HTTP endpoint
2017-08-10 08:38:25,120 INFO : org.glassfish.grizzly.http.server.NetworkListener - Started listener bound to [0.0.0.0:9000]
2017-08-10 08:38:25,121 INFO : org.glassfish.grizzly.http.server.HttpServer - [HttpServer] Started.
2017-08-10 08:38:25,121 INFO : org.graylog2.shared.initializers.JerseyService - Started REST API at <http://0.0.0.0:9000/api/>
2017-08-10 08:38:25,121 INFO : org.graylog2.shared.initializers.JerseyService - Started Web Interface at <http://0.0.0.0:9000/>
2017-08-10 08:38:25,122 INFO : org.graylog2.shared.initializers.ServiceManagerListener - Services are healthy
2017-08-10 08:38:25,122 INFO : org.graylog2.bootstrap.ServerBootstrap - Services started, startup times in ms: {InputSetupService [RUNNING]=1, OutputSetupService [RUNNING]=4, BufferSynchronizerService [RUNNING]=4, JournalReader [RUNNING]=4, KafkaJournal [RUNNING]=7, ConfigurationEtagService [RUNNING]=8, StreamCacheService [RUNNING]=14, LookupTableService [RUNNING]=28, PeriodicalsService [RUNNING]=106, JerseyService [RUNNING]=7487}
2017-08-10 08:38:25,122 INFO : org.graylog2.shared.initializers.InputSetupService - Triggering launching persisted inputs, node transitioned from Uninitialized [LB:DEAD] to Running [LB:ALIVE]
2017-08-10 08:38:25,127 INFO : org.graylog2.bootstrap.ServerBootstrap - Graylog server up and running.
2017-08-10 08:38:25,140 INFO : org.graylog2.inputs.InputStateListener - Input [GELF TCP/598c1a6c46e0fb0001f08423] is now STARTING
2017-08-10 08:38:25,172 WARN : org.graylog2.plugin.inputs.transports.NettyTransport - receiveBufferSize (SO_RCVBUF) for input GELFTCPInput{title=GelfTCP, type=org.graylog2.inputs.gelf.tcp.GELFTCPInput, nodeId=null} should be 1048576 but is 212992.
2017-08-10 08:38:25,176 INFO : org.graylog2.inputs.InputStateListener - Input [GELF TCP/598c1a6c46e0fb0001f08423] is now RUNNING
2017-08-10 08:40:41,355 INFO : org.apache.shiro.session.mgt.AbstractValidatingSessionManager - Enabling session validation scheduler...
2017-08-10 08:45:41,355 INFO : org.apache.shiro.session.mgt.AbstractValidatingSessionManager - Validating all active sessions...
2017-08-10 08:45:41,359 INFO : org.apache.shiro.session.mgt.AbstractValidatingSessionManager - Finished session validation. No sessions were stopped.
Here are my logs. I changed IP’s to XXX. There are no connection issues between Elasticsearch and Graylog servers. No firewall rules.
Elasticsearch version :
{
"name" : "node-0",
"cluster_name" : "dersosearch",
"cluster_uuid" : "L2FJkM_wTr27gImph-khhg",
"version" : {
"number" : "5.5.1",
"build_hash" : "19c13d0",
"build_date" : "2017-07-18T20:44:24.823Z",
"build_snapshot" : false,
"lucene_version" : "6.6.0"
},
"tagline" : "You Know, for Search"
}
Graylog Version is 2.3.1
I see that you’re using Docker to run Graylog.
What’s the configuration of your Docker container and Graylog?
docker run -v /data/graylog/journal:/usr/share/graylog/data/journal \
-v /data/graylog/mongo:/data/db \
-p 9000:9000 \
-p 12201:12201 \
-p 12202:12202 \
-e GRAYLOG_WEB_ENDPOINT_URI=http://XXX:9000/api \
-e GRAYLOG_ELASTICSEARCH_HOSTS=XXX:9200 \
-e GRAYLOG_MONGODB_URI=mongodb://XXX:27018 \
-d graylog2/server:latest
Other than this, i didn’t change anything in any config file. You can assume all URI’s and hosts are correct.
Judging from the description in your first post, they probably aren’t.
Are there any error messages in the Javascript Console of your web browser when using the Graylog web interface?
Yes there are actually :
Unhandled rejection TypeError: Cannot read property 'end' of undefined
at t._determineSearchDuration (http://XXX:9000/assets/53.8e41d69eb32b36020c26.js:1:3330)
at t._determineHistogramResolution (http://XXX:9000/assets/53.8e41d69eb32b36020c26.js:1:3527)
at promise.H.search.then.e.additional.e.additional.status.t.setState.error (http://XXX:9000/assets/53.8e41d69eb32b36020c26.js:1:2515)
From previous event:
at o.p [as _captureStackTrace] (http://XXX:9000/assets/plugin/org.graylog.plugins.pipelineprocessor.ProcessorPlugin/plugin.org.graylog.plugins.pipelineprocessor.PipelineProcessorPlugin.31a2373cd5652763947c.js:44:10310)
at o._then (http://XXX:9000/assets/plugin/org.graylog.plugins.pipelineprocessor.ProcessorPlugin/plugin.org.graylog.plugins.pipelineprocessor.PipelineProcessorPlugin.31a2373cd5652763947c.js:44:41761)
at o.then (http://XXX:9000/assets/plugin/org.graylog.plugins.pipelineprocessor.ProcessorPlugin/plugin.org.graylog.plugins.pipelineprocessor.PipelineProcessorPlugin.31a2373cd5652763947c.js:44:40117)
at s.listSearchesClusterConfig (http://XXX:9000/assets/plugin/org.graylog.plugins.pipelineprocessor.ProcessorPlugin/plugin.org.graylog.plugins.pipelineprocessor.PipelineProcessorPlugin.31a2373cd5652763947c.js:104:1068481)
at o.n (http://XXX:9000/assets/vendor.39412008ebba2755b8a2.js:28:195783)
at o.emit (http://XXX:9000/assets/vendor.39412008ebba2755b8a2.js:23:30345)
at Function.trigger (http://XXX:9000/assets/vendor.39412008ebba2755b8a2.js:28:196658)
at http://XXX:9000/assets/vendor.39412008ebba2755b8a2.js:28:196762
at t.exports (http://XXX:9000/assets/polyfill.8e41d69eb32b36020c26.js:1:18867)
at y.(anonymous function) (http://XXX:9000/assets/polyfill.8e41d69eb32b36020c26.js:1:25291)
at Number.d (http://XXX:9000/assets/polyfill.8e41d69eb32b36020c26.js:1:25149)
at MessagePort.g (http://XXX:9000/assets/polyfill.8e41d69eb32b36020c26.js:1:25171)
L @ plugin.org.graylog.plugins.pipelineprocessor.PipelineProcessorPlugin.31a2373cd5652763947c.js:44
Is this the only error?
Yes and a warning :
Warning: You cannot change <Router routes>; it will be ignored
The error only comes up if i click on the “Show received messages” button.
Hey @KatranAdam,
can you please go to the search page again and check the network tab in your browser’s script console and check for any requests going to the Graylog server starting with /search/universal? Please paste the response you get from the server, if it does not contain any sensitive information.
Thanks!
Hey @dennis,
I’ve waited for several minutes but no request starting with /search/universal. Not at all.
More information : Even though i can get messages and see that i’m getting them, search doesn’t bring up any results if i use time intervals. If i search all messages, loading takes forever.
Is there any request which contains /search/universal in the URL?
Oops my bad i guess, there is something :
{query: "gl2_source_input:598c564146e0fb0001bde36b",…}
built_query
:
"{↵ "from" : 0,↵ "size" : 150,↵ "query" : {↵ "bool" : {↵ "must" : {↵ "query_string" : {↵ "query" : "gl2_source_input:598c564146e0fb0001bde36b",↵ "allow_leading_wildcard" : true↵ }↵ },↵ "filter" : {↵ "bool" : {↵ "must" : {↵ "range" : {↵ "timestamp" : {↵ "from" : "1970-01-01 00:00:00.000",↵ "to" : "2017-08-10 13:39:19.677",↵ "include_lower" : true,↵ "include_upper" : true↵ }↵ }↵ }↵ }↵ }↵ }↵ },↵ "sort" : [ {↵ "timestamp" : {↵ "order" : "desc"↵ }↵ } ],↵ "highlight" : {↵ "fragment_size" : 0,↵ "number_of_fragments" : 0,↵ "require_field_match" : false,↵ "fields" : {↵ "*" : { }↵ }↵ }↵}"
decoration_stats
:
null
fields
:
[]
from
:
"1970-01-01T00:00:00.000Z"
messages
:
[]
query
:
"gl2_source_input:598c564146e0fb0001bde36b"
time
:
0
to
:
"2017-08-10T13:39:19.677Z"
total_results
:
0
used_indices
:
[]
That is not really helpful. Is this the response body? Can you paste it in a way which is preserving the actual structure of the response? If nothing else works, please create a gist on github.
How about this :
Params :
query: "gl2_source_input:598c564146e0fb0001bde36b"
range: "0"
limit: "150"
sort: "timestamp:desc"
Response :
query: "gl2_source_input:598c564146e0fb0001bde36b"
built_query: { "from" : 0, "size" : 150, "query" : { "bool" : { "must" : { "query_string" : { "query" : "gl2_source_input:598c564146e0fb0001bde36b", "allow_leading_wildcard" : true } }, "filter" : { "bool" : { "must" : { "range" : { "timestamp" : { "from" : "1970-01-01 00:00:00.000", "to" : "2017-08-10 14:51:09.162", "include_lower" : true, "include_upper" : true } } } } } } }, "sort" : [ { "timestamp" : { "order" : "desc" } } ], "highlight" : { "fragment_size" : 0, "number_of_fragments" : 0, "require_field_match" : false, "fields" : { "*" : { } } }}
used_indices: []
messages: []
fields: []
time: 0
total_results: 0
from: "1970-01-01T00:00:00.000Z"
to: "2017-08-10T14:51:09.162Z"
decoration_stats: null
Some more info :
If i bring down the Elasticsearch cluster and restart Graylog instance i got
2017-08-11 07:20:59,998 INFO : org.graylog2.periodical.IndexRetentionThread - Elasticsearch cluster not available, skipping index retention checks.
2017-08-11 07:20:59,998 ERROR: org.graylog2.indexer.cluster.Cluster - Couldn't read cluster health for indices [graylog_*] (Could not connect to http://XXX:9200)
Which i don’t get if the Elasticsearch is up and running which means that Graylog can connect to Elasticsearch no errors in my config or environment variables. I’m really lost right now.
If i docker up an Elasticsearch image this comes up in the logs which i believe means that Graylog successfully connected to Elasticsearch cluster.It doesn’t show up if i give my original Elasticsearch cluster as an environment variable.
graylog_1 | 2017-08-11 07:35:01,312 INFO : org.elasticsearch.node - [graylog-b434a95f-77e8-4c72-8f4b-a626f317b655] initialized
Issue is resolved after reinstalling Elasticsearch with ‘exact same’ configuration as we had before.
You can tag this as solved.
This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.