Spun up new Amazon AMI of Graylog 2.2.3, deleted local inputs and created new global input for GELF-UDP.
Tried to click “Start Input” and it reports and error, that it can’t start.
Try using Search to find the local logs, last log says the input is running.
How do I troubleshoot this?
What error did you see?
What’s in the logs of your Graylog nodes? http://docs.graylog.org/en/2.2/pages/configuration/file_location.html#omnibus-package
After clicking Start on any input (now determined to be even the initial inputs), wait after 15-30 seconds…
In the logs…
This happens during startup…
2017-06-20_17:26:12.65762 2017-06-20 17:26:12,657 INFO : org.graylog2.inputs.InputStateListener - Input [GELF UDP/59495ab42c949a1a68c288e2] is now STARTING
2017-06-20_17:26:12.66345 2017-06-20 17:26:12,663 INFO : org.graylog2.inputs.InputStateListener - Input [Syslog UDP/59495ab42c949a1a68c288f8] is now STARTING
2017-06-20_17:26:12.66587 2017-06-20 17:26:12,665 INFO : org.graylog2.migrations.V20161125161400_AlertReceiversMigration - No streams needed to be migrated.
2017-06-20_17:26:12.73894 2017-06-20 17:26:12,738 WARN : org.graylog2.plugin.inputs.transports.NettyTransport - receiveBufferSize (SO_RCVBUF) for input GELFUDPInput{title=appliance-gelf-udp, type=org.graylog2.inputs.gelf.udp.GELFUDPInput, nodeId=null} should be 1048576 but is 212992.
2017-06-20_17:26:12.74576 2017-06-20 17:26:12,745 INFO : org.graylog2.inputs.InputStateListener - Input [GELF UDP/59495ab42c949a1a68c288e2] is now RUNNING
2017-06-20_17:26:12.75850 2017-06-20 17:26:12,758 INFO : org.graylog2.inputs.InputStateListener - Input [Syslog UDP/59495ab42c949a1a68c288f8] is now RUNNING
2017-06-20_17:26:13.09749 2017-06-20 17:26:13,097 INFO : org.graylog2.shared.initializers.JerseyService - Enabling CORS for HTTP endpoint
2017-06-20_17:26:13.21756 2017-06-20 17:26:13,216 INFO : org.mongodb.driver.connection - Opened connection [connectionId{localValue:5, serverValue:9}] to 127.0.0.1:27017
2017-06-20_17:26:13.22174 2017-06-20 17:26:13,221 INFO : org.mongodb.driver.connection - Opened connection [connectionId{localValue:6, serverValue:10}] to 127.0.0.1:27017
2017-06-20_17:26:13.22729 2017-06-20 17:26:13,226 INFO : org.mongodb.driver.connection - Opened connection [connectionId{localValue:7, serverValue:11}] to 127.0.0.1:27017
2017-06-20_17:26:13.23418 2017-06-20 17:26:13,232 INFO : org.mongodb.driver.connection - Opened connection [connectionId{localValue:9, serverValue:13}] to 127.0.0.1:27017
2017-06-20_17:26:13.23825 2017-06-20 17:26:13,237 INFO : org.mongodb.driver.connection - Opened connection [connectionId{localValue:10, serverValue:14}] to 127.0.0.1:27017
2017-06-20_17:26:13.24443 2017-06-20 17:26:13,244 INFO : org.mongodb.driver.connection - Opened connection [connectionId{localValue:11, serverValue:15}] to 127.0.0.1:27017
2017-06-20_17:26:13.24721 2017-06-20 17:26:13,246 INFO : org.mongodb.driver.connection - Opened connection [connectionId{localValue:12, serverValue:16}] to 127.0.0.1:27017
2017-06-20_17:26:13.27972 2017-06-20 17:26:13,279 INFO : org.mongodb.driver.connection - Opened connection [connectionId{localValue:14, serverValue:17}] to 127.0.0.1:27017
2017-06-20_17:26:13.34388 2017-06-20 17:26:13,340 INFO : org.mongodb.driver.connection - Opened connection [connectionId{localValue:18, serverValue:22}] to 127.0.0.1:27017
2017-06-20_17:26:13.34439 2017-06-20 17:26:13,340 INFO : org.mongodb.driver.connection - Opened connection [connectionId{localValue:19, serverValue:24}] to 127.0.0.1:27017
2017-06-20_17:26:13.39797 2017-06-20 17:26:13,397 INFO : org.graylog2.inputs.InputStateListener - Input [GELF UDP/59495ab42c949a1a68c288e2] is now STOPPING
2017-06-20_17:26:13.46299 2017-06-20 17:26:13,462 INFO : org.graylog2.inputs.InputStateListener - Input [GELF UDP/59495ab42c949a1a68c288e2] is now STOPPED
2017-06-20_17:26:13.49017 2017-06-20 17:26:13,490 INFO : org.graylog2.inputs.InputStateListener - Input [GELF UDP/59495ab42c949a1a68c288e2] is now STARTING
2017-06-20_17:26:13.65595 2017-06-20 17:26:13,655 INFO : org.mongodb.driver.connection - Opened connection [connectionId{localValue:22, serverValue:26}] to 127.0.0.1:27017
2017-06-20_17:26:13.66442 2017-06-20 17:26:13,664 INFO : org.mongodb.driver.connection - Opened connection [connectionId{localValue:23, serverValue:27}] to 127.0.0.1:27017
2017-06-20_17:26:13.70079 2017-06-20 17:26:13,700 WARN : org.graylog2.plugin.inputs.transports.NettyTransport - receiveBufferSize (SO_RCVBUF) for input GELFUDPInput{title=appliance-gelf-udp, type=org.graylog2.inputs.gelf.udp.GELFUDPInput, nodeId=b0c6b85e-4ea8-4d16-ad0e-96e927d19a7a} should be 1048576 but is 212992.
2017-06-20_17:26:13.76710 2017-06-20 17:26:13,756 INFO : org.graylog2.inputs.InputStateListener - Input [GELF UDP/59495ab42c949a1a68c288e2] is now RUNNING
2017-06-20_17:26:15.35097 2017-06-20 17:26:15,350 INFO : org.elasticsearch.node - [graylog-b0c6b85e-4ea8-4d16-ad0e-96e927d19a7a] started
2017-06-20_17:26:20.36232 2017-06-20 17:26:20,361 INFO : org.graylog2.initializers.IndexerSetupService - If you're using multicast, check that it is working in your network and that Elasticsearch is accessible. Also check that the cluster name setting is correct.
2017-06-20_17:26:22.52234 2017-06-20 17:26:22,522 INFO : org.glassfish.grizzly.http.server.HttpServer - [HttpServer] Started.
2017-06-20_17:26:22.52473 2017-06-20 17:26:22,524 INFO : org.graylog2.shared.initializers.JerseyService - Started REST API at <http://0.0.0.0:12900/>
2017-06-20_17:26:24.34274 2017-06-20 17:26:24,342 INFO : org.glassfish.grizzly.http.server.NetworkListener - Started listener bound to [0.0.0.0:9000]
2017-06-20_17:26:24.34911 2017-06-20 17:26:24,348 INFO : org.graylog2.shared.initializers.InputSetupService - Triggering launching persisted inputs, node transitioned from Uninitialized?[LB:DEAD] to Running?[LB:ALIVE]
2017-06-20_17:26:24.35107 2017-06-20 17:26:24,350 INFO : org.graylog2.bootstrap.ServerBootstrap - Services started, startup times in ms: {InputSetupService [RUNNING]=6, OutputSetupService [RUNNING]=34, KafkaJournal [RUNNING]=35, BufferSynchronizerService [RUNNING]=37, JournalReader [RUNNING]=54, ConfigurationEtagService [RUNNING]=78, StreamCacheService [RUNNING]=142, PeriodicalsService [RUNNING]=352, IndexerSetupService [RUNNING]=8404, JerseyService [RUNNING]=12386}
2017-06-20_17:26:24.35482 2017-06-20 17:26:24,354 INFO : org.graylog2.bootstrap.ServerBootstrap - Graylog server up and running.
This piece loops into the logs periodically after.
2017-06-20_19:25:58.10752 2017-06-20 19:25:58,106 WARN : org.graylog2.shared.rest.resources.ProxiedResource - Unable to call https://graylogweb-uswest.insidesales.com:12900/system/inputstates on node <b0c6b85e-4ea8-4d16-ad0e-96e927d19a7a>
During the attempts to start the input, the logs don’t appear to change much or reflect the attempts to start the not-running inputs. This entry pops up alongside the previous error, that appears to be all that reflects the issue.
2017-06-21_16:54:31.05256 2017-06-21 16:54:30,928 WARN : org.graylog2.shared.rest.resources.ProxiedResource - Unable to call https://graylogweb-uswest.insidesales.com:12900/system/inputstates on node <b0c6b85e-4ea8-4d16-ad0e-96e927d
19a7a>
I’ll grab anything else you might want to see
Any suggestions on why inputs would not be able to start?
if you tail -f on the log file and hit “start input” what did you see in the log file?
I had this issue in the past if i wasn’t running graylog with the privileges to access the port. Try running graylog as root and see what happens
Try running graylog as root and see what happens
no - please do not do this!
This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.