An input has failed to start...but all my inputs are already running

lmm5247 · May 2, 2020, 12:58am

I’m running Graylog 3.2 on Docker (Ubuntu host). Everything else is working 100%, this is my only issue.

I’m receiving the message below every time I restart the Graylog docker container. I’ve tried to clear this notification, but it comes back every time I restart Graylog.
Input 5eacbada64904500124b51bb has failed to start on node 1df75265-96a0-49a4-b387-6303a72b0549 for this reason: »null«.

However, I only have 2 inputs, and they’re both started and working. Where is input 5eacbada64904500124b51bb? Is there a field in the database that I can remove?

Below is my logfile where you can see the input try to start, then fail.

2020-05-02 00:44:57,617 INFO : org.graylog2.inputs.InputStateListener - Input [Syslog UDP/5eacbada64904500124b51bb] is now STARTING
2020-05-02 00:44:57,621 INFO : org.graylog2.inputs.InputStateListener - Input [Syslog UDP/5eacbada64904500124b51bf] is now STARTING
2020-05-02 00:44:57,623 INFO : org.graylog2.inputs.InputStateListener - Input [Syslog UDP/5eacbada64904500124b51bf] is now STOPPING
2020-05-02 00:44:57,626 INFO : org.graylog2.inputs.InputStateListener - Input [Syslog UDP/5eacbada64904500124b51bf] is now TERMINATED
2020-05-02 00:44:57,629 INFO : org.graylog2.inputs.InputStateListener - Input [Syslog UDP/5eacbada64904500124b51bb] is now STOPPING
2020-05-02 00:44:57,631 INFO : org.graylog2.inputs.InputStateListener - Input [Syslog UDP/5eacbada64904500124b51bb] is now STOPPED
2020-05-02 00:44:57,634 INFO : org.graylog2.inputs.InputStateListener - Input [Syslog UDP/5eacbada64904500124b51bb] is now TERMINATED
2020-05-02 00:44:57,637 INFO : org.graylog2.inputs.InputStateListener - Input [Syslog UDP/5eacbada64904500124b51bf] is now STOPPED
2020-05-02 00:44:57,716 ERROR: org.graylog2.shared.inputs.InputLauncher - The [org.graylog2.inputs.syslog.udp.SyslogUDPInput] input with ID <5eacbada64904500124b51bb> misfired. Reason: null
org.graylog2.plugin.inputs.MisfireException: org.graylog2.plugin.inputs.MisfireException: java.lang.NullPointerException
	at org.graylog2.plugin.inputs.MessageInput.launch(MessageInput.java:158) ~[graylog.jar:?]
	at org.graylog2.shared.inputs.InputLauncher$1.run(InputLauncher.java:84) [graylog.jar:?]
	at com.codahale.metrics.InstrumentedExecutorService$InstrumentedRunnable.run(InstrumentedExecutorService.java:181) [graylog.jar:?]
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) [?:1.8.0_242]
	at java.util.concurrent.FutureTask.run(FutureTask.java:266) [?:1.8.0_242]
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:1.8.0_242]
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0_242]
	at java.lang.Thread.run(Thread.java:748) [?:1.8.0_242]
Caused by: org.graylog2.plugin.inputs.MisfireException: java.lang.NullPointerException
	at org.graylog2.inputs.transports.UdpTransport.launch(UdpTransport.java:135) ~[graylog.jar:?]
	at org.graylog2.plugin.inputs.MessageInput.launch(MessageInput.java:155) ~[graylog.jar:?]
	... 7 more
Caused by: java.lang.NullPointerException
	at org.graylog2.inputs.transports.UdpTransport.launch(UdpTransport.java:130) ~[graylog.jar:?]
	at org.graylog2.plugin.inputs.MessageInput.launch(MessageInput.java:155) ~[graylog.jar:?]
	... 7 more
2020-05-02 00:44:57,718 ERROR: org.graylog2.shared.inputs.InputLauncher - The [org.graylog2.inputs.syslog.udp.SyslogUDPInput] input with ID <5eacbada64904500124b51bf> misfired. Reason: null
org.graylog2.plugin.inputs.MisfireException: org.graylog2.plugin.inputs.MisfireException: java.lang.NullPointerException
	at org.graylog2.plugin.inputs.MessageInput.launch(MessageInput.java:158) ~[graylog.jar:?]
	at org.graylog2.shared.inputs.InputLauncher$1.run(InputLauncher.java:84) [graylog.jar:?]
	at com.codahale.metrics.InstrumentedExecutorService$InstrumentedRunnable.run(InstrumentedExecutorService.java:181) [graylog.jar:?]
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) [?:1.8.0_242]
	at java.util.concurrent.FutureTask.run(FutureTask.java:266) [?:1.8.0_242]
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:1.8.0_242]
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0_242]
	at java.lang.Thread.run(Thread.java:748) [?:1.8.0_242]
Caused by: org.graylog2.plugin.inputs.MisfireException: java.lang.NullPointerException
	at org.graylog2.inputs.transports.UdpTransport.launch(UdpTransport.java:135) ~[graylog.jar:?]
	at org.graylog2.plugin.inputs.MessageInput.launch(MessageInput.java:155) ~[graylog.jar:?]
	... 7 more
Caused by: java.lang.NullPointerException
	at org.graylog2.inputs.transports.UdpTransport.launch(UdpTransport.java:130) ~[graylog.jar:?]
	at org.graylog2.plugin.inputs.MessageInput.launch(MessageInput.java:155) ~[graylog.jar:?]
	... 7 more
2020-05-02 00:44:57,727 INFO : org.graylog2.inputs.InputStateListener - Input [Syslog UDP/5eacbada64904500124b51bb] is now FAILED
2020-05-02 00:44:57,731 INFO : org.graylog2.inputs.InputStateListener - Input [Syslog UDP/5eacbada64904500124b51bf] is now FAILED

lmm5247 · May 2, 2020, 2:40am

Some more investigating…

I deleted both inputs from the Graylog web interface and verified in the Mongo console they were gone.

> use graylog
switched to db graylog
> db.inputs.find().pretty()
>

I then created a new input, called pfSense. Note the node_id is the same in both.

> db.inputs.find().pretty()
{
        "_id" : ObjectId("5eacdc496ce0450012252eae"),
        "creator_user_id" : "logan",
        "configuration" : {
                "expand_structured_data" : false,
                "recv_buffer_size" : 262144,
                "port" : 8515,
                "number_worker_threads" : 4,
                "override_source" : null,
                "force_rdns" : false,
                "allow_override_date" : true,
                "bind_address" : "0.0.0.0",
                "store_full_message" : false
        },
        "name" : "Syslog UDP",
        "created_at" : ISODate("2020-05-02T02:34:49.494Z"),
        "global" : false,
        "type" : "org.graylog2.inputs.syslog.udp.SyslogUDPInput",
        "title" : "pfSense",
        "content_pack" : null,
        "node_id" : "163d8203-580b-46bc-91b9-fc79a562c218"
}
> db.nodes.find().pretty()
{
        "_id" : ObjectId("5eacdb3f6ce0450012252d8d"),
        "is_master" : true,
        "hostname" : "80c88c0c7799",
        "last_seen" : 1588386923,
        "transport_address" : "http://192.168.160.3:9000/api/",
        "type" : "SERVER",
        "node_id" : "163d8203-580b-46bc-91b9-fc79a562c218"
}
>

However, after a restart of Graylog, the input still wasn’t starting (with the same error). But I noticed my node_id is changing. Notice how the input still has the old node_id, but the actual node_id is changed? Why is this happening? Can I set this manually with Docker?

> db.inputs.find().pretty()
{
        "_id" : ObjectId("5eacdc496ce0450012252eae"),
        "creator_user_id" : "logan",
        "configuration" : {
                "expand_structured_data" : false,
                "recv_buffer_size" : 262144,
                "port" : 8515,
                "number_worker_threads" : 4,
                "override_source" : null,
                "force_rdns" : false,
                "allow_override_date" : true,
                "bind_address" : "0.0.0.0",
                "store_full_message" : false
        },
        "name" : "Syslog UDP",
        "created_at" : ISODate("2020-05-02T02:34:49.494Z"),
        "global" : false,
        "type" : "org.graylog2.inputs.syslog.udp.SyslogUDPInput",
        "title" : "pfSense",
        "content_pack" : null,
        "node_id" : "163d8203-580b-46bc-91b9-fc79a562c218"
}
> db.nodes.find().pretty()
{
        "_id" : ObjectId("5eacdce03d0045001266f503"),
        "is_master" : true,
        "hostname" : "6890e410deeb",
        "last_seen" : 1588387122,
        "transport_address" : "http://192.168.192.3:9000/api/",
        "type" : "SERVER",
        "node_id" : "86e9b9fb-0cf0-4df9-a1d7-8fbb3ec9f10e"
}
>

system · May 16, 2020, 2:40am

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.

Topic		Replies	Views
Can't start inputs Graylog Central (peer support) docker	2	1616	February 11, 2022
Input start fail Graylog Central (peer support)	9	859	August 8, 2018
Can't start input on Graylog v3.0.0-alpha.4+6e0b9cd Graylog Central (peer support)	2	6258	December 27, 2018
Graylog not able to start inputs Graylog Central (peer support)	2	397	March 20, 2019
Input fails in UI but shows running in cli Graylog Central (peer support)	11	605	August 31, 2018

An input has failed to start...but all my inputs are already running

Related topics