An input has failed to start...but all my inputs are already running

lmm5247 · May 2, 2020, 12:58am

I’m running Graylog 3.2 on Docker (Ubuntu host). Everything else is working 100%, this is my only issue.

I’m receiving the message below every time I restart the Graylog docker container. I’ve tried to clear this notification, but it comes back every time I restart Graylog.
Input 5eacbada64904500124b51bb has failed to start on node 1df75265-96a0-49a4-b387-6303a72b0549 for this reason: »null«.

However, I only have 2 inputs, and they’re both started and working. Where is input 5eacbada64904500124b51bb? Is there a field in the database that I can remove?

Below is my logfile where you can see the input try to start, then fail.

2020-05-02 00:44:57,617 INFO : org.graylog2.inputs.InputStateListener - Input [Syslog UDP/5eacbada64904500124b51bb] is now STARTING
2020-05-02 00:44:57,621 INFO : org.graylog2.inputs.InputStateListener - Input [Syslog UDP/5eacbada64904500124b51bf] is now STARTING
2020-05-02 00:44:57,623 INFO : org.graylog2.inputs.InputStateListener - Input [Syslog UDP/5eacbada64904500124b51bf] is now STOPPING
2020-05-02 00:44:57,626 INFO : org.graylog2.inputs.InputStateListener - Input [Syslog UDP/5eacbada64904500124b51bf] is now TERMINATED
2020-05-02 00:44:57,629 INFO : org.graylog2.inputs.InputStateListener - Input [Syslog UDP/5eacbada64904500124b51bb] is now STOPPING
2020-05-02 00:44:57,631 INFO : org.graylog2.inputs.InputStateListener - Input [Syslog UDP/5eacbada64904500124b51bb] is now STOPPED
2020-05-02 00:44:57,634 INFO : org.graylog2.inputs.InputStateListener - Input [Syslog UDP/5eacbada64904500124b51bb] is now TERMINATED
2020-05-02 00:44:57,637 INFO : org.graylog2.inputs.InputStateListener - Input [Syslog UDP/5eacbada64904500124b51bf] is now STOPPED
2020-05-02 00:44:57,716 ERROR: org.graylog2.shared.inputs.InputLauncher - The [org.graylog2.inputs.syslog.udp.SyslogUDPInput] input with ID <5eacbada64904500124b51bb> misfired. Reason: null
org.graylog2.plugin.inputs.MisfireException: org.graylog2.plugin.inputs.MisfireException: java.lang.NullPointerException
	at org.graylog2.plugin.inputs.MessageInput.launch(MessageInput.java:158) ~[graylog.jar:?]
	at org.graylog2.shared.inputs.InputLauncher$1.run(InputLauncher.java:84) [graylog.jar:?]
	at com.codahale.metrics.InstrumentedExecutorService$InstrumentedRunnable.run(InstrumentedExecutorService.java:181) [graylog.jar:?]
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) [?:1.8.0_242]
	at java.util.concurrent.FutureTask.run(FutureTask.java:266) [?:1.8.0_242]
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:1.8.0_242]
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0_242]
	at java.lang.Thread.run(Thread.java:748) [?:1.8.0_242]
Caused by: org.graylog2.plugin.inputs.MisfireException: java.lang.NullPointerException
	at org.graylog2.inputs.transports.UdpTransport.launch(UdpTransport.java:135) ~[graylog.jar:?]
	at org.graylog2.plugin.inputs.MessageInput.launch(MessageInput.java:155) ~[graylog.jar:?]
	... 7 more
Caused by: java.lang.NullPointerException
	at org.graylog2.inputs.transports.UdpTransport.launch(UdpTransport.java:130) ~[graylog.jar:?]
	at org.graylog2.plugin.inputs.MessageInput.launch(MessageInput.java:155) ~[graylog.jar:?]
	... 7 more
2020-05-02 00:44:57,718 ERROR: org.graylog2.shared.inputs.InputLauncher - The [org.graylog2.inputs.syslog.udp.SyslogUDPInput] input with ID <5eacbada64904500124b51bf> misfired. Reason: null
org.graylog2.plugin.inputs.MisfireException: org.graylog2.plugin.inputs.MisfireException: java.lang.NullPointerException
	at org.graylog2.plugin.inputs.MessageInput.launch(MessageInput.java:158) ~[graylog.jar:?]
	at org.graylog2.shared.inputs.InputLauncher$1.run(InputLauncher.java:84) [graylog.jar:?]
	at com.codahale.metrics.InstrumentedExecutorService$InstrumentedRunnable.run(InstrumentedExecutorService.java:181) [graylog.jar:?]
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) [?:1.8.0_242]
	at java.util.concurrent.FutureTask.run(FutureTask.java:266) [?:1.8.0_242]
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:1.8.0_242]
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0_242]
	at java.lang.Thread.run(Thread.java:748) [?:1.8.0_242]
Caused by: org.graylog2.plugin.inputs.MisfireException: java.lang.NullPointerException
	at org.graylog2.inputs.transports.UdpTransport.launch(UdpTransport.java:135) ~[graylog.jar:?]
	at org.graylog2.plugin.inputs.MessageInput.launch(MessageInput.java:155) ~[graylog.jar:?]
	... 7 more
Caused by: java.lang.NullPointerException
	at org.graylog2.inputs.transports.UdpTransport.launch(UdpTransport.java:130) ~[graylog.jar:?]
	at org.graylog2.plugin.inputs.MessageInput.launch(MessageInput.java:155) ~[graylog.jar:?]
	... 7 more
2020-05-02 00:44:57,727 INFO : org.graylog2.inputs.InputStateListener - Input [Syslog UDP/5eacbada64904500124b51bb] is now FAILED
2020-05-02 00:44:57,731 INFO : org.graylog2.inputs.InputStateListener - Input [Syslog UDP/5eacbada64904500124b51bf] is now FAILED

lmm5247 · May 2, 2020, 2:40am

Some more investigating…

I deleted both inputs from the Graylog web interface and verified in the Mongo console they were gone.

> use graylog
switched to db graylog
> db.inputs.find().pretty()
>

I then created a new input, called pfSense. Note the node_id is the same in both.

> db.inputs.find().pretty()
{
        "_id" : ObjectId("5eacdc496ce0450012252eae"),
        "creator_user_id" : "logan",
        "configuration" : {
                "expand_structured_data" : false,
                "recv_buffer_size" : 262144,
                "port" : 8515,
                "number_worker_threads" : 4,
                "override_source" : null,
                "force_rdns" : false,
                "allow_override_date" : true,
                "bind_address" : "0.0.0.0",
                "store_full_message" : false
        },
        "name" : "Syslog UDP",
        "created_at" : ISODate("2020-05-02T02:34:49.494Z"),
        "global" : false,
        "type" : "org.graylog2.inputs.syslog.udp.SyslogUDPInput",
        "title" : "pfSense",
        "content_pack" : null,
        "node_id" : "163d8203-580b-46bc-91b9-fc79a562c218"
}
> db.nodes.find().pretty()
{
        "_id" : ObjectId("5eacdb3f6ce0450012252d8d"),
        "is_master" : true,
        "hostname" : "80c88c0c7799",
        "last_seen" : 1588386923,
        "transport_address" : "http://192.168.160.3:9000/api/",
        "type" : "SERVER",
        "node_id" : "163d8203-580b-46bc-91b9-fc79a562c218"
}
>

However, after a restart of Graylog, the input still wasn’t starting (with the same error). But I noticed my node_id is changing. Notice how the input still has the old node_id, but the actual node_id is changed? Why is this happening? Can I set this manually with Docker?

> db.inputs.find().pretty()
{
        "_id" : ObjectId("5eacdc496ce0450012252eae"),
        "creator_user_id" : "logan",
        "configuration" : {
                "expand_structured_data" : false,
                "recv_buffer_size" : 262144,
                "port" : 8515,
                "number_worker_threads" : 4,
                "override_source" : null,
                "force_rdns" : false,
                "allow_override_date" : true,
                "bind_address" : "0.0.0.0",
                "store_full_message" : false
        },
        "name" : "Syslog UDP",
        "created_at" : ISODate("2020-05-02T02:34:49.494Z"),
        "global" : false,
        "type" : "org.graylog2.inputs.syslog.udp.SyslogUDPInput",
        "title" : "pfSense",
        "content_pack" : null,
        "node_id" : "163d8203-580b-46bc-91b9-fc79a562c218"
}
> db.nodes.find().pretty()
{
        "_id" : ObjectId("5eacdce03d0045001266f503"),
        "is_master" : true,
        "hostname" : "6890e410deeb",
        "last_seen" : 1588387122,
        "transport_address" : "http://192.168.192.3:9000/api/",
        "type" : "SERVER",
        "node_id" : "86e9b9fb-0cf0-4df9-a1d7-8fbb3ec9f10e"
}
>

system · May 16, 2020, 2:40am

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.

Topic		Replies	Views
Can't start inputs Graylog Central (peer support) docker	2	1406	February 11, 2022
An input has failed to start Graylog Central (peer support)	5	6545	July 30, 2018
Graylog Docker Container doesn't start Graylog Central (peer support) docker	6	996	April 27, 2023
Inputs show failed, but ports are open and logs are coming in Graylog Central (peer support) basic-configuration	7	802	April 28, 2022
Inputs not showing as running, but they seem to be anyway Graylog Central (peer support)	12	3101	March 25, 2021

An input has failed to start...but all my inputs are already running

Related Topics