New OVA Install, Server currently unavailable


(sirkus) #1

Today I downloaded the 2.4.3 OVA, imported it. I looked at the generated MAC address and added a static IP mapping in my DHCP server so the VM would be given a known internal IP when it started (10.5.1.23). I started it up, and received the IP successfully. I ssh’d to it and logged in successfully. I pointed my browser at 10.5.1.23 and the web interface gave a “Server currently unavailable” message, with the graylog b/w background. I waited, but still the server did not become available. I looked at the rest of the message and tried to follow the clues, looking at the logs. Requests to 10.5.1.23:9000/api/ just timeout.

I was following the instructions on the graylog docs site for importing the OVA, which looked pretty simple and straightforward, but no go for me. So, I googled around, and found a recommendation to set the “rest_listen_uri” and “web_listen_uri” to the IP, rather than the 0.0.0.0 wildcard. So I set them both, replacing the 0.0.0.0, with 10.5.1.23. Now they look like this:
rest_listen_uri = http://10.5.1.23:9000/api
web_listen_uri = http://10.5.1.23:9000/

Now I get a perpetual “Graylog is restarting…” error at the web interface. At this point, I decided to wipe the VM, re-import and try again, just to see if I screwed something up. Same result.

Anyone else know of any issues with the 2.4.3 OVA, or know of something I’m missing?


(sirkus) #2

Okay, I tried again, looking for errors. I’m not sure if this is the correct place to post this kind of info, so please redirect my efforts if there is a more appropriate forum.

I wiped out the VM, re-imported again clean, fired it up, then just watched the logs. In the server/current log, I see the service on 0.0.0.0:9000 port firing up. Then I see an error related to an InputLauncher, “misfired. Reason: Address already in use.”

Given this is the stock OVA with nothing modified on it, is this config bug in the OVA, or have I missed some setup steps? Could this be related to the fact that requests to 9000 time out?

For reference, here is the relevant log entries I’m referring to, and the only error I can see on a clean boot up.

$ tail -50 /var/log/graylog/server/current:
    2018-04-09_16:34:27.26835 INFO  [NetworkListener] Started listener bound to [0.0.0.0:9000]
    2018-04-09_16:34:27.27436 INFO  [HttpServer] [HttpServer] Started.
    2018-04-09_16:34:27.27514 INFO  [JerseyService] Started REST API at <http://0.0.0.0:9000/api/>
    2018-04-09_16:34:27.27734 INFO  [JerseyService] Started Web Interface at <http://0.0.0.0:9000/>
    2018-04-09_16:34:27.28002 INFO  [ServerBootstrap] Services started, startup times in ms: {OutputSetupService [RUNNING]=61, BufferSynchronizerService [RUNNING]=67, KafkaJournal [RUNNING]=85, StreamCacheService [RUNNING]=299, JournalReader [RUNNING]=351, InputSetupService [RUNNING]=388, LookupTableService [RUNNING]=460, ConfigurationEtagService [RUNNING]=474, PeriodicalsService [RUNNING]=588, JerseyService [RUNNING]=24810}
    2018-04-09_16:34:27.28473 INFO  [ServerBootstrap] Graylog server up and running.
    2018-04-09_16:34:27.28557 INFO  [ServiceManagerListener] Services are healthy
    2018-04-09_16:34:27.35423 WARN  [NettyTransport] receiveBufferSize (SO_RCVBUF) for input SyslogUDPInput{title=appliance-syslog-udp, type=org.graylog2.inputs.syslog.udp.SyslogUDPInput, nodeId=1688517d-4c03-4d0a-8c76-da2468297ab0} should be 262144 but is 212992.
    2018-04-09_16:34:27.38783 INFO  [InputStateListener] Input [Syslog UDP/5acb95fc41a04705cf5c9d99] is now RUNNING
    2018-04-09_16:34:27.47013 ERROR [InputLauncher] The [org.graylog2.inputs.gelf.udp.GELFUDPInput] input with ID <5acb95fc41a04705cf5c9d9d> misfired. Reason: Address already in use.
    2018-04-09_16:34:27.47093 org.graylog2.plugin.inputs.MisfireException: org.graylog2.plugin.inputs.MisfireException: org.jboss.netty.channel.ChannelException: Failed to bind to: /0.0.0.0:12201
    2018-04-09_16:34:27.47836 	at org.graylog2.plugin.inputs.MessageInput.launch(MessageInput.java:158) ~[graylog.jar:?]
    2018-04-09_16:34:27.47839 	at org.graylog2.shared.inputs.InputLauncher$1.run(InputLauncher.java:84) [graylog.jar:?]
    2018-04-09_16:34:27.48035 	at com.codahale.metrics.InstrumentedExecutorService$InstrumentedRunnable.run(InstrumentedExecutorService.java:176) [graylog.jar:?]
    2018-04-09_16:34:27.48236 	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) [?:1.8.0_161]
    2018-04-09_16:34:27.48568 	at java.util.concurrent.FutureTask.run(FutureTask.java:266) [?:1.8.0_161]
    2018-04-09_16:34:27.50319 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:1.8.0_161]
    2018-04-09_16:34:27.50488 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0_161]
    2018-04-09_16:34:27.50825 	at java.lang.Thread.run(Thread.java:748) [?:1.8.0_161]
    2018-04-09_16:34:27.50914 Caused by: org.graylog2.plugin.inputs.MisfireException: org.jboss.netty.channel.ChannelException: Failed to bind to: /0.0.0.0:12201
    2018-04-09_16:34:27.51445 	at org.graylog2.plugin.inputs.transports.NettyTransport.launch(NettyTransport.java:155) ~[graylog.jar:?]
    2018-04-09_16:34:27.52049 	at org.graylog2.plugin.inputs.MessageInput.launch(MessageInput.java:155) ~[graylog.jar:?]
    2018-04-09_16:34:27.54420 	... 7 more
    2018-04-09_16:34:27.54530 Caused by: org.jboss.netty.channel.ChannelException: Failed to bind to: /0.0.0.0:12201
    2018-04-09_16:34:27.56102 	at org.jboss.netty.bootstrap.ConnectionlessBootstrap.bind(ConnectionlessBootstrap.java:204) ~[graylog.jar:?]
    2018-04-09_16:34:27.56158 	at org.graylog2.plugin.inputs.transports.NettyTransport.launch(NettyTransport.java:136) ~[graylog.jar:?]
    2018-04-09_16:34:27.57426 	at org.graylog2.plugin.inputs.MessageInput.launch(MessageInput.java:155) ~[graylog.jar:?]
    2018-04-09_16:34:27.57751 	... 7 more
    2018-04-09_16:34:27.57928 Caused by: java.net.BindException: Address already in use
    2018-04-09_16:34:27.59966 	at sun.nio.ch.Net.bind0(Native Method) ~[?:1.8.0_161]
    2018-04-09_16:34:27.60407 	at sun.nio.ch.Net.bind(Net.java:433) ~[?:1.8.0_161]
    2018-04-09_16:34:27.61371 	at sun.nio.ch.DatagramChannelImpl.bind(DatagramChannelImpl.java:691) ~[?:1.8.0_161]
    2018-04-09_16:34:27.62646 	at sun.nio.ch.DatagramSocketAdaptor.bind(DatagramSocketAdaptor.java:91) ~[?:1.8.0_161]
    2018-04-09_16:34:27.63064 	at org.jboss.netty.channel.socket.nio.NioDatagramPipelineSink.bind(NioDatagramPipelineSink.java:129) ~[graylog.jar:?]
    2018-04-09_16:34:27.64560 	at org.jboss.netty.channel.socket.nio.NioDatagramPipelineSink.eventSunk(NioDatagramPipelineSink.java:77) ~[graylog.jar:?]
    2018-04-09_16:34:27.65773 	at org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendDownstream(DefaultChannelPipeline.java:779) ~[graylog.jar:?]
    2018-04-09_16:34:27.66688 	at org.jboss.netty.channel.SimpleChannelHandler.bindRequested(SimpleChannelHandler.java:299) ~[graylog.jar:?]
    2018-04-09_16:34:27.67221 	at org.jboss.netty.channel.SimpleChannelHandler.handleDownstream(SimpleChannelHandler.java:265) ~[graylog.jar:?]
    2018-04-09_16:34:27.67679 	at org.jboss.netty.channel.DefaultChannelPipeline.sendDownstream(DefaultChannelPipeline.java:591) ~[graylog.jar:?]
    2018-04-09_16:34:27.68835 	at org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendDownstream(DefaultChannelPipeline.java:784) ~[graylog.jar:?]
    2018-04-09_16:34:27.69252 	at org.jboss.netty.channel.SimpleChannelHandler.bindRequested(SimpleChannelHandler.java:299) ~[graylog.jar:?]
    2018-04-09_16:34:27.69615 	at org.jboss.netty.channel.SimpleChannelHandler.handleDownstream(SimpleChannelHandler.java:265) ~[graylog.jar:?]
    2018-04-09_16:34:27.69632 	at org.jboss.netty.channel.DefaultChannelPipeline.sendDownstream(DefaultChannelPipeline.java:591) ~[graylog.jar:?]
    2018-04-09_16:34:27.69678 	at org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendDownstream(DefaultChannelPipeline.java:784) ~[graylog.jar:?]
    2018-04-09_16:34:27.69738 	at org.jboss.netty.channel.SimpleChannelHandler.bindRequested(SimpleChannelHandler.java:299) ~[graylog.jar:?]
    2018-04-09_16:34:27.69793 	at org.jboss.netty.channel.SimpleChannelHandler.handleDownstream(SimpleChannelHandler.java:265) ~[graylog.jar:?]
    2018-04-09_16:34:27.69876 	at org.jboss.netty.channel.DefaultChannelPipeline.sendDownstream(DefaultChannelPipeline.java:591) ~[graylog.jar:?]
    2018-04-09_16:34:27.69994 	at org.jboss.netty.channel.DefaultChannelPipeline.sendDownstream(DefaultChannelPipeline.java:582) ~[graylog.jar:?]
    2018-04-09_16:34:27.70073 	at org.jboss.netty.channel.Channels.bind(Channels.java:561) ~[graylog.jar:?]

(Jochen) #3

There’s another process already using the socket you’ve configured Graylog to use.

You can check this with the following commands:

# sudo lsof -i
# sudo netstat -tulpen

(sirkus) #4

Thanks for your response. Yeah, I saw that. The problem is that I didn’t configure anything on this VM. This is how it came “out of the box” straight from downloading and starting it up. So, my conclusion was that this problem is a misconfiguration in the OVA that anyone would have, through I haven’t seen any other posts indicating this. But if that’s true, then I have two questions:

  1. where do I report that?
  2. How does someone who is trying the OVA fix this so they can get to trying out graylog?

I’m new to graylog and the community, so I appreciate the direction. Thanks.


(Jochen) #5

I’m unable to reproduce your issue with the latest stable version of the Graylog OVA (graylog-2.4.3-1.ova).

Do you remember which commands you’ve executed on the OVA to configure it and what the output was?
Maybe you can try to reproduce that?

Also keep in mind, that you’re not supposed to modify the configuration files manually but always use the graylogctl utility.


(sirkus) #6

Interesting. I’m on a headless VM host, so imported it from the command line. I imported, assigned the bridged vm nic to eth1 on the host (standard for all our vms), then I started it.
Here are 3 commands and their output:

$ vboxmanage import graylog-2.4.3-1.ova --vsys 0 --vmname "graylog-2.4"
0%...10%...20%...30%...40%...50%...60%...70%...80%...90%...100%
Interpreting /home/virtualbox/virtual-appliances/graylog-2.4.3-1.ova...
OK.
Disks:  vmdisk1	20000	3033333760	http://www.vmware.com/interfaces/specifications/vmdk.html#streamOptimized	graylog-disk1.vmdk	1180584448	-1	
Virtual system 0:
 0: Suggested OS type: "Ubuntu_64"
    (change with "--vsys 0 --ostype <type>"; use "list ostypes" to list all possible values)
 1: VM name specified with --vmname: "graylog-2.4"
 2: Number of CPUs: 2
    (change with "--vsys 0 --cpus <n>")
 3: Guest memory: 4096 MB
    (change with "--vsys 0 --memory <MB>")
 4: Network adapter: orig NAT Network, config 3, extra type=Bridged
 5: IDE controller, type PIIX4
    (disable with "--vsys 0 --unit 5 --ignore")
 6: IDE controller, type PIIX4
    (disable with "--vsys 0 --unit 6 --ignore")
 7: Hard disk image: source image=graylog-disk1.vmdk, target path=/home/virtualbox/VirtualBox VMs/graylog_1/graylog-disk1.vmdk, controller=5;channel=0
    (change target path with "--vsys 0 --unit 7 --disk path";
    disable with "--vsys 0 --unit 7 --ignore")
0%...10%...20%...30%...40%...50%...60%...70%...80%...90%...100%
Successfully imported the appliance.

$ vboxmanage modifyvm graylog-2.4 --bridgeadapter1 eth1

$ vboxmanage startvm graylog-2.4 --type headless
Waiting for VM "graylog-2.4" to power on...
VM "graylog-2.4" has been successfully started.

After this, I wait for a few mins, then point my browser to the host, and I get:

GraylogServerUnavailable1

Again, the only error I see when logging into the console is the one that I shared above, though I may not be looking in all the right places.


(Jochen) #7

Please refer to the documentation about the basic setup of the virtual appliances:
http://docs.graylog.org/en/2.4/pages/installation/virtual_machine_appliances.html#basic-configuration


(sirkus) #8

Thanks, I had reviewed that and saw the “graylog-ctl” script info. I understood those to be optional, and I wasn’t sure which of the settings would ultimately be helpful. On my first import attempt, I did try using graylog-ctl to set the listening and external IPs – though that’s when I received the everlasting “restarting” message. At that point, I realized I don’t know enough about graylog as a system to know what to set, which is why I reached out for help.

Also, its strange to me that on a default startup of the vm (with no configuration) there is a conflicting port. Even more strange if that’s only when I import and run.

If you have any recommendations as to which settings would be relevant to my problem, I"ll certainly give it a shot.

Thanks for your help.


(Ofentse) #9

Looks familiar on 2.4.3. I however havent experienced this on 2.4.0


(sirkus) #10

Okay, well it seems I’m the only one experiencing this – which seems strange since I’m just running the downloaded OVA provided with no customization or changes to it. So, I guess there must be something different about how I’m doing it.

== Edit ==
I did just find the github graylog2-server issues site, so I’ll spend some time searching there as well.

Thanks!


(system) #11

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.