I’m trying to generate a report using the Enterprise 3.0 feature set. I get the following error from the UI (IPs removed):
{"type":"ApiError","message":"Timed out waiting for driver server to start.\nBuild info: version: 'unknown', revision: 'unknown', time: 'unknown'\nSystem info: host: '<HOST HERE REMOVED>', ip: '<IP HERE REMOVED>', os.name: 'Linux', os.arch: 'amd64', os.version: '3.10.0-957.5.1.el7.x86_64', java.version: '1.8.0_191'\nDriver info: driver.version: unknown"}
I see the following error message in /var/log/graylog/server.log:
2019-02-15T11:50:12.632Z ERROR [OsProcess] org.apache.commons.exec.ExecuteException: Process exited with an error: 127 (Exit value: 127)
2019-02-15T11:50:12.745Z ERROR [AnyExceptionClassMapper] Unhandled exception in REST resource
org.openqa.selenium.WebDriverException: Timed out waiting for driver server to start.
Build info: version: 'unknown', revision: 'unknown', time: 'unknown'
System info: host: '<HOST REMOVED>', ip: '<IP REMOVED>', os.name: 'Linux', os.arch: 'amd64', os.version: '3.10.0-957.5.1.el7.x86_64', java.version: '1.8.0_191'
Driver info: driver.version: unknown
at org.openqa.selenium.remote.service.DriverService.waitUntilAvailable(DriverService.java:202) ~[?:?]
at org.openqa.selenium.remote.service.DriverService.start(DriverService.java:188) ~[?:?]
at org.graylog.plugins.report.render.RemoteBrowserService.connectDriver(RemoteBrowserService.java:135) ~[?:?]
at org.graylog.plugins.report.render.RemoteBrowserService.connectDriver(RemoteBrowserService.java:111) ~[?:?]
at org.graylog.plugins.report.api.ReportService.fetchPdf(ReportService.java:259) ~[?:?]
at org.graylog.plugins.report.rest.ReportResource.generateReport(ReportResource.java:156) ~[?:?]
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:1.8.0_191]
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:1.8.0_191]
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_191]
at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_191]
at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory$1.invoke(ResourceMethodInvocationHandlerFactory.java:81) ~[graylog.jar:?]
at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:144) ~[graylog.jar:?]
at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:161) ~[graylog.jar:?]
at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:160) ~[graylog.jar:?]
at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:99) ~[graylog.jar:?]
at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:389) ~[graylog.jar:?]
at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:347) ~[graylog.jar:?]
at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:102) ~[graylog.jar:?]
at org.glassfish.jersey.server.ServerRuntime$2.run(ServerRuntime.java:326) [graylog.jar:?]
at org.glassfish.jersey.internal.Errors$1.call(Errors.java:271) [graylog.jar:?]
at org.glassfish.jersey.internal.Errors$1.call(Errors.java:267) [graylog.jar:?]
at org.glassfish.jersey.internal.Errors.process(Errors.java:315) [graylog.jar:?]
at org.glassfish.jersey.internal.Errors.process(Errors.java:297) [graylog.jar:?]
at org.glassfish.jersey.internal.Errors.process(Errors.java:267) [graylog.jar:?]
at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:317) [graylog.jar:?]
at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:305) [graylog.jar:?]
at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:1154) [graylog.jar:?]
at org.glassfish.jersey.grizzly2.httpserver.GrizzlyHttpContainer.service(GrizzlyHttpContainer.java:384) [graylog.jar:?]
at org.glassfish.grizzly.http.server.HttpHandler$1.run(HttpHandler.java:224) [graylog.jar:?]
at com.codahale.metrics.InstrumentedExecutorService$InstrumentedRunnable.run(InstrumentedExecutorService.java:181) [graylog.jar:?]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:1.8.0_191]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0_191]
at java.lang.Thread.run(Thread.java:748) [?:1.8.0_191]
Caused by: org.openqa.selenium.net.UrlChecker$TimeoutException: Timed out waiting for [http://localhost:9515/status] to be available after 20000 ms
at org.openqa.selenium.net.UrlChecker.waitUntilAvailable(UrlChecker.java:100) ~[?:?]
at org.openqa.selenium.remote.service.DriverService.waitUntilAvailable(DriverService.java:197) ~[?:?]
... 32 more
Caused by: java.util.concurrent.TimeoutException
at java.util.concurrent.FutureTask.get(FutureTask.java:205) ~[?:1.8.0_191]
at com.google.common.util.concurrent.SimpleTimeLimiter.callWithTimeout(SimpleTimeLimiter.java:156) ~[graylog.jar:?]
at org.openqa.selenium.net.UrlChecker.waitUntilAvailable(UrlChecker.java:75) ~[?:?]
at org.openqa.selenium.remote.service.DriverService.waitUntilAvailable(DriverService.java:197) ~[?:?]
... 32 more
Note: If I monitor the system for port 9515 – I never see a listener brought up for the report generator.
Please use proper formatting - without that your posting is nearly impossible. So quick, edit your posting and surround the log message with ``` like described in the FAQ:
I installed libX11 (yum install libX11) which resolved the chromedriver execution issue. However, here is the new error message I receive when trying to invoke a report from the UI:
{"type":"ApiError","message":"unknown error: Chrome failed to start: exited abnormally\n (unknown error: DevToolsActivePort file doesn't exist)\n (The process started from chrome location /usr/share/graylog-server/bin/headless_shell is no longer running, so ChromeDriver is assuming that Chrome has crashed.)\n (Driver info: chromedriver=72.0.3583.1 (9ae827a6f6bbc9186d4e79681de885ce77d5bfe9),platform=Linux 3.10.0-957.5.1.el7.x86_64 x86_64) (WARNING: The server did not provide any stacktrace information)\nCommand duration or timeout: 67 milliseconds\nBuild info: version: 'unknown', revision: 'unknown', time: 'unknown'\nSystem info: host: '<HOST>', ip: '<IP>', os.name: 'Linux', os.arch: 'amd64', os.version: '3.10.0-957.5.1.el7.x86_64', java.version: '1.8.0_191'\nDriver info: driver.version: RemoteWebDriver"}
@unixfun looks like you are running a RedHat/CentOS system with a pretty old kernel. Either update to a 4.x kernel or disable reporting sandbox with: report_disable_sandbox = true
In the server configuration file.
Interesting note is that I used the default AWS 7.5 Centos Base image and it has a 3.x kernel. I upgraded the kernel to 4.2 and still received an error with headless_shell looking for libGLEXv2.so I installed chromium-libs-71.0.3578.98 after running yum whatprovides '*/libGLESv2.so*' and restarted graylog. With the addition of this package, I was able to successfully create a report.
I suggest additional Linux requirements be added to the documentation for future users. Thanks for the help!
Cool that it is working now for you! We will add more details to the documentation soon. The error regarding libGLES could be ignored, since in headless_mode no graphic acceleration is needed anyways.
