[Graylog3] Report generation error

Graylog Central (peer support)
1

Hi,

I try to generate reports but I’ve an error :

2019-04-29T15:14:22.719+02:00 INFO  [BrowserDriverService] Starting report render engine: http://localhost:9515
2019-04-29T15:14:42.748+02:00 ERROR [OsProcess] org.apache.commons.exec.ExecuteException: Process exited with an error: 127 (Exit value: 127)
2019-04-29T15:14:42.836+02:00 ERROR [AnyExceptionClassMapper] Unhandled exception in REST resource
org.openqa.selenium.WebDriverException: Timed out waiting for driver server to start.
Build info: version: 'unknown', revision: 'unknown', time: 'unknown'
System info: host: 'Unknown', ip: 'Unknown', os.name: 'Linux', os.arch: 'amd64', os.version: '3.10.0-957.10.1.el7.x86_64', java.version: '1.8.0_201'
Driver info: driver.version: unknown
        at org.openqa.selenium.remote.service.DriverService.waitUntilAvailable(DriverService.java:202) ~[?:?]
        at org.openqa.selenium.remote.service.DriverService.start(DriverService.java:188) ~[?:?]
        at org.graylog.plugins.report.render.BrowserDriverService.start(BrowserDriverService.java:98) ~[?:?]
        at org.graylog.plugins.report.render.RemoteBrowserService.connectDriver(RemoteBrowserService.java:100) ~[?:?]
        at org.graylog.plugins.report.render.RemoteBrowserService.connectDriver(RemoteBrowserService.java:87) ~[?:?]
        at org.graylog.plugins.report.api.ReportService.fetchPdf(ReportService.java:236) ~[?:?]
        at org.graylog.plugins.report.rest.ReportResource.generateReport(ReportResource.java:156) ~[?:?]
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:1.8.0_201]
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:1.8.0_201]
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_201]
        at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_201]
        at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory$1.invoke(ResourceMethodInvocationHandlerFactory.java:81) ~[graylog.jar:?]
        at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:144) ~[graylog.jar:?]
        at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:161) ~[graylog.jar:?]
        at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:160) ~[graylog.jar:?]
        at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:99) ~[graylog.jar:?]
        at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:389) ~[graylog.jar:?]
        at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:347) ~[graylog.jar:?]
        at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:102) ~[graylog.jar:?]
        at org.glassfish.jersey.server.ServerRuntime$2.run(ServerRuntime.java:326) [graylog.jar:?]
        at org.glassfish.jersey.internal.Errors$1.call(Errors.java:271) [graylog.jar:?]
        at org.glassfish.jersey.internal.Errors$1.call(Errors.java:267) [graylog.jar:?]
        at org.glassfish.jersey.internal.Errors.process(Errors.java:315) [graylog.jar:?]
        at org.glassfish.jersey.internal.Errors.process(Errors.java:297) [graylog.jar:?]
        at org.glassfish.jersey.internal.Errors.process(Errors.java:267) [graylog.jar:?]
        at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:317) [graylog.jar:?]
        at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:305) [graylog.jar:?]
        at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:1154) [graylog.jar:?]
        at org.glassfish.jersey.grizzly2.httpserver.GrizzlyHttpContainer.service(GrizzlyHttpContainer.java:384) [graylog.jar:?]
        at org.glassfish.grizzly.http.server.HttpHandler$1.run(HttpHandler.java:224) [graylog.jar:?]
        at com.codahale.metrics.InstrumentedExecutorService$InstrumentedRunnable.run(InstrumentedExecutorService.java:181) [graylog.jar:?]
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:1.8.0_201]
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0_201]
        at java.lang.Thread.run(Thread.java:748) [?:1.8.0_201]
Caused by: org.openqa.selenium.net.UrlChecker$TimeoutException: Timed out waiting for [http://localhost:9515/status] to be available after 20004 ms
        at org.openqa.selenium.net.UrlChecker.waitUntilAvailable(UrlChecker.java:100) ~[?:?]
        at org.openqa.selenium.remote.service.DriverService.waitUntilAvailable(DriverService.java:197) ~[?:?]
        ... 33 more
Caused by: java.util.concurrent.TimeoutException
        at java.util.concurrent.FutureTask.get(FutureTask.java:205) ~[?:1.8.0_201]
        at com.google.common.util.concurrent.SimpleTimeLimiter.callWithTimeout(SimpleTimeLimiter.java:156) ~[graylog.jar:?]
        at org.openqa.selenium.net.UrlChecker.waitUntilAvailable(UrlChecker.java:75) ~[?:?]
        at org.openqa.selenium.remote.service.DriverService.waitUntilAvailable(DriverService.java:197) ~[?:?]

The /usr/share/graylog-server/ directory have graylog:root permission.

The server.conf configuration file have the new bin_dir parameter.

In the bin folder, I’ve the chromedriver files:

$ sudo ls /usr/share/graylog-server/bin/
chromedriver  chromedriver_start.sh  graylog-server  headless_shell

An idea ?

Thanks

2

what is the output when you run the headless_shell ?

that should indicate what the problem is.

3

@jan

The output was

# /usr/share/graylog-server/bin/headless_shell
[0429/154229.746696:WARNING:resource_bundle.cc(358)] locale_file_path.empty() for locale
[0429/154229.757148:ERROR:zygote_host_impl_linux.cc(89)] Running as root without --no-sandbox is not supported. See https://crbug.com/638180.

With the --no-sandbox option:

[0429/154957.187987:WARNING:resource_bundle.cc(358)] locale_file_path.empty() for locale
Fontconfig error: Cannot load default config file
[0429/154957.198826:ERROR:gpu_process_transport_factory.cc(967)] Lost UI shared context.
[0429/154957.217600:FATAL:platform_font_linux.cc(83)] Check failed: InitDefaultFont(). Could not find the default font
#0 0x000003efdfdf <unknown>
#1 0x000003e81a50 <unknown>
#2 0x000004a5c3b4 <unknown>
#3 0x000004a5d2cb <unknown>
#4 0x000004a579be <unknown>
#5 0x00000323ddc2 <unknown>
#6 0x000003349caf <unknown>
#7 0x0000030d948e <unknown>
#8 0x0000030d8e47 <unknown>
#9 0x0000030d89e2 <unknown>
#10 0x0000030944d9 <unknown>
#11 0x0000030b2b23 <unknown>
#12 0x00000309938e <unknown>
#13 0x000003098e18 <unknown>
#14 0x000003f1bbff <unknown>
#15 0x000003f1f249 <unknown>
#16 0x000003e66977 <unknown>
#17 0x000002e7353f <unknown>
#18 0x0000025e3887 <unknown>
#19 0x000003008ade <unknown>
#20 0x000003ed601a <unknown>
#21 0x000003e87711 <unknown>
#22 0x000003e8697f <unknown>
#23 0x000003e86d02 <unknown>
#24 0x000003f10c09 <unknown>
#25 0x000003ea0415 <unknown>
#26 0x000002f79004 <unknown>
#27 0x000002f78dc3 <unknown>
#28 0x000002f7b502 <unknown>
#29 0x000003f2201f <unknown>
#30 0x000003e345e6 <unknown>
#31 0x0000051645bb <unknown>
#32 0x000003e2d3a1 <unknown>
#33 0x000003f20ee8 <unknown>
#34 0x000003f20f75 <unknown>
#35 0x000003e6aa72 <unknown>
#36 0x7fa773bea3d5 __libc_start_main
#37 0x00000259202a _start

Received signal 6
#0 0x000003efdfdf <unknown>
#1 0x000003efdb51 <unknown>
#2 0x7fa77528e5d0 <unknown>
#3 0x7fa773bfe207 __GI_raise
#4[0429/154957.269090:WARNING:resource_bundle.cc(358)] locale_file_path.empty() for locale
[0429/154957.289165:ERROR:egl_util.cc(59)] Failed to load GLES library: /usr/share/graylog-server/bin/swiftshader/libGLESv2.so: Ne peut ouvrir le fichier d'objet partagé: Aucun fichier ou dossier de ce type
 0x7fa773bff8f8 __GI_abort
#5 0x000003efc985 <unknown>
#6 0x000003e81ec2 <unknown>
#7 0x000004a5c3b4 <unknown>
#8 0x000004a5d2cb <unknown>
#9 0x000004a579be <unknown>
#10 0x00000323ddc2 <unknown>
#11 0x000003349caf <unknown>
#12 0x0000030d948e <unknown>
#13 0x0000030d8e47 <unknown>
#14 0x0000030d89e2 <unknown>
#15 0x0000030944d9 <unknown>
#16 0x0000030b2b23 [0429/154957.310667:ERROR:viz_main_impl.cc(184)] Exiting GPU process due to errors during initialization
<unknown>
#17 0x00000309938e <unknown>
#18 0x000003098e18 <unknown>
#19 0x000003f1bbff <unknown>
#20 0x000003f1f249 <unknown>
#21 0x000003e66977 <unknown>
#22 0x000002e7353f <unknown>
#23 0x0000025e3887 <unknown>
#24 0x000003008ade <unknown>
#25 0x000003ed601a <unknown>
#26 0x000003e87711 <unknown>
#27 0x000003e8697f <unknown>
#28 0x000003e86d02 <unknown>
#29 0x[0429/154957.333158:WARNING:resource_bundle.cc(358)] locale_file_path.empty() for locale
[0429/154957.334327:ERROR:egl_util.cc(59)] Failed to load GLES library: /usr/share/graylog-server/bin/swiftshader/libGLESv2.so: Ne peut ouvrir le fichier d'objet partagé: Aucun fichier ou dossier de ce type
000003f10c09 <unknown>
#30 0x000003ea0415 <unknown>
#31 0x000002f79004 <unknown>
#32 0x000002f78dc3 <unknown>
#33 0x000002f7b502 <unknown>
#34 0x000003f2201f <unknown>
#35 0x000003e345e6 <unknown>
#36 0x0000051645bb <unknown>
#37 0x000003e2d3a1 <unknown>
#38 0x000003f20ee8 <unknown>
#39 0x000003f20f75 <unknown>
#40 0x000003e6aa72 <unknown>
#41 0x7fa773bea3d5 __libc_start_main
#42 0x00000259202a _start
  r8: 00007fa7758b07c0  r9: 0000000000000030 r10: 0000000000000008 r11: 0000000000000206
 r12: 00007ffd500ffc78 r13: 0000000000000076 r14: 00007ffd500ffc80 r15: 00007ffd500ffc88
  di: 0000000000003ef6  si: 0000000000003ef6  bp: 00007ffd500ff2d0  bx: [0429/154957.359817:ERROR:viz_main_impl.cc(184)] Exiting GPU process due to errors during initialization
00007ffd500ff330
  dx: 0000000000000006  ax: 0000000000000000  cx: ffffffffffffffff  sp: 00007ffd500ff198
  ip: 00007fa773bfe207 efl: 0000000000000206 cgf: 0000000000000033 erf: 0000000000000000
 trp: 0000000000000000 msk: 0000000000000000 cr2: 0000000000000000
[end of stack trace]
Calling _exit(1). Core file will not be generated.

The data_dir configuration:

# cat /etc/graylog/server/server.conf | grep data_dir
data_dir = /var/lib/graylog-server
# ls -l /var/lib/graylog-server
total 0
drwxr-xr-x. 3 graylog graylog 121 29 avril 15:51 journal

EDIT :

I add in the server.conf configuration file :

report_disable_sandbox = true

If I installed chromium-libs-67.0.3396.79-1.el7.x86_64, I’ve this output :

[0429/160355.039261:WARNING:resource_bundle.cc(358)] locale_file_path.empty() for locale
[0429/160355.055469:WARNING:resource_bundle.cc(358)] locale_file_path.empty() for locale
[0429/160355.081604:ERROR:gpu_process_transport_factory.cc(967)] Lost UI shared context.
[0429/160355.117098:WARNING:resource_bundle.cc(358)] locale_file_path.empty() for locale
[0429/160355.118515:ERROR:egl_util.cc(59)] Failed to load GLES library: /usr/share/graylog-server/bin/swiftshader/libGLESv2.so: Ne peut ouvrir le fichier d'objet partagé: Aucun fichier ou dossier de ce type
[0429/160355.137810:ERROR:viz_main_impl.cc(184)] Exiting GPU process due to errors during initialization
[0429/160355.147663:WARNING:resource_bundle.cc(358)] locale_file_path.empty() for locale
[0429/160355.148979:ERROR:egl_util.cc(59)] Failed to load GLES library: /usr/share/graylog-server/bin/swiftshader/libGLESv2.so: Ne peut ouvrir le fichier d'objet partagé: Aucun fichier ou dossier de ce type
[0429/160355.153488:ERROR:viz_main_impl.cc(184)] Exiting GPU process due to errors during initialization

But in this topic : Report Generation Error - #11 by marius, @marius said :

“Cool that it is working now for you! We will add more details to the documentation soon. The error regarding libGLES could be ignored, since in headless_mode no graphic acceleration is needed anyways.”

After this, the report generation works.

 2019-04-29T16:07:23.124+02:00 INFO  [BrowserDriverService] Starting report render engine: http://localhost:9515
2019-04-29T16:07:24.355+02:00 INFO  [ProtocolHandshake] Detected dialect: OSS
2019-04-29T16:08:06.321+02:00 INFO  [connection] Opened connection [connectionId{localValue:12, serverValue:179}] to <IP>:27017
2019-04-29T16:08:07.875+02:00 WARN  [RemoteBrowserService] Chrome sandboxing is currently disabled. Please validate your security settings!

EDIT 2:

Without the chromium libs package, the report generation works but the logs are differents:

2019-04-29T16:13:24.711+02:00 INFO  [BrowserDriverService] Starting report render engine: http://localhost:9515
2019-04-29T16:13:25.723+02:00 INFO  [ProtocolHandshake] Detected dialect: OSS
2019-04-29T16:14:13.556+02:00 WARN  [RemoteBrowserService] Chrome sandboxing is currently disabled. Please validate your security settings!
2019-04-29T16:14:14.502+02:00 WARN  [ProxiedResource] Unable to call http://<IP>:9000/api/system/metrics/multiple on node <-51e6-4d51-b455->, result: Unauthorized
2019-04-29T16:14:14.521+02:00 WARN  [ProxiedResource] Unable to call http://<IP>:9000/api/system/metrics/multiple on node <-b57d-4259-bedc->, result: Unauthorized
2019-04-29T16:14:14.522+02:00 WARN  [ProxiedResource] Unable to call http://<IP>:9000/api/system/metrics/multiple on node <-9580-4c82-836d->, result: Unauthorized
4

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.