Reports (Scheduled or On-Demand) Not Working Since 5.0 Upgrade

Graylog Central (peer support)
1

We were running the latest version of Graylog 4.x and scheduled reports worked without issue. We upgraded to version 5.0.2, and since the upgrade, reports both scheduled and by selecting “More Actions > Download Report Now” do not work. Selecting Download Report Now produces this error:

{"type":"ChromedriverError","message":"Error occurred during startup of chromedriver.\nStartup Logs: <EMPTY>\nRoot Cause: Could not start a new session. Possible causes are invalid address of the remote server or browser start-up failure. \nHost info: host: 'graylog.redacted.com', ip: '127.0.1.1'"}

I have tried creating a new report to see if there was an incompatibility in the way reports were previously configured, but that results in the same error/issue.

Ubuntu 20.04
Graylog Enterprise 5.0.2-1
MongoDB 5.0
ElasticSearch 7.10.2

/usr/share/graylog-server/bin$ ls -al
total 216596
drwxr-xr-x 3 graylog root       163 Jan 31 15:05 .
drwxr-xr-x 6 root    root       152 Jan 18 14:55 ..
-rwxr-xr-x 1 graylog root  18275816 Jan  4 14:52 chromedriver_amd64
-rwxr-xr-x 1 graylog root       152 Jan  4 14:52 chromedriver_start.sh
-rwxr-xr-x 1 graylog root       915 Jan  4 14:52 graylog-server
-rwxr-xr-x 1 graylog root 203509296 Jan  4 14:52 headless_shell_amd64
lrwxrwxrwx 1 graylog root        37 Jan 31 15:05 libEGL.so -> /usr/lib/x86_64-linux-gnu/libEGL.so.1
lrwxrwxrwx 1 graylog root        40 Jan 31 15:02 libGLESv2.so -> /usr/lib/x86_64-linux-gnu/libGLESv2.so.2
drwx------ 2 graylog root         6 Jan 31 14:37 locales

server.conf is also set with these two options (among others):

report_accept_insecure_certs = true
report_disable_sandbox = true
2

hey @gsgmc

This would be the second time i seen this post about reports not functioning after update.
Unlike the other post it was the size of the report that was unable to be sent.
What caught my attention was this

Possible causes are invalid address of the remote server or browser start-up failure.
\nHost info: host: ‘graylog.redacted.com’, ip: ‘127.0.1.1’"}

Is your loopback address actually 127.0.1.1 OR 127.0.0.1?

3

Thanks for responding @gsmith

The host is configured with a loopback of 127.0.1.1 to the FQDN, and a loopback of 127.0.0.1 to localhost.

4

can you create a report with one widget? so to generate the report is under 30s?
i must restart the graylog service after a generated report error to become the little test report.
the error is the same to send email or to download report.

i have the same error and changed the hosts file from 127.0.1.1 to 127.0.0.1 with the same error.

5

@halfmoon Now that is interesting. I followed your suggestion. I created a very small report with one widget. I tried to download it, and received the same error as listed above. Restarted the graylog service. Attempted to download the small report again. After a few moments, the new small report generated successfully.

Then, I attempted to download one of the reports that had worked previously, prior to the upgrade. It sat there for a number of seconds, and then produced this error message:

{"type":"ApiError","message":"Unable to render report"}

After that, another attempt to download a report produced the same error as I indicated in my first post:

{"type":"ChromedriverError","message":"Error occurred during startup of chromedriver.\nStartup Logs: <EMPTY>\nRoot Cause: Could not start a new session. Possible causes are invalid address of the remote server or browser start-up failure. \nHost info: host: 'graylog.redacted.com', ip: '127.0.1.1'"}
6

Hey,

After reading over this it seams , depending on the size of the report it wil time out , And it doesnt like 127.0.1.1. If there was success in a small report using 127.0.0.1, I would keep that address and work on timeout session, just an idea.

If you believe this is a bug you can post it here

7

Thanks, @gsmith - the interesting thing is that this configuration in our environment is not any different than it was when we were running version 4.x. All to say, I am not sure whether or not that would constitute a bug. If you advise that I submit this as a bug, I can.

For what it is worth, we have upgraded to 5.0.3 and the issue remains.

8

Hey,

Only thing I can think of after executing an upgrade would be permissions or sosmething might be install incorrect.

Im going to take a guess, After looking at you directory /usr/share/graylog-server/bin then comparing it to mine

image
image1066×727 22.3 KB

One of these things is not like the other :smiley:

I found out the packages to install eneterprise is a little different.

Opensource

sudo apt-get update && sudo apt-get install graylog-server

Enterprise

sudo apt-get update && sudo apt-get install graylog-enterprise

Perhaps roll back any configuration made and try reinstall graylog-enterprise .

` # sudo apt reinstall graylog-enterprise,

Make sure to copy any configuration made.`

image
image934×380 10.3 KB

9

@gsmith Thanks for the help! Before considering a roll-back/reinstall, do you know: is there a log somewhere that I may be overlooking which might give a little more insight to the problem? I appreciate it!

10

Update: We upgraded to the new 5.0.4 release today, and reporting now works! In reviewing the change log, nothing seems to directly relate to reporting. However, all of our reports work again, and are functional.

1 Like
11

If anyone is still having problems with the schedule or the report is not generating, try using the system cron to send the created reports.
I created a bash script to call the Graylog Rest API using curl:

curl -u user:password http://ipaddress:9000/api/plugins/org.graylog.plugins.report/reports/<report_id_no>/email
sleep 160  #pause between report generation

image
image970×405 28.1 KB

1 Like
12

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.