Multiple errors trying to access OpenSearch node

[TheWalrus]

Hi, I’ve been looking forever and I can’t seem to find someone who has had this exact problem. I’m running Graylog 6.0 on a Ubuntu server (24.04), and having it access an OpenSearch node which is running on a second Ubuntu server.

Everything works fine for the most part, I’m able to send logs to Graylog and they get stored in my Opensearch node, however when I go to the “Nodes” tab of the web interface, it says “System information is currently unavailable.”. If I try to click on the node, I get shown the image of the funny monkey and the following error messages:

Could not get plugins
Getting plugins on node “d20ae0a7-3ce9-4f70-8048-7d86dc4d4b73” failed: FetchError: There was an error fetching a resource: Bad Gateway. Additional information: Forbidden

Could not get JVM information
Getting JVM information for node ‘d20ae0a7-3ce9-4f70-8048-7d86dc4d4b73’ failed: FetchError: There was an error fetching a resource: Bad Gateway. Additional information: Forbidden

When I tail the server.log file of the Graylog server, I get the following warnings:

2024-06-25T13:18:58.366-04:00 WARN [ClusterSystemResource] Unable to get plugin list on node d20ae0a7-3ce9-4f70-8048-7d86dc4d4b73: Forbidden

2024-06-25T13:18:58.366-04:00 WARN [ClusterSystemResource] Unable to get jvm information on node d20ae0a7-3ce9-4f70-8048-7d86dc4d4b73: Forbidden

2024-06-25T13:18:58.366-04:00 WARN [ProxiedResource] Failed to call API on node , cause: null (duration: 26 ms)

There is a proxy set up so that might be causing errors, but excluded every relevant address from the proxy.

[gsmith]

Hey @TheWalrus

Judging from the error “bad gateway”, I would look into the connection configuration form Graylog to OS and/or network configuration. On a side note make sure you the correct ports open or needed since these two application are not on the same node.

Are you using HTTP or HTTPs between Graylog and OpenSearch?
Did you check OpenSearch and MongoDb logs?

Can you show your GL configuration file, perhaps grep it? Ensure you remove personal info

[TheWalrus]

Thanks for your response, it turns out I didn’t add Graylog’s ip to http_non_proxy_hosts, I didn’t think I needed to but it makes sense in retrospect. Your response indirectly led me to finding the solution, so thank you!