Graylog - Unauthorized errors in the log

Graylog Central (peer support)
1

Hi,

I have 3 Graylog nodes running behind an Oauth2 proxy and a Loadbalancer. My setup is like

user -->> LB → Nginx -->> Oauth2_proxy —>> Graylog Clsuter.

I get lots of following warning in the log

Oct 07 20:10:00 node01.graylog. docker[2616115]: 2020-10-07 18:10:00,421 WARN : org.graylog2.shared.rest.resources.ProxiedResource - Unable to call http://private-ip-of-the-vm:9000/api/system on node , result: Unauthorized
Oct 07 20:10:02 node01.graylog. docker[2616115]: 2020-10-07 18:10:02,151 WARN : org.graylog2.shared.rest.resources.ProxiedResource - Unable to call http://private-ip-of-the-vm:9000/api/system/metrics/multiple on node , result: Unauthorized

I am running Graylog version 3.3.6 in Docker with the following config

GRAYLOG_IS_MASTER=true 
GRAYLOG_HTTP_EXTERNAL_URI=https://public-url-pointing-LB/ 
GRAYLOG_ROOT_PASSWORD_SHA2=************ 
GRAYLOG_PASSWORD_SECRET=************** 
GRAYLOG_ELASTICSEARCH_HOSTS=http://private-ip-of-the-vm:9200/ 
GRAYLOG_MONGODB_URI=mongodb://********************27017/graylog?replicaSet=graylog-rs0
GRAYLOG_WEB_ENABLE=true
GRAYLOG_OUTPUT_BATCH_SIZE=5000
GRAYLOG_TRUSTED_PROXIES=\"127.0.0.1/32, 172.17.0.0/16, 10.0.0.0/16\" \
GRAYLOG_SERVER_JAVA_OPTS=\"-XX:+UnlockExperimentalVMOptions -XX:+UseCGroupMemoryLimitForHeap -XX:NewRatio=1 -XX:MaxMetaspaceSize=256m -server -XX:+ResizeTLAB -XX:+UseConcMarkSweepGC -XX:+CMSConcurrentMTEnabled -XX:+CMSClassUnloadingEnabled -XX:+UseParNewGC -XX:-OmitStackTraceInFastThrow -Xms6g -Xmx6g\""

I am also getting the following error in the web interface while accessing thee nodes and all.

Could not get JVM information
Getting JVM information for node ‘d686b638-3175-4d61-b529-6d4e0025bca1’ failed: Error: cannot GET https://Public-URL/api/cluster/d686b638-3175-4d61-b529-6d4e0025bca1/jvm (502)

If I try curl from the host machine, it works

curl -i “http://10.10.0.14:9000/api/?pretty=true”
HTTP/1.1 200 OK
X-Graylog-Node-ID: d686b638-3175-4d61-b529-6d4e0025bca1
X-Runtime-Microseconds: 3573
Content-Type: application/json
Date: Wed, 07 Oct 2020 18:23:54 GMT
Content-Length: 253
{
“cluster_id” : “73031db4-cf74-4e2d-a7bd-126f923dc3cf”,
“node_id” : “d686b638-3175-4d61-b529-6d4e0025bca1”,
“version” : “3.3.6+92fb41e”,
“tagline” : “Manage your logs in the dark and have lasers going and make it look like you’re from space!”
}

I am running Graylog with almost same settings (no proxy and Oauth2_proxy) in k8s without any issues.

Any help would be highly appreciated.

Thanks,
Arun Sasidharan

1 Like
2

Check your Configure Provider Order
in System / Authentication / Configure Provider Order

3

I have following order

image

4

Try to elaborate with order, if it helps, it’s only guess.

5

Sorry @shoothub, I didn’t understand your last comment.

I changed the Authentication provider order to enable SSO with Azure AD. I am using SSO plugin version 3.3.0.

6

I mean to change order of providers, maybe it helps.

7

I tried the following order of providers. But still the same error.

Try #1
image

Try #2
image

8

Hello there,

Any suggestions?

Thanks in advance,
Arun

9

Hey Arun, I’m little new to K8s env. is possible to share your the steps to run your setup with Oauth2_proxy? Thanks

10

Hello
Any inputs for the issue raised here?

11

I am still facing the issue.
Any help would be highly appreciated.

Thanks,
Arun

12

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.