Graylog-datanode curl commands

Graylog Central (peer support)
, ,
1

1. Describe your incident:
I’m using the Rotation & Retention legacy option, to close indexes and not delete them, it is working as expected.
When I was using opensearch, I used to delete closed indexes after sometime with a CURL script:

#!/bin/bash
for counter in {65..96}; do  curl -X DELETE http://localhost:9200/ipfix_$counter; done

But this script doesn’t work anymore, I’m getting the following error:

curl: (52) Empty reply from server

Question: Is there a way to use CURL to delete my indexes as mentioned above ?

2. Describe your environment:

  • OS Information: Raspberry PI 5 8GB running Raspbian OS 64 bits (Debian 12)

  • Package Version: graylog-server:arm64 (6.1.1-1), graylog-datanode:arm64 (6.1.1-1)

3. What steps have you already taken to try and solve the problem?
Tried to open the page on port 9200 and got :

Authentication finally failed

Tried to use the credentials on the login: https://admin:mypassword@hostname:9200 and got the same error.

4. How can the community help?
I would like to ask if what I’m trying to do is possible or not.
I thought that graylog-datanode would be similar to opensearch/elasticsearch in that regard.

Note that I wasn’t just using the CURL -X delete option, I was using other opensearch CURL options, such as for backup purposes.

Thanks.

2

Hi,
You have two options. Either you can use the built-in proxy in the graylog server to forward authenticated requests directly to the underlying opensearch. This will add the needed JWT auth header for you. The URL format is /api/datanodes/{hostname}/opensearch/{path: .*}

for example http://graylog-server-host:port/api/datanodes/any/opensearch/_cat/indices?h=index,status

The {hostname} part can be used to target a specific datanode/opensearch instance. The any keyword will forward your request to a random connected opensearch.

By default the proxy is limited to a few read-only opensearch endpoints. You’ll need to disable the allowlist if you want to delete indices. The graylog-server setting is called datanode_proxy_api_allowlist. Set it to false if you want to disable it.

You’ll also need your graylog auth credentials provided as basic auth header in the request.

The other option is to generate client certificates which you can use to communicate directly. They can be configured and downloaded in the System->Datanodes->Configuration menu.

Best regards,
Tomas

1 Like
3

Hello Tomas, thanks for answering.

After following your instructions, I was able to delete indices through the graylog: DataNodes/API : Proxy direct access to Data Node’s API

In addition to that, I was also able to use CURL command to communicate directly with the following command:

curl -X DELETE --key key_datanode.crt --cert cert_datanode.crt --cacert ca_datanode.crt "https://<redacted>:9200/unifi_1" --pass <redacted>
{"acknowledged":true}

Much appreciated for your time and support on this.

1 Like
4

Hey, I am glad I could help. Nice to hear that both options are working for you. If you are going to use the certificates method then keep in mind that these will expire sooner or later and you’ll need to renew them.

1 Like
5

Thanks for the info, I’ll change the auto renew from 30 days to 2 years also.
Just generated new cert with those settings.

I’ll also include that date on my calendar :+1:

1 Like