Delete messages from Graylog?

Graylog Central (peer support)
1

I’m setting up my graylog instance and I’m trying to get all my data normalized correctly. I’ve got everything normalized as I want, but I’d like to be able to remove specific messages from ElasticSearch.

I’ve been trying curl -XDELETE ‘http://localhost:9200/graylog_*/message/_query’ -d ‘{“query_string”:{“query”:“Classification”:"{}"}}’

Classification is a field I extracted out in the messages. I’m wanting to delete the messages that have the classification field, and the value in it is {}

The command works, but it returns that it didn’t find anything.
I also tried pasting the Webgui query for the messages I wanted to delete, and it results the same.

2

Does the field have to have that value, or just the presence of the field?

3

Did you install the Delete-by-query plugin before using that command?

4

Just to add to what jochen mentioned, I needed to set write blocks to “false” on the given indices first, remove offending message(s), then re-enabled write block, because Graylog flushes and sets a given index to read-only after cycling the deflector (at your designated interval).

curl -XPUT ‘localhost:9200/graylog_0/_settings’ -d ‘{“index” : {“blocks”: {“write”: false}}}’

then

curl -XPUT ‘localhost:9200/graylog_0/_settings’ -d ‘{“index” : {“blocks”: {“write”: true}}}’

5

Didn’t realize it was a plugin. I tried installing it, but it’s saying that the bin/plugin command isn’t found. I suppose I have to install the plugin manually?

6

Did you execute it from within your Elasticsearch installation folder? bin/plugin is a relative path. It’s the plugin executable in the bin folder of Elasticsearch.

8

I found it. I now realize what you meant. /bin/plugin being the path to the script.
Thanks for the help!

9

Thanks for mentioning this. I remember reading that the old indices are set as read-only.

My last issue is my query structure. What I posted I was using is still failing.

curl -XDELETE ‘http://localhost:9200/graylog_4/message/_query’ -d ‘{“query_string”:{“query”:“Classification”:"{}"}}’
OR
curl -XDELETE ‘http://localhost:9200/graylog_4/message/_query’ -d ‘{“query”:{“query_string”:{“query”:“Classification”:"{}"}}}’

It’s still returning false. I’m trying to delete all messages where the Classification field contains “{}” (no quotes).

10

Something else I found helpful was installing the Sense app in Kibana and doing this work there rather than via CURL. Not that curl won’t work, but it was easier for me via Sense.

11

Does graylog use Kibana? I was trying to get this up while installing as few external components as possible.

Correct me if I’m wrong, but isn’t Kibana part of the ELK stack? Which begs the question, if I’m going to use Kibana, why use Graylog? Not trying to sound rude, actually asking because I don’t understand.

12

Valid question. Graylog has a different philosophy than ELK so I suggest you try out both, especially at scale/production load and then choose the tool you like the most.

13

Fair enough. I may have to do that. But for now, I have to stick with Graylog and just try to get this query working.

Thanks for all the help!

14

Can you get the query returning the results you want within Graylog? If so, you can go to MORE ACTIONS → SHOW Query and then use that as the query field in your CURL command.

15

I can. I have tried this and the results are the same.

EDIT: for clarification, the response I get for the queries I’ve run is:

{“found”:false,“_index”:“graylog_5”,“_type”:“message”,“_id”:“_query”,“_version”:1,“_shards”:{“total”:1,“successful”:1,“failed”:0}}

16

I think the index name should be graylog_5 instead of graylog5.