Bulk Data Removal Question


(CW) #1

I’m running Graylog 2.4.5 on a CentOS 6.9 server. I’ve gotten to the point where everything seems to be working fine. I spent time to standardize my Fields and also implement GROK patterns for Syslog messages that have various routers/switches sending in.

I would like to purge all previous message data and start fresh while obviously keeping all my extractors and configuration intact. Due to inexperience, I’m unsure and both uncomfortable with some of the options I’ve ready about in other posts using Delete By Queries or the API options.

What is the easiest way to accomplish my start a new on the message data goal?

Also, thought the graylog-ctl scripts were awesome in the OVA I used initially for some testing which everyone recommended against using for production…hence why I built my current system. Anyway to get the graylog-ctl scripts into my non-OVA base graylog build on CentOS6.9?

Thank you in advance for taking time to help me!


#2

Graylog uses Mongodb to store all the configuration and Elasticsearch to store all the logs. I haven’t tried this myself but you can gracefully stop graylog and delete all the indices in elasticsearch to get rid of all the past data.

I’ve done this when using ELK stack but never had to do with Graylog.


(Jochen) #3

You can delete old indices on the System/Indices/Index Set page to remove old messages.

No, the graylog-ctl script is heavily customized towards the OVA and won’t work on any other platform.


(CW) #4

First off, thanks to both of you for your replies. I just want to make solid sense of this, so I apologize in advance for questions that may seem obvious to you all but not necessarily to me yet.

How do I make the Indice old (I believe currently I only have the 1)?

Currently I only have a single Node and when I look under Indices & Index Sets I show only 1 index (Default index set). I do see under the “More Actions” drop down button there is a “Delete” option.

If I were to delete the “Default index set” does a brand new one get automatically created for me or do I need to do something additional?

Anything I need to do pre/post Deleting the Default index set?

Thank you!


(Jochen) #5

Delete an index, not the complete index set.

See http://docs.graylog.org/en/2.4/pages/configuration/index_model.html for detailed documentation.