How can I remove the default index set and All messages stream

[joseraeiro]

1. Describe your incident:

I have an All messages stream and a Default index set which I would like to remove from Graylog. What would be the best way to accomplish this?

2. Describe your environment:

• OS Information:
  Ubuntu Server 22.04
• Package Version:
  Graylog v5.0.2+59d96f8

[tmacgbay]

I believe you have to have a Default Index/Stream. You don’t have to use it though. They way I have my Graylog is setup, message processing handles all incoming messages and routes them to a new stream and index. That way if anything shows up in the Default stream/Index, I know I have some adjustments to make.

[StefanAustin]

It is possible to remote the index via command line/api call, but I believe there is a good reason why this is not possible via GUI.
Try it, but backup your system before you do it.

[joseraeiro]

@StefanAustin

It is possible to remote the index via command line/api call

And what would be the command/API call to achieve this? And what exactly do you mean by remote?

How about deleting it directly on the Elasticsearch instance?

[tmacgbay]

@joseraeiro

How about deleting it directly on the Elasticsearch instance?

There is information about in the Graylog MongoDB database as well…

Honestly it’s a better idea to not use it than to delete it. You can run into all sorts of future issues… perhaps with upgrading…

[joseraeiro]

@tmacgbay

message processing handles all incoming messages and routes them to a new stream and index

And how can I achieve the same? Can you provide a simple tutorial? Sorry, I’m new to graylog!

[gsmith]

Hey @joseraeiro

Here is my 2 cents,

Example Linux nodes:

Create a INPUT called Linux
Create a Index called Linux
Create a stream called linux /w a rule.

Message/log comes in Syslog INPUT called Linux From that Imput it routed into stream and attached to a differetn index

Add Stream “Rule” ( FYI for routing)

[joseraeiro]

Dear @gsmith,

Thanks again for the excellent help provided.

So what you suggest is that I create a stream for a different index? The problem with that approach is that if I associate a stream to my whole input I’ll get the same problem. Since I’m using a mono-input source (Wazuh logs), it would just create a duplicate index with all the information already contained in the different indices/streams, defined with rules to include all logs, but segregated by client.

[gsmith]

@joseraeiro

So what you suggest is that I create a stream for a different index? The problem with that approach is that if I associate a stream to my whole input I’ll get the same problem

My applogies, I though it was about default index. What I have been doing to help you further was read about Wazuh environment.

How it works - Log data collection · Wazuh documentation

I was looking for a way to separt the logs before sending them to Graylog.
Example somehting like this…

output {
    udp {
        port => 5064
        type=>"syslog"
        }
    }
 
output {
    beats {
        port => 12201
        type=>"gelf"
    }
}

It would be better to separate the logs before the y hit Graylog input. Just an idea

Second, using the piepline to filter out the devices, etc… from Wazuh, pretty much something like this.

Stage 0

rule "testing route to input"
when
 from_input(id:"61930ee96cfcf9713fe14bf2") AND contains  (to_string($message.message), "joseraeiro")
then 
route_to_stream(id:"619f001ede134c6ec5760c9f");
end

The idea behind this would be route_to_stream(id:“619f001ede134c6ec5760c9f”); is the the stream called “LINUX” and only “joseraeiro” in th emessage would be routed there. you can make a broader search but that is just an example, hope that helps.