Content Pack content selection broken

Graylog Central (peer support)
1

Before you post: Your responses to these questions will help the community help you. Please complete this template if you’re asking a support question.
Don’t forget to select tags to help index your topic!

1. Describe your incident:

At first I noticed that sometimes the content packs are completely empty.

After a few days and a lot of experimentation I now know for certain:

The content selection has strange bugs.

The parameters page shows stuff I did not select. Selecting and deselecting does not fix this.

And this seems to be reflected in what is exported. So far I assume that all the previous are correct. As at least they always reliably told me when it completely ignored all selections and was about to create an empty content pack.

Example for an “empty pack“, entire content of the file. Yes it is only ~200 bytes.

{
  "v": 1,
  "id": "bcc2fd8b-4e0c-44c7-ac63-59fc645c87d8",
  "rev": 1,
  "name": "streams",
  "summary": "streams",
  "description": "streams",
  "vendor": "removed",
  "url": "https://removed",
  "parameters": [],
  "entities": []
}

2. Describe your environment:

Graylog 6.3.2

Ubuntu LTS 22

3. What steps have you already taken to try and solve the problem?

Exporting several subsets of data.

Classic workaround of selecting and deselecting again.

Starting the entire process again. And again after several days.

4. How can the community help?

Is this a known Bug? Any ideas on how to diagnose this bizarre behavior?

This makes it really hard to use that feature as it makes super unreliable.

2

Update: I narrowed it down to certain entries. That cause this. But I have not yet found a pattern.

3

could you share those patterns to help us gathering ideas?

4

That’s the issue. I really can’t find one.

2 streams that cause this problem have only one “must contain“ rule with only [A-Z_]

Example: source must contain THIS_TEXT

So they are really close to being as simple as they could be from my understanding. Much more complex regex stuff does not cause a problem.

hobs’ solution of just exporting all mongodb contents and the importing them seems promising:

At least the streams all seem to be there and complete.

5

Correction, I have now found a pattern and I made mistake.

the pattern seems to be “-“ in a must contain rule. I accidentally bundled the one with the _ in during trial and error to find out which streams cause problems.

And now I know why that happened. Starting the process once selecting one problematic stream breaks the process. Ad it has to be started new to even get a chance of getting reliable results. Going back and deselcting problematic streams does not help. Making the trial and error process very misleading and frustrating.

6

So the good news is:

At least the new assumed pattern contains more characters that could cause escape sequence problems.

The bad news:

I have no idea if that is the problem or if it is how to escape it without braking functionality.

I am far away from being certain on how to handle this. I will probably just not use content packs as they seem way too unpredictable in behavior.

The way to go for me seems to be mongodump/mongorestore

7

Hey @daniel2

What is the pattern you suspect might be causing this, could you provide an example?

8

I don’t know what suggests to people that I have more information then I already shared. That post was just supposed to be a closing summary in case I don’t pursue this an further and therefore won’t post in this topic anymore.

Meaning: If I don’t post it, I probably did not find it.

9

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.