Graylog 3.1 new streams and index sets do not conform to configured prefix

Graylog Central (peer support)
1

After upgrading to version 3.1, two new index sets and 2 new streams were created by default. These new features are great and I would like to thank the Graylog team for them.

I realize graylog 3.1 is a very recent release, however I would like to point out what I consider to be a bad practice.

So far I have not found any documentation regarding the specific configuration of these streams and index sets, they just appear with a default prefix (gl-events_* and gl-system-events_*) that cannot be configured by and administrator in the graylog config file.

In a stack that runs a dedicated ES cluster, this is not a breaking change, however in our case this can impact production, since our ES restricts index usage based on prefix, meaning that graylog cannot create new indices with a prefix not explicitly authorized.

Also should an ES cluster be shared by more than one Graylog cluster, both Graylogs would attempt to write to the same indices.

More importantly, rebuilding these index sets and streams manually to conform to a standard prefix nomenclature does not solve this problem, in fact after restarting a graylog 3.1 cluster, graylog will rebuild these index sets and streams again, regardless of the index set names already existing.

With this in mind I would like to request this behaviour be patched, and an option for prefixing these index sets be introduced or that they be created with the elasticsearch_index_prefix in order to maintain predictability, and that configuration options regarding the behaviour of these new features be introduced.

Thank you

2

if you see this as a bug - or a missing feature, please create an issue over at Github:

I can see the point you argue with, but during development you need to make assumptions and one that we are speaking public of is that the Elasticsearch Cluster is exclusive for Graylog.

Taking your Point the restored-archives-* as part of the enterprise plugins is in the same area. no custom index names/prefix

3

From graylog’s documentation:

elasticsearch_index_prefix = graylog !!

    Prefix for all Elasticsearch indices and index aliases managed by Graylog.

This strongly implies that it was intended as a global root prefix for every index that graylog creates for itself in ES.

I agree it is necessary to make assumptions, but why have a prefix for one case, and no prefix for others? It’s quite inconsistent.

It makes sense to at least have the option to configure these prefixes since they are going to be forcefully created.

With that said I will create the feature request.

4

for reference: https://github.com/Graylog2/graylog2-server/issues/6340

5

Just to give the complete picture. The setting was already given, but not yet documented. The Settings are added to the docs now.

Option was given, but not documented.

1 Like
6

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.