Long story short, we are using gGaylog to read from elastic indexes of course, but, i have a need to write my indexes in the Elastic Common Scheme (ECS) format that the native elastic tools such as “Beats” create, which they are not or do not appear to be now.
Is there any way i can tell Graylog to use this format?
The idea behind this is to write indexes in a manner that would be usable by Elastic Security for example, without having to use the beats or any other agents, using the existing indexes im already collecting rather than replicating them multiple times.
I believe you are delving into how Graylog queries Elasticsearch… Graylog isn’t a front end for all of Elasticsearch’s (and by extension, beats) features and generally expects to only find what it has placed into Elasticsearch (beats, routed to Graylog via sidecar, then stored in Elasticsearch. Add on to that that Graylog only supports Elasticsearch to 7.10.2 - which may be too old for your Elastic Security… Graylog direction for backend DB is pointing to Opensearch now… though Elastic works to the aforementioned version.
Not really an answer, more like my view of how Graylog fits to your question…
I appreciate the response, yeah… ive been told this may be possible, but nothing about the how.
The trick is that elastic agents write in a different format, and i dont have the knowledge to change it. If i could i would solve a ton of issues and be able to support my existing architecture how we use them all.
Im aware of the versioning issues, but i can work around that.
The bottom line is i somehow need to either covert the indexes, or start new ones written in, the ECS format, if possible.
A while back I wrote up some detail on custom mappings and changing existing indexes to match your desired field type… perhaps you can pull some detail out of there to help bend Graylog/ElasticSearch to your will?
I wonder if someone using Elastic Security can maybe post the mappings for the relevant indices? I don’t have a live one available to me currently. Im trying to build one but info from an actual working one would be helpful.
Pose this as a new question - other members of the forum won’t see your request as it is embedded in the thread of your original question… you might have better luck with in depth questions on Elastic Security on the Elasticsearch forums…
