I just deployed a graylog cluster inspired by a standalone graylog node configuration that was in place
This cluster has 3 nodes
Each node has graylog, elasticsearch and mongodb
2 of them have an nginx server configured
They are connected to each other on a dedicated network with the adresses 10.0.0.1/2/3 for node 1/2/3
Logs come from devices in 2 networks to node 2 for the first network and 3 for the second network
Node 1 has the role of graylog master node and the privileged web interface of the cluster (nginx)
Node 3 has a web interface too (nginx) for redundancy purposes (in case of the first node goes down)
MongoDB replica set is operationnal (replication, election etc)
Elasticsearch cluster is green (replication, election etc)
Graylog nodes are operationnal
Logs are coming and viewable in searches, dasboards etc
I can import dashboards, extactors, streams, notifications, event definitions and searches
There is not a single error message in either graylog, elasticsearch, mongodb or nginx
nginx from node 1 receive https requests on 10.0.1.2:443 and forward them on 10.0.0.1:9000 (and nginx node 3 does the same bettween 10.0.2.1:443 and 10.0.0.3:9000)
Versions :
Graylog : 4.0.8
Elasticsearch : 7.13.2
OS : CentOS 7.9.2009
MongoDB : 4.2
Nginx : 1.20.1
When i try to list all available fields, i don’t even see the built-in ones like “source” or “timestamp” :
(there is no stream in filter so it is the default “all messages” stream)
I feel like the problem is after the reception of the API response and the display but that’s just my insight
Feel free to ask for more informations, i’ll try my best to provide them while respecting my company’s confidentiality rules.
Please help. I’m in intership and this is like the last wall between me and my project’s completion.
As you can see, if the whole dashboard is imported with corresponding extractors, fields are used.
But when i want to create a new widget, i can’t select any field :
As you can see, if the whole dashboard is imported with corresponding extractors, fields are used.
But when i want to create a new widget, i can’t select any field :
It might have something to do with your fields, but I’m not 100% sure.
For some reason I’m unable to see most of your screen shots, they normally pop out.
Was there any errors/warnings in Elasticsearch log file that may pertain to this issue?
And when you open/view one of these logs I assume there are fields shown but there just NOT showing in the search bar on the left of Web UI? Is this correct?
That first screen shot that shows “Unsaved Search” was that a Global search and was it confgiured as *“search in all messages” ?
If you naviagte to System → Configuration do you have something similar like this?
I saw the version compatibility matrix but elasticsearch 7.13 was already in place and working on the standalone node so i assumed that there wasn’t any problem in the features used beforehand. Plus, from what i saw there :
The problem with too recent versions of elasticsearch is more around indexing, so if it was broken because of the version, i shouldn’t have been able to store, exploit and retrieve logs. That said, it’s just my assumption.
For the screenshots issue, it’s weird because i tried to see them in firefox and chrome and both are showing them correctly. Maybe try to update your browsers or ask a colleague if he has the same problem?
I just checked elasticsearch and graylog logs and there is no warning/error at all except “Elasticsearch built-in security features are not enabled…”
Yes, this is correct. when i open a log message in the search interface, i can see each field and value but the list on the left pannel is empty and when i try to put a field in the search bar, there is no autocompletion, and when i try to create an agregation, i can select any field for rows or columns.
Yes it was a global search on all messages, no filters at all.
Yes i have something similar, not exactly the same but very close :
@Thomas_V fwiw while I did note later in that thread that we’re on 7.12 without issue (except for when a given index set first rolled over after the upgrade to 7.12) we have put ES updates on hold at 7.12.1. Elastic has continued to make several changes with potential to impact how graylog-server interacts with elasticsearch in the meantime, including significant breaking changes at each minor revision and several security changes.
I debated noting in the thread you linked that it works with us for 7.12.1 because I didn’t want to imply that I’m encouraging others to ignore the guides and requirements, but like you we had already upgraded before we realized it was an issue so we simply moved forward and got lucky that it still works.
I checked in “all” and “all including reserved fields” and nothing.
I also rotated the indices and nothing.
I checked both git links and the difference between their issue and mine is that i don’t have any field even without filter by any streams. The problem is more upstream.
Unfortunately I’m unable to read this, is it posible to redo this screen shot?
If you restart your Graylog Service and “tail” your Graylog log file do you see any warning, errors or some type of information that may pertain to this issue.
Is there anything that may show issues in your elasticsearch log files?
