No fields to show

Hello everyone,

I’m having some issues concerning fields usage.

Here’s the context :

  • I just deployed a graylog cluster inspired by a standalone graylog node configuration that was in place
  • This cluster has 3 nodes
  • Each node has graylog, elasticsearch and mongodb
  • 2 of them have an nginx server configured
  • They are connected to each other on a dedicated network with the adresses 10.0.0.1/2/3 for node 1/2/3
  • Logs come from devices in 2 networks to node 2 for the first network and 3 for the second network
  • Node 1 has the role of graylog master node and the privileged web interface of the cluster (nginx)
  • Node 3 has a web interface too (nginx) for redundancy purposes (in case of the first node goes down)
  • MongoDB replica set is operationnal (replication, election etc)
  • Elasticsearch cluster is green (replication, election etc)
  • Graylog nodes are operationnal
  • Logs are coming and viewable in searches, dasboards etc
  • I can import dashboards, extactors, streams, notifications, event definitions and searches
  • There is not a single error message in either graylog, elasticsearch, mongodb or nginx
  • nginx from node 1 receive https requests on 10.0.1.2:443 and forward them on 10.0.0.1:9000 (and nginx node 3 does the same bettween 10.0.2.1:443 and 10.0.0.3:9000)

Versions :

  • Graylog : 4.0.8
  • Elasticsearch : 7.13.2
  • OS : CentOS 7.9.2009
  • MongoDB : 4.2
  • Nginx : 1.20.1

When i try to list all available fields, i don’t even see the built-in ones like “source” or “timestamp” :
image
(there is no stream in filter so it is the default “all messages” stream)

I feel like the problem is after the reception of the API response and the display but that’s just my insight
Feel free to ask for more informations, i’ll try my best to provide them while respecting my company’s confidentiality rules.
Please help. I’m in intership and this is like the last wall between me and my project’s completion.

Thomas

Here is a dasboard’s widget for mail analysis where the “status_1” field is from an imported extractor :
image

Here is nginx access logs when i load the search page of graylog’s web interface :
image

As you can see, if the whole dashboard is imported with corresponding extractors, fields are used.
But when i want to create a new widget, i can’t select any field :
image

As you can see, if the whole dashboard is imported with corresponding extractors, fields are used.
But when i want to create a new widget, i can’t select any field :
image

Here is a curl result on Graylog’s API to retrieve known fields :
image

Here is a scheme of my architecture (with modified addresses and names obviously) :
image

Hello && Welcome

The first post I noticed you are using ES 7.13. Have you seen this?

https://docs.graylog.org/en/4.0/pages/configuration/elasticsearch.html#elasticsearch-versions

It might have something to do with your fields, but I’m not 100% sure.
For some reason I’m unable to see most of your screen shots, they normally pop out. :thinking:

Was there any errors/warnings in Elasticsearch log file that may pertain to this issue?

And when you open/view one of these logs I assume there are fields shown but there just NOT showing in the search bar on the left of Web UI? Is this correct?

That first screen shot that shows "Unsaved Search" was that a Global search and was it confgiured as *“search in all messages” ?

If you naviagte to System → Configuration do you have something similar like this?

Hello && thanks,

I saw the version compatibility matrix but elasticsearch 7.13 was already in place and working on the standalone node so i assumed that there wasn’t any problem in the features used beforehand. Plus, from what i saw there :

The problem with too recent versions of elasticsearch is more around indexing, so if it was broken because of the version, i shouldn’t have been able to store, exploit and retrieve logs. That said, it’s just my assumption.

For the screenshots issue, it’s weird because i tried to see them in firefox and chrome and both are showing them correctly. Maybe try to update your browsers or ask a colleague if he has the same problem?

I just checked elasticsearch and graylog logs and there is no warning/error at all except “Elasticsearch built-in security features are not enabled…”

Yes, this is correct. when i open a log message in the search interface, i can see each field and value but the list on the left pannel is empty and when i try to put a field in the search bar, there is no autocompletion, and when i try to create an agregation, i can select any field for rows or columns.

Yes it was a global search on all messages, no filters at all.

Yes i have something similar, not exactly the same but very close :

Thanks for your time.

@Thomas_V fwiw while I did note later in that thread that we’re on 7.12 without issue (except for when a given index set first rolled over after the upgrade to 7.12) we have put ES updates on hold at 7.12.1. Elastic has continued to make several changes with potential to impact how graylog-server interacts with elasticsearch in the meantime, including significant breaking changes at each minor revision and several security changes.

I debated noting in the thread you linked that it works with us for 7.12.1 because I didn’t want to imply that I’m encouraging others to ignore the guides and requirements, but like you we had already upgraded before we realized it was an issue so we simply moved forward and got lucky that it still works.

@Thomas_V

Have you tried different configuration with your Message Processors Configuration? Maybe something like this?

Here is some information.

The importance of message processor ordering

If you do adjust these confgiurations I think it may take a couple minutes.

I just put the same configuration as it is on your screenshot and stilll no fields…

Hello,

Have you tried clicking the 'All" link, if so what happened?

image

Have you tried to rotate active indices after reconfgiuring “Messgae Processors Configuration” , just an idea.

image

I also found these.

Hello,

I checked in “all” and “all including reserved fields” and nothing.

I also rotated the indices and nothing.

I checked both git links and the difference between their issue and mine is that i don’t have any field even without filter by any streams. The problem is more upstream.

Any other idea?

Hello.

Unfortunately I’m unable to read this, is it posible to redo this screen shot?

If you restart your Graylog Service and “tail” your Graylog log file do you see any warning, errors or some type of information that may pertain to this issue.
Is there anything that may show issues in your elasticsearch log files?

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.