Unusual query behaviour in Graylog 3

I have a graylog cluster that has the 2 inputs, gelf_tcp and syslog_udp.
The gelf_tcp input has an extractor applied to extract fields related to our kubernetes environment, for example: kubernetes_label_system.
The syslog_udp input does not have any extractors applied and logs do not contain any kubernetes fields.
Both inputs have the standard fields, eg. message.

From the graylog home screen search, as an admin user, which searches across all inputs.

Query A: kubernetes_label_system: podA

  • The search runs correctly and only displays results that match from the gelf_tcp input only.

Query B: message: healthcheck

  • This search runs correctly and displays results that match from both gelf_tcp and syslog_udp.

Query C: kubernetes_label_system: podA AND NOT message: healthcheck

  • This search does not run correctly and displays messages
  • I have also tried this query with or without quotes, and different order with no luck:
    • NOT message: "healthcheck" AND kubernetes_label_system: "podA"

Query D: kubernetes_labels_component:/webservices/ AND NOT message: healthcheck

  • This search runs correctly and returns results that match from the gelf_tcp input only.

Can someone please help explain:

  1. Why does Query C return messages from both inputs and not work as expected?
  2. Why does Query D work correctly?
  3. The queries worked as expected with Graylog 2.5 and older. Are the queries and/or the prescence of fields handled differently with Graylog 3?

Graylog version: Graylog 3.0.2
Elasticsearch: AWS ES Elasticsearch 6.5

While i am able to work around these changes i would really appreciate any help with understanding this.
Thanks

1 Like

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.