- Describe you incident:
When I test an “Extract grok to fields” rule with the Pipeline Simulator, everything works as expected. It pulls the key:value pairs I want from the message field and adds the new fields that I want. When I check the stream though, incoming syslog messages that should match the Grok Pattern do not have the additional fields. Any of these new messages can be tested with the Pipeline Simulator and they will all match and add new fields. The same Grok Pattern can also be used to create an Extractor that updates incoming syslog messages correctly (I do not want to use Extractors though). If I remove the rule from the Pipeline, the Simulator shows that no messages have been updated. As soon as I add the rule back to the Pipeline, the Simulator shows the additional fields are added. I check the stream, no additional fields are added.
- Describe your environment:
I have Graylog deployed in a lab environment as a pilot. I am testing to see if Graylog is something we might want to roll into production in the future. This is a single server implementation with Graylog, OpenSearch and MongoDB all running on the same box. The server is running Ubuntu 20.04. This is the Graylog implementation version information:
dpkg -l | grep -E “.(opensearch|graylog|mongo).” | awk ‘{print $2, $3}’
graylog-5.0-repository 1-2
graylog-5.2-repository 1-2
graylog-server 5.2.7-1
mongodb-database-tools 100.9.4
mongodb-mongosh 2.1.3
mongodb-org 6.0.13
mongodb-org-database 6.0.13
mongodb-org-database-tools-extra 6.0.13
mongodb-org-mongos 6.0.13
mongodb-org-server 6.0.13
mongodb-org-shell 6.0.13
mongodb-org-tools 6.0.13
opensearch 2.11.1
I have multiple Inputs setup but the relevant Input to this issue is named “Linux Server Syslog” and it is a Syslog UDP Input listening on port 1514. The input is receiving syslog messages and forwarding them to the “Linux Server Syslog” stream. The “Linux Server Syslog” Stream is connected to the “Linux Server Syslog” Index Set and “Remove matches from ‘Default Stream’” is checked. There is a Stream rule on the “Linux Server Syslog” Stream that matches the gl2_source_input of the “Linux Server Syslog” input. All of this is working.
I also have a Pipeline named “Linux Server Syslog” and it is connected to the “Linux Server Syslog” stream. Finally, I have one Stage named “Stage 1” (I have also tried Stage 0) and that Stage has one rule named “Linux Invalid User”. That rule has the following components in the Rules Builder:
When
Check if string value in ‘message’ contains ‘Invalid user’ (ignore case: false)
Then
Match grok pattern ‘Invalid user %{USERNAME:invalid_user} from %{IP:source_ip} port %{POSINT:port}’ on field ‘message’ and set fields for matches
An example of the syslog message I am trying to match is:
<38>May 9 00:19:11 gns3vm sshd[1966905]: Invalid user db2fenc1 from 68.183.94.223 port 38870
- What steps have you already taken to try to solve the problem?
I have rebuilt the Stream, Index, Pipeline and Pipeline Rule multiple times. I have verified that the grok pattern matches correctly with both the Simulator and an Extractor.
- How can the community help?
Any ideas as to what to look at next would be really appreciated. This seems like a pretty simple thing but I have been hitting a wall for hours.
I have added a screenshot of the Pipeline Simulator working to hopefully provide some help.
