Raw tomcat logs are not saved to "message" field

[jikistro (Łukasz)]

Description of your problem

Hi,
I am using graylog 4.1.3 configured in a 3 node cluster and encounter problem as depicted by following screenshot:

It seems that tomcat logs are saved to “filebeat_event_original” instead of “message” field and i would like to know if that’s an expected behavior or i am doing something wrong?
Preferably i would like it to be saved into message field which is displayed on default dashboard as it is in case of other servers i am managing logs from like mongodb, haproxy - only tomcat logs get that special treatment so to speak. How would i go about it?

Tomcat logs are forwarded to graylog through beats input by filebeat as is in the case with other mentioned services. Please let me know what other info would be useful in tackling this problem.

Operating system information

• Debian 11

Package versions

• Graylog 4.1.3
• MongoDB 4.4.8
• Elasticsearch 7.10.2

Here’s the filebeat tomcat module config file:

```
- module: tomcat
  # All logs
  log:
    enabled: true

    # Set which input to use between syslog (default) or file.
    var.input: file

    # Set custom paths for the log files. If left empty,
    # Filebeat will choose the paths depending on your OS.
    var.paths:
    - /opt/tomcat/current/logs/*.log
    - /opt/tomcat/current/logs/catalina.out

```

For longer code or configuration bits, please enclose your snippet in a summary block like this:

Summary of your code snippet or config here

Your code goes inside the triple backticks

[tmacgbay]

Is the “filebeat tomcat module config” what you are using in graylog->system->sidecars-configuration as a Collector Configuration?

Here is what we have as a default Collector Configuration that is applied to our Linux machines…

# Needed for Graylog
fields_under_root: true
fields.collector_node_id: ${sidecar.nodeName}
fields.gl2_source_collector: ${sidecar.nodeId}

filebeat.inputs:
- type: log
  enabled: true
  paths:
    - /var/log/auth.log
  ignore_older: 72h
  tags:
    - linux

output.logstash:
   hosts: 
   - ${user.BeatsInput}

path:
  data: /var/cache/graylog-sidecar/filebeat/data
  logs: /var/log/graylog-sidecar

[jikistro (Łukasz)]

I haven’t got arround to concept of sidecar as of yet, there is some default configuration that i didn’t change in any way but knowing now it’s some centralized configuration system i doubt it’s source of my problem as it would have an impact on all my services logs.

I only pasted a dynamic configuration of filebeat agent for tomcat → /etc/filebeat/modules.d/tomcat.yml
Thanks though.

[tmacgbay]

If you aren’t processing logs in a manner that Graylog expects (using sidecars managing the configuration) then you are held to the results you can get by working with Elasticsearch modules/configs to get fields like message. As such, the fields you are receiving are those defined by you and Elasticsearch, rather than you and Graylog.

One of the advantages of using sidecar is you can create a single configuration instance for all filebeat clients and apply them from the Graylog GUI.

[jikistro (Łukasz)]

Hell yeah! works like a charm through sidecar, thanks!