Graylog how to sync fields from logstash?

[tzm7614]

Hello

Environment info:
elasticsearch-6.8.6
graylog-3.1.3
logstash-7.5.1
mongodb-3.0.8

Logstash output log format content:
{
“type” => “system_kernel”,
“host” => “pc-zjdssdstest01.zj.chinamobile.com”,
“@timestamp” => 2020-02-21T02:03:07.595Z,
“program” => “systemd”,
“path” => “/var/log/messages”,
“timestamp” => “Feb 21 10:03:01”,
“logsource” => “pc-zjdssdstest01”,
“message” => “Feb 21 10:03:01 pc-zjdssdstest01 systemd: Started Session 39820 of user root.”,
“@version” => “1”
}
{
“type” => “system_kernel”,
“host” => “pc-zjdssdstest01.zj.chinamobile.com”,
“@timestamp” => 2020-02-21T02:03:07.595Z,
“program” => “systemd”,
“path” => “/var/log/messages”,
“timestamp” => “Feb 21 10:03:01”,
“logsource” => “pc-zjdssdstest01”,
“message” => “Feb 21 10:03:01 pc-zjdssdstest01 systemd: Starting Session 39820 of user root.”,
“@version” => “1”
}

graylog result: Graylog does not have those fields in logstash:

How can I synchronize fields from logstash ?

[jan (Jan Doberstein)]

you did not share the most important part.

How did you ingest the messages from logstash to Graylog?

What transport did you use in logstash and what input did you use on Graylog.

[tzm7614]

hello!
Hello, I am logstash output to kafka, and then graylog reads logs from kafka

Configuration file:
input {
file {
type =>“system_kernel”
path => [ “/var/log/messages” ]
start_position => beginning
#ignore_older => 31557600
ignore_older => 86400
#sincedb_path => “/dev/null”
}
}
filter {
grok {
match => [
“message”, “%{SYSLOGBASE}”
]
}
}

output {
if [type] == “system_kernel” {
kafka {
acks => “0”
topic_id => “system_kernel”
bootstrap_servers => “10.78.230.201:9092,10.78.230.202:9092,10.78.230.203:9092”
#bootstrap_servers => “10.78.182.78:9092,10.78.182.79:9092,10.78.182.80:9092,10.78.182.81:9092,10.78.182.82:9092”
}
}
}

graylog intput:

[jan (Jan Doberstein)]

syslog is a semi structured message - you might want to check how the message looks in RAW Kafka input … maybe this will clarify that more.

[tzm7614]

Thank you very much. After using GELF Kafka, I got the desired result. However, my version of logstash-2.3.2-1 is normal, logstash-7.5.1 has errors
.
error info:
com.fasterxml.jackson.core.JsonParseException: Unexpected character (’-’ (code 45)): Expected space separating root-level values

Excuse me, which version of logstash matches graylog3?