1. Describe your incident:
I collect logs from fortigate.
Logs as raw text UDP.
I have message where have all information (ip, action, policy, type ect.)
I trying use grok pattern for extract all this information to single value:
ip=192.168.0.0
action=block
type=test
ect.
Some grok patterns wokring, but graylog recived new messages from fortigate but not showing in search window. I see only old logs.
I use one grok pattern one value (ex. dstintf=%{QUOTEDSTRING:dstintf}) Its possible check why new messages not showing?
EDIT
I found these errors, ex:
22 minutes ago graylog_0 46f2e1f2-b441-11ed-9e18-ac220b793aac OpenSearchException[OpenSearch exception [type=mapper_parsing_exception, reason=failed to parse field [level] of type [long] in document with id '46f2e1f2-b441-11ed-9e18-ac220b793aac'. Preview of field's value: 'notice']]; nested: OpenSearchException[OpenSearch exception [type=illegal_argument_exception, reason=For input string: "notice"]];
22 minutes ago graylog_0 46589dc1-b441-11ed-9e18-ac220b793aac OpenSearchException[OpenSearch exception [type=mapper_parsing_exception, reason=failed to parse field [level] of type [long] in document with id '46589dc1-b441-11ed-9e18-ac220b793aac'. Preview of field's value: 'information']]; nested: OpenSearchException[OpenSearch exception [type=illegal_argument_exception, reason=For input string: "information"]];
2. Describe your environment:
Ubuntu 20.04
Graylog v5.0.3
Openserach
The error message is saying that the data from field level is coming in as text (“notice”) but Opensearch has set the field type as “long” so it can only accept numbers. Since the message has incorrect field types it can’t be put in the index and won’t show.
When dealing with fields and there datatype this would be the index itself, which is Opensearch/elasticsearch. You would need to either reroute your logs to a different index, or modify the currecnt index set field.
hey,
That would take a while to teach ya, but something simple you can create a new index here
Create a new stream called " Fortigate" and attach it to the new Index set… Found here
What needs to happen now is create a rule that send your Fortigate log to the new stream, There are multiple ways to do this. Example when using the field source:
Im not sure what other do but for every type of device I have a different Index set, for example Firewall , Linux , Windows , Switchs, Netflow, Accesspoints, etc…
And what will I achieve with such a solution? Just that the logs from fortigate will be in a person index, but will not have specific data mapped to specific fields?
After the data is flowing from the fortigate to graylog, you can set up processing rules to parse the message into various fields such as source, destination, action, etc. You can also enrich the logs by doing things like geo-ip or whois lookups.
Those rules could be in the form of a “pipeline” or an “extractor” within Graylog.
You can then use those fields to search data for specific information, or set up dashboard reports
