Hi,
is there any default functionality in terms of key value pairs extraction?
Why I am asking - I found some fields in my graylog instance which were NOT defined in any extractor. So i deleted the whole INPUT, created a new (empty) one without any extractors and … suprise … all the fields are extracted again (looks like KVP)? Its a little bit spooky for me 
I am running latest Graylog 2.4.0 Version cluster with 3 nodes and 5.4.x ES.
Many Thanks & BR
Oliver
What type of input have you been using?
What is the full configuration of the input?
What kind of log messages has Graylog been receiving?
Please provide some examples.
Input typ: Syslog UDP
Full configuration:
allow_override_date: true
bind_address: 0.0.0.0
expand_structured_data: false
force_rdns: false
override_source:
port: 5001
recv_buffer_size: 262144
store_full_message: false
transportflag: direct
vendorflag: fortigate
log message example:
date=2018-01-15 time=16:14:05 devname=xxxxx devid=FGT1111111 logid=0000000013 type=traffic subtype=forward level=notice vd=root srcip=10.xx.xx.xx srcport=68 srcintf=“xxxxx-xxxxx” dstip=xx.xx.xx.xx dstport=67 dstintf=“xxx-xxxxx” sessionid=0000000 proto=17 action=deny policyid=0 policytype=policy dstcountry=“Reserved” srccountry=“Reserved” trandisp=noop service=“DHCP” duration=0 sentbyte=0 rcvdbyte=0 sentpkt=0 appcat=“unscanned” crscore=30 craction=131072 crlevel=high
Its default Fortigat Syslog format.
which is
Received by
GLOB Fortigate UDP on 12ef998f / nw-glog3
Stored in index
graylog_5
Input Extractors:
Extractors of GLOB Fortigate UDP
Extractors are applied on every message that is received by this input. Use them to extract and transform any text data into fields that allow you easy filtering and analysis later on. Example: Extract the HTTP response code from a log message, transform it to a numeric field and attach it as http_response_code to the message.Find more information about extractors in the documentation.
Add extractor
Start by loading a message to have an example to work on. You can decide whether to load a recent message received by this input, or manually select a message giving its ID.Get started
Configured extractors
This input has no configured extractors.
br
oliver
Yes, these are parsed by default:
Many THX Jochen!
And it seems I am hitting the same bug as described in bottom - url field in syslog will create a lot of unusable fields (until maximum fields limit in ES reached and ES stops … yihaa). Any chance to disable this functionality? further … we are using different field names … so no need for using default kvp extraction…
Thx & BR
Oliver
No, but you could use a Raw/Plaintext UDP input and extract the information you want via some extractors or pipeline rules.
Plaintext UDP works as exprected - many thanks
This default behavior also effects GELF Inputs which receives Fortinet syslogs from collectors?
BR
No, only Syslog TCP and Syslog UDP inputs.
This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.
