Good Morning ,
i have a issue with the “source” tab.
I really dont know why but a new resources apears everytime that i get a msg with the word source on the url_dst !! , i try to edit the extractor but i couldnt fix the issue… The next picture show my trouble
I have a lot of these sources , i onli have two input with two real resources…
For some reason, the “source” message field contains parts of a URL.
You’ll have to check why these messages weren’t parsed correctly and fix that.
Without more specific information about your inputs, your extractors, your clients, and some example messages, that’s pretty much all we can help.
Well here ge go.
Mi input is a FORTIGATE firewall
My extractos is the fortigate_content_pack
And here we can see a example of the messages
It seems like if the graylog take the word source from the url and think that is a new source and added to the sources list. I read all the information obout that , and something ocurrs to me , create a pipeline rule and try to avoid the source .
I think that these is the extractor with the trouble
Well thats all , i dont know what else can i show u turtle.
Regarts.
This solution dosnt work for me , because when i create a new input on raw format… The new mesagues commes without any information , onli the ip_source and nothing else, so the dasboards dosnt work neither !!! With the UDP SYSLOG , everithing goes well ,but the onli trouble is that everitime that a message gets on the url a = or something like that create a new source… I tried with pipelines , but nothing and also read that i can not add a static field on GROCK ,. so i really dont know how to fix that.
if you have a raw input that receives the fortigate syslog messages you would need to parse all fields yourself out of the raw text string.
Maybe you should ask the creator of the fortigate contentpack if he is able to help. I do not have such a device so I’m not able to help on that.
Thanks for the answer , i already did for my self… Tomorrow i will send to the creator for the content pack my .json , so he can commited to github with the problem solved. Know all is working ,dasboards , the source issue , and the parse messagues.
Regards
Hello,
Maybe you can try this extractor config for the RAW input using regex.
(It will keep the original field names coming from the logs, so you don’t need to keep any of them manually extracting one by one…)
{
"extractors": [
{
"title": "KV",
"extractor_type": "copy_input",
"converters": [
{
"type": "tokenizer",
"config": {}
}
],
"order": 2,
"cursor_strategy": "copy",
"source_field": "message",
"target_field": "message",
"extractor_config": {},
"condition_type": "none",
"condition_value": ""
},
{
"title": "Remove level key",
"extractor_type": "regex_replace",
"converters": [],
"order": 1,
"cursor_strategy": "copy",
"source_field": "message",
"target_field": "message",
"extractor_config": {
"replacement": " ",
"regex": "level=(notice|information|warning|error|alert)"
},
"condition_type": "none",
"condition_value": ""
},
{
"title": "Remove initial numeric tag",
"extractor_type": "regex_replace",
"converters": [],
"order": 0,
"cursor_strategy": "copy",
"source_field": "message",
"target_field": "message",
"extractor_config": {
"replacement": " ",
"regex": "^(\\<\\d+\\>)"
},
"condition_type": "none",
"condition_value": ""
}
],
"version": "2.1.0-SNAPSHOT"
}
Maybe an improvement would be using the same extractor config to add a fortinet_ prefix in each key to avoid having to remove the level field from Fortinet logs (to avoid having issues with ingestion that is present in the versions I had contact =< 2.3)
This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.