Good Morning , im here bothering again
Well thats my problem ,
Error processing message RawMessage{id=db71dd60-4fa9-11e8-9498-0050568c6dce, journalOffset=72355629, codec=syslog, payloadSize=543, timestamp=2018-05-04T14:45:58.454Z, remoteAddress=/10.5.16.34:10017}
I try to get logs from the fortigate to my graylog , i see the incoming msgs but in sources show me like 20msg/s but in incoming showme like 500/s !! I read something obout import fortigate pack , i already import that but the error continues , the server.log show me the same messague so ? , or if is that not the solution , how i have to configure the format to msg on fortigate , hope u can help me… I read all the post obout these and the solution is the import pack i guess… Regrats turtle.
you ingest the messages as Syslog and it looks like fortigate did not send the messages as syslog messages that follow any standard. Maybe you change die Input to RAW just to get the messages into Graylog and then do the processing via extractors and/or processing pipelines.
With the little information you provide it is hard to tell you what is really wrong and what is happening.
I already fix that , i create another input so there is where i recibe the msgs… Know im triying to calculate the size for disk and nodes… I recibe like 500msg/s !! With ur experience , how many nodes do yo think i could need… Mi servidor has 8 cpus and 8gb de ram , with a 20gb of disk !! I think like 32 nodes with a rotation every 12 hours…
Regrats
Jorge.
I already fix that , know i have a problem on extractors to fortigate.
If i use the content pack from fortigate , and all the extractors , the buffer go crazy… So what is the solution for that , if i start the input and recibe like 800msg/s but over a input without extractor , everithing is great !!! But if a start the input with all the extractor from the content pack from graylog… The buffer overload , so my question is. ¿ There is a good extractors from fortigate , or shoul i create my owns ?
Regards-
those Content Packs are user submitted. They might not be optimal tuned for all type of input. If you have issues with the current content pack, modify the current one or create your own, based on that.
I already did that :3 , it was a problem with one expression…
{
“condition_type”: “none”,
“condition_value”: “”,
“converters”: [],
“cursor_strategy”: “copy”,
“extractor_config”: {
“regex_value”: “.+\svirus=\”(.+?)\""
},
“extractor_type”: “regex”,
“order”: 0,
“source_field”: “message”,
“target_field”: “virus”,
“title”: “FGTvirus”
},
Makes the buffer overload , so i tried to configure but is not working so i take off the extractor…
Now I edit a content pack , and its working the buffer and recibe like 800msg/s and its ok … Thanks a lot …
Regrats.