Config Elastic Packetbeat graylog

[bahram]

hi, guyes

I install packetbeat version 7.4.2 on VM linux centos7 machine
i used graylog version 3.1
even add file graylog-plugin-beats-2.4.0-beta.1.jar to /usr/share/graylog-server/plagin/

and test connection packetbeat is OK
//
[root@centos_Host ~]# packetbeat test output
logstash: 192.100.102.25:15999…
connection…
parse host… OK
dns lookup… OK
addresses: 192.100.102.25
dial up… OK
TLS… WARN secure connection disabled
talk to server… OK //
but even though Network IO The show is receiving a packet,
in graylog input nothing for see

please guide me

[shoothub]

Please don’t install beast plugin for graylog, it is deprecated.
Normal Input - Beast would be enough.
Check also this blog post:

Tapping Wires for Lean Security Monitoring: DNS Request Analysis with Open...

And this:

Packebeat Input

Hi Jan, thanks for your support. Actually I solved the problem. I’m writing the solution right here in the hope somebody would find it useful. I found that the only way to have Packetbeat working with Graylog is to let it be run by Sidecar. So I needed to configure a new log collector manually (I found only winlogbeat, filebeat and nxlog template, despite on what I read on the website announcing version 3) and pushing this configuration: # Needed for Graylog fields_under_root: true fields.col…

[jan (Jan Doberstein)]

even add file graylog-plugin-beats-2.4.0-beta.1.jar to /usr/share/graylog-server/plagin/

why did you do this?

The beats input is part of the Graylog core. Having that non matching plugin will confuse/destroy only your Graylog.

---

• Remove the 2.4 beta plugin.
• configure the input and add a static field. Search for that static field not for the input id.
• check your Graylog server.log for errors.

[bahram]

Hi,
very, very thank you for answering
OK I removed the Beat plugin But there is still the problem
I see Packet in Network IO and sending traffic to Graylog input
But I don’t see anything filed
Why do you think?
How do beats work in Garylog in general?

[bahram]

Hi, jan
Thank you for your guidance
Okay. I didn’t know beats input is part of the Graylog core
1- I Remove the 2.4 beta plugin
2- configre and add static filed
3- check your Graylog server.log for errors.
but : log error
2019-11-12T06:08:34.493+03:30 ERROR [Messages] Failed to index [4] messages. Please check the index error log in your web interface for the reason. Error: One or more of the items in the Bulk request failed, check BulkResult.getItems() for more information.
2019-11-12T06:08:36.562+03:30 WARN [Messages] Failed to index message: index=<graylog_0> id=<85b4f8f0-04f5-11ea-9ce3-005056bfa72b> error=<{“type”:“illegal_argument_exception”,“reason”:“Limit of total fields [1000] in index [graylog_0] has been exceeded”}>
2019-11-12T06:08:36.562+03:30 WARN [Messages] Failed to index message: index=<graylog_0> id=<85b4f8f1-04f5-11ea-9ce3-005056bfa72b> error=<{“type”:“illegal_argument_exception”,“reason”:“Limit of total fields [1000] in index [graylog_0] has been exceeded”}>
2019-11-12T06:08:36.562+03:30 WARN [Messages] Failed to index message: index=<graylog_0> id=<85b742e2-04f5-11ea-9ce3-005056bfa72b> error=<{“type”:“illegal_argument_exception”,“reason”:“Limit of total fields [1000] in index [graylog_0] has been exceeded”}>
2019-11-12T06:08:36.562+03:30 WARN [Messages] Failed to index message: index=<graylog_0> id=<85b742e3-04f5-11ea-9ce3-005056bfa72b> error=<{“type”:“illegal_argument_exception”,“reason”:“Limit of total fields [1000] in index [graylog_0] has been exceeded”}>
2019-11-12T06:08:36.562+03:30 WARN [Messages] Failed to index message: index=<graylog_0> id=<85ea12b0-04f5-11ea-9ce3-005056bfa72b> error=<{“type”:“illegal_argument_exception”,“reason”:“Limit of total fields [1000] in index [graylog_0] has been exceeded”}>
2019-11-12T06:08:36.562+03:30 WARN [Messages] Failed to index message: index=<graylog_0> id=<864ddd90-04f5-11ea-9ce3-005056bfa72b> error=<{“type”:“illegal_argument_exception”,“reason”:“Limit of total fields [1000] in index [graylog_0] has been exceeded”}>
2019-11-12T06:08:36.562+03:30 WARN [Messages] Failed to index message: index=<graylog_0> id=<86500070-04f5-11ea-9ce3-005056bfa72b> error=<{“type”:“illegal_argument_exception”,“reason”:“Limit of total fields [1000] in index [graylog_0] has been exceeded”}>
2019-11-12T06:08:36.562+03:30 ERROR [Messages] Failed to index [7] messages. Please check the index error log in your web interface for the reason. Error: One or more of the items in the Bulk request failed, check BulkResult.getItems() for more information.
2019-11-12T06:08:38.472+03:30 WARN [Messages] Failed to index message: index=<graylog_0> id=<86e6e940-04f5-11ea-9ce3-005056bfa72b> error=<{“type”:“illegal_argument_exception”,“reason”:“Limit of total fields [1000] in index [graylog_0] has been exceeded”}>
2019-11-12T06:08:38.472+03:30 WARN [Messages] Failed to index message: index=<graylog_0> id=<86e8e510-04f5-11ea-9ce3-005056bfa72b> error=<{“type”:“illegal_argument_exception”,“reason”:“Limit of total fields [1000] in index [graylog_0] has been exceeded”}>
2019-11-12T06:08:38.473+03:30 WARN [Messages] Failed to index message: index=<graylog_0> id=<877fa6d0-04f5-11ea-9ce3-005056bfa72b> error=<{“type”:“illegal_argument_exception”,“reason”:“Limit of total fields [1000] in index [graylog_0] has been exceeded”}>
2019-11-12T06:08:38.473+03:30 WARN [Messages] Failed to index message: index=<graylog_0> id=<8781c9b0-04f5-11ea-9ce3-005056bfa72b> error=<{“type”:“illegal_argument_exception”,“reason”:“Limit of total fields [1000] in index [graylog_0] has been exceeded”}>
2019-11-12T06:08:38.473+03:30 ERROR [Messages] Failed to index [4] messages. Please check the index error log in your web interface for the reason. Error: One or more of the items in the Bulk request failed, check BulkResult.getItems() for more information.
2019-11-12T06:08:39.508+03:30 WARN [Messages] Failed to index message: index=<graylog_0> id=<8818d991-04f5-11ea-9ce3-005056bfa72b> error=<{“type”:“illegal_argument_exception”,“reason”:“Limit of total fields [1000] in index [graylog_0] has been exceeded”}>
2019-11-12T06:08:39.508+03:30 WARN [Messages] Failed to index message: index=<graylog_0> id=<8818d990-04f5-11ea-9ce3-005056bfa72b> error=<{“type”:“illegal_argument_exception”,“reason”:“Limit of total fields [1000] in index [graylog_0] has been exceeded”}>
2019-11-12T06:08:39.508+03:30 WARN [Messages] Failed to index message: index=<graylog_0> id=<881aae51-04f5-11ea-9ce3-005056bfa72b> error=<{“type”:“illegal_argument_exception”,“reason”:“Limit of total fields [1000] in index [graylog_0] has been exceeded”}>
2019-11-12T06:08:39.509+03:30 WARN [Messages] Failed to index message: index=<graylog_0> id=<881aae50-04f5-11ea-9ce3-005056bfa72b> error=<{“type”:“illegal_argument_exception”,“reason”:“Limit of total fields [1000] in index [graylog_0] has been exceeded”}>
2019-11-12T06:08:39.509+03:30 ERROR [Messages] Failed to index [4] messages. Please check the index error log in your web interface for the reason. Error: One or more of the items in the Bulk request failed, check BulkResult.getItems() for more information.

[bahram]

Hi, jan

Unfortunately, the installation of this plugin (graylog-plugin-beats-2.4.0-beta.1.jar) disrupted Garylog server performance.
In the new Graylog node I can see all Packet (إٍBEATS ==> packetbeat, filebeat, metricbeat,heartbeat) without any problems

Thank you for your good guidance