Hi, we are trying to send metric beats information into Graylog. I have installed metricbeat on a Linux machine and have configured it to point to our Graylog server. Input is created in Graylog. Firewall rule is in place on Graylog server to allow traffic. Able to telnet from source server to Graylog server on specified port. Metricbeat service appears to start successfully on source server, no discernable errors in metricbeat.log, just looks like its sending data. When I hit ‘show messages’ on the Input in Graylog it shows no messages. Setting up an extractor yields the same thing. Not sure where to look to find the metrics data. I have complete configurations saved in zip file for your review but can’t post it. Please let me know how to post the zip file.
Check the logs of your Graylog and Elasticsearch nodes.
http://docs.graylog.org/en/2.4/pages/configuration/file_location.html
You can post larger text files by using a pastebin service such as https://gist.github.com or https://0bin.net/ and share the link.
The pastebins only seem to work for .txt files. I thought I saw someone attach a zip file to a Graylog Topic. How did they accomplish that?
They didn’t.
Only image files (GIF, JPEG, PNG) are allowed up to a size of 3 MB.
Maybe you’re mixing that up with GitHub issues.
Ok fair enough. Here are the links to 2 gists, that contain a total of 5 txt files for troubleshooting. Also to this reply I will attach 3 .JPGs.
part 1 of 2
Part 2 of 2
[2018-06-12 08:52:56,219][ERROR][index.engine ] [graylog] [graylog_1][0] failed to merge java.io.IOException: No space left on device
From here: Files for Graylog / Metricbeat Troubleshooting Part 1 of 2 · GitHub
Thanks for the update. That error was related to an older issue where /var had filled up and had no more disk space. That has since been rectified and the index rotation in graylog has been tweaked to rotate by size rather than number of messages and /var currently has 10+GB free. I should not have posted the elasticsearch log since its latest entry is much older than the most recent attempt to send metricbeat data into graylog.
Is there any load-balancer or reverse proxy between your Beats and the Graylog inputs?
2018-06-12T14:14:21.640-07:00 ERROR [NettyTransport] Error in Input [Beats/5b203386cb28d91ab691cfc8] (channel [id: 0xabdace1e, /162.70.26.89:58090 :> /162.70.27.206:15043]) java.lang.Exception: Unknown beats protocol version: 69
No these are simply two machines that live in the same domain and reside on the same subnet. There are no load balancer or reverse proxy between them. Can you confirm that when data does come in, I should see it under System > Inputs > Beats Input > Show Received Messages?
FYI: Easiest way to see if an Input is receiving anything at all is the stats part on the right of an input settings field in the UI.
Greetings,
Philipp
Hi Philipp, thanks for the reply! I do see some small activity in the status part next to the input settings but its not a lot. I am just confused as to where I will see the actual metrics of the machine that is sending in data, like cpu disk etc… I haven’t even been able to google an image of it online at this point. So I am not sure if it would show under received messages or if it would show elsewhere like for the sidecar-collectors under System > Collectors. Ive attached screenshots of the Input status and one of our current sidecar collectors which shows some slight system metrics.
Thanks to all for your continued help on this.
Hi guys, any updates here? And can someone please confirm where I would expect to see the metricbeat data once it does start coming in?
The data points would appear as normal messages in Graylog.
Thank you, I appreciate the clarification. Is there anything else I can provide to help figure out the issue?
You can try running Metricbeat in debug mode or with verbose logging to see what it’s doing exactly.
https://www.elastic.co/guide/en/beats/metricbeat/6.3/enable-metricbeat-debugging.html
https://www.elastic.co/guide/en/beats/metricbeat/6.3/configuration-logging.html
Ok I am now running metricbeat in full debug mode. Please see the GitHubGist pasted below for a sample of the metricbeat log output. It appears to be sending in data and I do see some data in Beats Input > Show Metrics, but no data points are showing up in Beats Input > Show Received Messages.
I’m unable to reproduce the issue with Graylog 2.4.5 (Docker), Elasticsearch 5.6.10 (Docker), and Metricbeat 6.2.4 (which seems to be what you’re using).
Here’s all the information you would need to reproduce:
Beats input configuration
bind_address: 0.0.0.0
override_source: <empty>
port: 5044
recv_buffer_size: 1048576
tcp_keepalive: false
tls_cert_file: <empty>
tls_client_auth: disabled
tls_client_auth_cert_file: <empty>
tls_enable: false
tls_key_file: <empty>
tls_key_password: ********
Metricbeat configuration
#========================== Modules configuration ============================
metricbeat.config.modules:
# Glob pattern for configuration loading
path: ${path.config}/modules.d/*.yml
# Set to true to enable config reloading
reload.enabled: false
# Period on which files under path should be checked for changes
#reload.period: 10s
metricbeat.modules:
#------------------------------- System Module -------------------------------
- module: system
metricsets:
- cpu # CPU usage
- load # CPU load averages
- memory # Memory usage
- uptime # System Uptime
enabled: true
period: 10s
processes: ['.*']
#================================ General =====================================
# The name of the shipper that publishes the network data. It can be used to group
# all the transactions sent by a single shipper in the web interface.
#name:
# The tags of the shipper are included in their own field with each
# transaction published.
#tags: ["service-X", "web-tier"]
# Optional fields that you can specify to add additional information to the
# output.
#fields:
# env: staging
#================================ Outputs =====================================
#----------------------------- Logstash output --------------------------------
output.logstash:
# The Logstash hosts
hosts: ["localhost:5044"]
# Optional SSL. By default is off.
# List of root certificates for HTTPS server verifications
#ssl.certificate_authorities: ["/etc/pki/root/ca.pem"]
# Certificate for SSL client authentication
#ssl.certificate: "/etc/pki/client/cert.pem"
# Client Certificate Key
#ssl.key: "/etc/pki/client/cert.key"
Metricbeat configuration test
$ ./metricbeat test config
Config OK
$ ./metricbeat test output
logstash: localhost:5044...
connection...
parse host... OK
dns lookup... OK
addresses: ::1, 127.0.0.1
dial up... OK
TLS... WARN secure connection disabled
talk to server... OK
Metricbeat logs
$ ./metricbeat -v -e
2018-06-19T22:00:03.171+0200 INFO instance/beat.go:468 Home path: [/Users/joschi/Downloads/metricbeat-6.2.4-darwin-x86_64] Config path: [/Users/joschi/Downloads/metricbeat-6.2.4-darwin-x86_64] Data path: [/Users/joschi/Downloads/metricbeat-6.2.4-darwin-x86_64/data] Logs path: [/Users/joschi/Downloads/metricbeat-6.2.4-darwin-x86_64/logs]
2018-06-19T22:00:03.171+0200 INFO instance/beat.go:475 Beat UUID: 7a06eaf6-98b0-47f4-9899-5f5f0966baf0
2018-06-19T22:00:03.171+0200 INFO instance/beat.go:213 Setup Beat: metricbeat; Version: 6.2.4
2018-06-19T22:00:03.172+0200 INFO pipeline/module.go:76 Beat name: joschi-mbp15-lan.lan
2018-06-19T22:00:03.172+0200 INFO instance/beat.go:301 metricbeat start running.
2018-06-19T22:00:03.173+0200 INFO [monitoring] log/log.go:97 Starting metrics logging every 30s
2018-06-19T22:00:03.175+0200 INFO cfgfile/reload.go:127 Config reloader started
2018-06-19T22:00:03.177+0200 INFO cfgfile/reload.go:258 Starting 3 runners ...
2018-06-19T22:00:03.177+0200 INFO cfgfile/reload.go:219 Loading of config files completed.
2018-06-19T22:00:33.178+0200 INFO [monitoring] log/log.go:124 Non-zero metrics in the last 30s {"monitoring": {"metrics": {"beat":{"cpu":{"system":{"ticks":110,"time":110},"total":{"ticks":532,"time":532,"value":532},"user":{"ticks":422,"time":422}},"info":{"ephemeral_id":"b33e4d33-7901-47af-937a-c6e82f98da88","uptime":{"ms":30026}},"memstats":{"gc_next":7506448,"memory_alloc":3771832,"memory_total":408141320,"rss":30531584}},"libbeat":{"config":{"module":{"running":3,"starts":3},"reloads":1},"output":{"events":{"acked":95,"batches":9,"total":95},"read":{"bytes":54},"type":"logstash","write":{"bytes":14242}},"pipeline":{"clients":7,"events":{"active":3,"filtered":1,"published":98,"retry":38,"total":99},"queue":{"acked":95}}},"metricbeat":{"system":{"cpu":{"events":7,"success":7},"filesystem":{"events":9,"success":9},"fsstat":{"events":1,"success":1},"load":{"events":7,"success":7},"memory":{"events":7,"success":7},"network":{"events":39,"success":39},"process":{"events":22,"success":22},"process_summary":{"events":3,"success":3},"uptime":{"events":4,"success":4}}},"system":{"cpu":{"cores":8},"load":{"1":2.5127,"15":2.0269,"5":2.2207,"norm":{"1":0.3141,"15":0.2534,"5":0.2776}}}}}}
2018-06-19T22:01:03.173+0200 INFO [monitoring] log/log.go:124 Non-zero metrics in the last 30s {"monitoring": {"metrics": {"beat":{"cpu":{"system":{"ticks":175,"time":175},"total":{"ticks":853,"time":853,"value":853},"user":{"ticks":678,"time":678}},"info":{"ephemeral_id":"b33e4d33-7901-47af-937a-c6e82f98da88","uptime":{"ms":60024}},"memstats":{"gc_next":7600368,"memory_alloc":5312760,"memory_total":662848344,"rss":143360}},"libbeat":{"config":{"module":{"running":3}},"output":{"events":{"acked":83,"batches":9,"total":83},"read":{"bytes":54},"write":{"bytes":12942}},"pipeline":{"clients":7,"events":{"active":0,"published":80,"total":80},"queue":{"acked":83}}},"metricbeat":{"system":{"cpu":{"events":5,"success":5},"load":{"events":5,"success":5},"memory":{"events":5,"success":5},"network":{"events":39,"success":39},"process":{"events":20,"success":20},"process_summary":{"events":3,"success":3},"uptime":{"events":3,"success":3}}},"system":{"load":{"1":2.2461,"15":2.0181,"5":2.1826,"norm":{"1":0.2808,"15":0.2523,"5":0.2728}}}}}}
2018-06-19T22:01:33.170+0200 INFO [monitoring] log/log.go:124 Non-zero metrics in the last 30s {"monitoring": {"metrics": {"beat":{"cpu":{"system":{"ticks":247,"time":247},"total":{"ticks":1213,"time":1213,"value":1213},"user":{"ticks":966,"time":966}},"info":{"ephemeral_id":"b33e4d33-7901-47af-937a-c6e82f98da88","uptime":{"ms":90022}},"memstats":{"gc_next":7538416,"memory_alloc":3775440,"memory_total":915796616,"rss":81920}},"libbeat":{"config":{"module":{"running":3}},"output":{"events":{"acked":90,"batches":9,"total":90},"read":{"bytes":54},"write":{"bytes":13618}},"pipeline":{"clients":7,"events":{"active":0,"filtered":1,"published":90,"total":91},"queue":{"acked":90}}},"metricbeat":{"system":{"cpu":{"events":6,"success":6},"filesystem":{"events":9,"success":9},"fsstat":{"events":1,"success":1},"load":{"events":6,"success":6},"memory":{"events":6,"success":6},"network":{"events":39,"success":39},"process":{"events":18,"success":18},"process_summary":{"events":3,"success":3},"uptime":{"events":3,"success":3}}},"system":{"load":{"1":2.4492,"15":2.0386,"5":2.2295,"norm":{"1":0.3062,"15":0.2548,"5":0.2787}}}}}}
2018-06-19T22:02:03.170+0200 INFO [monitoring] log/log.go:124 Non-zero metrics in the last 30s {"monitoring": {"metrics": {"beat":{"cpu":{"system":{"ticks":316,"time":316},"total":{"ticks":1568,"time":1568,"value":1568},"user":{"ticks":1252,"time":1252}},"info":{"ephemeral_id":"b33e4d33-7901-47af-937a-c6e82f98da88","uptime":{"ms":120024}},"memstats":{"gc_next":7540912,"memory_alloc":3776736,"memory_total":1167849264,"rss":32768}},"libbeat":{"config":{"module":{"running":3}},"output":{"events":{"acked":81,"batches":9,"total":81},"read":{"bytes":54},"write":{"bytes":12442}},"pipeline":{"clients":7,"events":{"active":0,"published":81,"total":81},"queue":{"acked":81}}},"metricbeat":{"system":{"cpu":{"events":6,"success":6},"load":{"events":6,"success":6},"memory":{"events":6,"success":6},"network":{"events":39,"success":39},"process":{"events":18,"success":18},"process_summary":{"events":3,"success":3},"uptime":{"events":3,"success":3}}},"system":{"load":{"1":2.4258,"15":2.0532,"5":2.252,"norm":{"1":0.3032,"15":0.2567,"5":0.2815}}}}}}
2018-06-19T22:02:33.171+0200 INFO [monitoring] log/log.go:124 Non-zero metrics in the last 30s {"monitoring": {"metrics": {"beat":{"cpu":{"system":{"ticks":388,"time":388},"total":{"ticks":1909,"time":1909,"value":1909},"user":{"ticks":1521,"time":1521}},"info":{"ephemeral_id":"b33e4d33-7901-47af-937a-c6e82f98da88","uptime":{"ms":150025}},"memstats":{"gc_next":7537856,"memory_alloc":3782432,"memory_total":1419999040,"rss":40960}},"libbeat":{"config":{"module":{"running":3}},"output":{"events":{"acked":90,"batches":9,"total":90},"read":{"bytes":54},"write":{"bytes":13602}},"pipeline":{"clients":7,"events":{"active":0,"filtered":1,"published":90,"total":91},"queue":{"acked":90}}},"metricbeat":{"system":{"cpu":{"events":6,"success":6},"filesystem":{"events":9,"success":9},"fsstat":{"events":1,"success":1},"load":{"events":6,"success":6},"memory":{"events":6,"success":6},"network":{"events":39,"success":39},"process":{"events":18,"success":18},"process_summary":{"events":3,"success":3},"uptime":{"events":3,"success":3}}},"system":{"load":{"1":2.4741,"15":2.0679,"5":2.2754,"norm":{"1":0.3093,"15":0.2585,"5":0.2844}}}}}}
2018-06-19T22:03:03.167+0200 INFO [monitoring] log/log.go:124 Non-zero metrics in the last 30s {"monitoring": {"metrics": {"beat":{"cpu":{"system":{"ticks":458,"time":458},"total":{"ticks":2248,"time":2248,"value":2248},"user":{"ticks":1790,"time":1790}},"info":{"ephemeral_id":"b33e4d33-7901-47af-937a-c6e82f98da88","uptime":{"ms":180023}},"memstats":{"gc_next":7619056,"memory_alloc":3825568,"memory_total":1672069208,"rss":16384}},"libbeat":{"config":{"module":{"running":3}},"output":{"events":{"acked":82,"batches":9,"total":82},"read":{"bytes":54},"write":{"bytes":12723}},"pipeline":{"clients":7,"events":{"active":0,"published":82,"total":82},"queue":{"acked":82}}},"metricbeat":{"system":{"cpu":{"events":6,"success":6},"load":{"events":6,"success":6},"memory":{"events":6,"success":6},"network":{"events":39,"success":39},"process":{"events":19,"success":19},"process_summary":{"events":3,"success":3},"uptime":{"events":3,"success":3}}},"system":{"load":{"1":2.2744,"15":2.0649,"5":2.2466,"norm":{"1":0.2843,"15":0.2581,"5":0.2808}}}}}}
2018-06-19T22:03:33.169+0200 INFO [monitoring] log/log.go:124 Non-zero metrics in the last 30s {"monitoring": {"metrics": {"beat":{"cpu":{"system":{"ticks":527,"time":527},"total":{"ticks":2587,"time":2587,"value":2587},"user":{"ticks":2060,"time":2060}},"info":{"ephemeral_id":"b33e4d33-7901-47af-937a-c6e82f98da88","uptime":{"ms":210027}},"memstats":{"gc_next":7625424,"memory_alloc":5331944,"memory_total":1924462480,"rss":28672}},"libbeat":{"config":{"module":{"running":3}},"output":{"events":{"acked":90,"batches":9,"total":90},"read":{"bytes":54},"write":{"bytes":13910}},"pipeline":{"clients":7,"events":{"active":0,"filtered":1,"published":90,"total":91},"queue":{"acked":90}}},"metricbeat":{"system":{"cpu":{"events":6,"success":6},"filesystem":{"events":9,"success":9},"fsstat":{"events":1,"success":1},"load":{"events":6,"success":6},"memory":{"events":6,"success":6},"network":{"events":39,"success":39},"process":{"events":18,"success":18},"process_summary":{"events":3,"success":3},"uptime":{"events":3,"success":3}}},"system":{"load":{"1":2.1597,"15":2.062,"5":2.2212,"norm":{"1":0.27,"15":0.2578,"5":0.2776}}}}}}
2018-06-19T22:04:03.165+0200 INFO [monitoring] log/log.go:124 Non-zero metrics in the last 30s {"monitoring": {"metrics": {"beat":{"cpu":{"system":{"ticks":597,"time":597},"total":{"ticks":2921,"time":2921,"value":2921},"user":{"ticks":2324,"time":2324}},"info":{"ephemeral_id":"b33e4d33-7901-47af-937a-c6e82f98da88","uptime":{"ms":240023}},"memstats":{"gc_next":7560048,"memory_alloc":5294856,"memory_total":2175712000,"rss":28672}},"libbeat":{"config":{"module":{"running":3}},"output":{"events":{"acked":81,"batches":9,"total":81},"read":{"bytes":54},"write":{"bytes":12603}},"pipeline":{"clients":7,"events":{"active":0,"published":81,"total":81},"queue":{"acked":81}}},"metricbeat":{"system":{"cpu":{"events":6,"success":6},"load":{"events":6,"success":6},"memory":{"events":6,"success":6},"network":{"events":39,"success":39},"process":{"events":18,"success":18},"process_summary":{"events":3,"success":3},"uptime":{"events":3,"success":3}}},"system":{"load":{"1":1.9331,"15":2.0479,"5":2.1685,"norm":{"1":0.2416,"15":0.256,"5":0.2711}}}}}}
2018-06-19T22:04:33.165+0200 INFO [monitoring] log/log.go:124 Non-zero metrics in the last 30s {"monitoring": {"metrics": {"beat":{"cpu":{"system":{"ticks":664,"time":664},"total":{"ticks":3260,"time":3260,"value":3260},"user":{"ticks":2596,"time":2596}},"info":{"ephemeral_id":"b33e4d33-7901-47af-937a-c6e82f98da88","uptime":{"ms":270024}},"memstats":{"gc_next":7623024,"memory_alloc":3818032,"memory_total":2427066584,"rss":20480}},"libbeat":{"config":{"module":{"running":3}},"output":{"events":{"acked":90,"batches":9,"total":90},"read":{"bytes":54},"write":{"bytes":13570}},"pipeline":{"clients":7,"events":{"active":0,"filtered":1,"published":90,"total":91},"queue":{"acked":90}}},"metricbeat":{"system":{"cpu":{"events":6,"success":6},"filesystem":{"events":9,"success":9},"fsstat":{"events":1,"success":1},"load":{"events":6,"success":6},"memory":{"events":6,"success":6},"network":{"events":39,"success":39},"process":{"events":18,"success":18},"process_summary":{"events":3,"success":3},"uptime":{"events":3,"success":3}}},"system":{"load":{"1":2.3267,"15":2.0801,"5":2.2451,"norm":{"1":0.2908,"15":0.26,"5":0.2806}}}}}}
2018-06-19T22:05:03.162+0200 INFO [monitoring] log/log.go:124 Non-zero metrics in the last 30s {"monitoring": {"metrics": {"beat":{"cpu":{"system":{"ticks":738,"time":738},"total":{"ticks":3606,"time":3606,"value":3606},"user":{"ticks":2868,"time":2868}},"info":{"ephemeral_id":"b33e4d33-7901-47af-937a-c6e82f98da88","uptime":{"ms":300023}},"memstats":{"gc_next":7567552,"memory_alloc":3792352,"memory_total":2681510624,"rss":24576}},"libbeat":{"config":{"module":{"running":3}},"output":{"events":{"acked":82,"batches":9,"total":82},"read":{"bytes":54},"write":{"bytes":12762}},"pipeline":{"clients":7,"events":{"active":0,"published":82,"total":82},"queue":{"acked":82}}},"metricbeat":{"system":{"cpu":{"events":6,"success":6},"load":{"events":6,"success":6},"memory":{"events":6,"success":6},"network":{"events":39,"success":39},"process":{"events":19,"success":19},"process_summary":{"events":3,"success":3},"uptime":{"events":3,"success":3}}},"system":{"load":{"1":2.4971,"15":2.0996,"5":2.2861,"norm":{"1":0.3121,"15":0.2625,"5":0.2858}}}}}}
2018-06-19T22:05:33.162+0200 INFO [monitoring] log/log.go:124 Non-zero metrics in the last 30s {"monitoring": {"metrics": {"beat":{"cpu":{"system":{"ticks":808,"time":808},"total":{"ticks":3942,"time":3942,"value":3942},"user":{"ticks":3134,"time":3134}},"info":{"ephemeral_id":"b33e4d33-7901-47af-937a-c6e82f98da88","uptime":{"ms":330024}},"memstats":{"gc_next":7577328,"memory_alloc":3803840,"memory_total":2941614416,"rss":-1552384}},"libbeat":{"config":{"module":{"running":3}},"output":{"events":{"acked":94,"batches":9,"total":94},"read":{"bytes":54},"write":{"bytes":14420}},"pipeline":{"clients":7,"events":{"active":0,"filtered":1,"published":94,"total":95},"queue":{"acked":94}}},"metricbeat":{"system":{"cpu":{"events":6,"success":6},"filesystem":{"events":9,"success":9},"fsstat":{"events":1,"success":1},"load":{"events":6,"success":6},"memory":{"events":6,"success":6},"network":{"events":39,"success":39},"process":{"events":22,"success":22},"process_summary":{"events":3,"success":3},"uptime":{"events":3,"success":3}}},"system":{"load":{"1":2.8877,"15":2.1465,"5":2.4009,"norm":{"1":0.361,"15":0.2683,"5":0.3001}}}}}}
2018-06-19T22:06:03.159+0200 INFO [monitoring] log/log.go:124 Non-zero metrics in the last 30s {"monitoring": {"metrics": {"beat":{"cpu":{"system":{"ticks":874,"time":874},"total":{"ticks":4275,"time":4275,"value":4275},"user":{"ticks":3401,"time":3401}},"info":{"ephemeral_id":"b33e4d33-7901-47af-937a-c6e82f98da88","uptime":{"ms":360022}},"memstats":{"gc_next":7577520,"memory_alloc":4065024,"memory_total":3198443952,"rss":12288}},"libbeat":{"config":{"module":{"running":3}},"output":{"events":{"acked":84,"batches":9,"total":84},"read":{"bytes":54},"write":{"bytes":13151}},"pipeline":{"clients":7,"events":{"active":0,"published":84,"total":84},"queue":{"acked":84}}},"metricbeat":{"system":{"cpu":{"events":6,"success":6},"load":{"events":6,"success":6},"memory":{"events":6,"success":6},"network":{"events":39,"success":39},"process":{"events":21,"success":21},"process_summary":{"events":3,"success":3},"uptime":{"events":3,"success":3}}},"system":{"load":{"1":2.269,"15":2.1172,"5":2.2974,"norm":{"1":0.2836,"15":0.2646,"5":0.2872}}}}}}
2018-06-19T22:06:33.161+0200 INFO [monitoring] log/log.go:124 Non-zero metrics in the last 30s {"monitoring": {"metrics": {"beat":{"cpu":{"system":{"ticks":941,"time":941},"total":{"ticks":4622,"time":4622,"value":4622},"user":{"ticks":3681,"time":3681}},"info":{"ephemeral_id":"b33e4d33-7901-47af-937a-c6e82f98da88","uptime":{"ms":390025}},"memstats":{"gc_next":7647312,"memory_alloc":5338840,"memory_total":3451945912,"rss":12288}},"libbeat":{"config":{"module":{"running":3}},"output":{"events":{"acked":95,"batches":9,"total":95},"read":{"bytes":54},"write":{"bytes":14493}},"pipeline":{"clients":7,"events":{"active":0,"filtered":1,"published":95,"total":96},"queue":{"acked":95}}},"metricbeat":{"system":{"cpu":{"events":6,"success":6},"filesystem":{"events":9,"success":9},"fsstat":{"events":1,"success":1},"load":{"events":6,"success":6},"memory":{"events":6,"success":6},"network":{"events":39,"success":39},"process":{"events":23,"success":23},"process_summary":{"events":3,"success":3},"uptime":{"events":3,"success":3}}},"system":{"load":{"1":2.3394,"15":2.1289,"5":2.3145,"norm":{"1":0.2924,"15":0.2661,"5":0.2893}}}}}}
2018-06-19T22:07:03.161+0200 INFO [monitoring] log/log.go:124 Non-zero metrics in the last 30s {"monitoring": {"metrics": {"beat":{"cpu":{"system":{"ticks":1013,"time":1013},"total":{"ticks":4973,"time":4973,"value":4973},"user":{"ticks":3960,"time":3960}},"info":{"ephemeral_id":"b33e4d33-7901-47af-937a-c6e82f98da88","uptime":{"ms":420026}},"memstats":{"gc_next":7581648,"memory_alloc":3797264,"memory_total":3704808240,"rss":20480}},"libbeat":{"config":{"module":{"running":3}},"output":{"events":{"acked":84,"batches":9,"total":84},"read":{"bytes":54},"write":{"bytes":13210}},"pipeline":{"clients":7,"events":{"active":0,"published":84,"total":84},"queue":{"acked":84}}},"metricbeat":{"system":{"cpu":{"events":7,"success":7},"load":{"events":7,"success":7},"memory":{"events":6,"success":6},"network":{"events":39,"success":39},"process":{"events":21,"success":21},"process_summary":{"events":3,"success":3},"uptime":{"events":3,"success":3}}},"system":{"load":{"1":2.0508,"15":2.1055,"5":2.2373,"norm":{"1":0.2563,"15":0.2632,"5":0.2797}}}}}}
^C2018-06-19T22:07:04.155+0200 INFO cfgfile/reload.go:222 Dynamic config reloader stopped
2018-06-19T22:07:04.155+0200 INFO instance/beat.go:308 metricbeat stopped.
2018-06-19T22:07:04.156+0200 INFO [monitoring] log/log.go:132 Total non-zero metrics {"monitoring": {"metrics": {"beat":{"cpu":{"system":{"ticks":1034,"time":1034},"total":{"ticks":5069,"time":5069,"value":5069},"user":{"ticks":4035,"time":4035}},"info":{"ephemeral_id":"b33e4d33-7901-47af-937a-c6e82f98da88","uptime":{"ms":421021}},"memstats":{"gc_next":5718256,"memory_alloc":3248752,"memory_total":3785868328,"rss":29483008}},"libbeat":{"config":{"module":{"running":3,"starts":3},"reloads":1},"output":{"events":{"acked":1221,"batches":126,"total":1221},"read":{"bytes":756},"type":"logstash","write":{"bytes":187688}},"pipeline":{"clients":6,"events":{"active":36,"filtered":8,"published":1257,"retry":38,"total":1265},"queue":{"acked":1221}}},"metricbeat":{"system":{"cpu":{"events":86,"success":86},"filesystem":{"events":72,"success":72},"fsstat":{"events":8,"success":8},"load":{"events":85,"success":85},"memory":{"events":85,"success":85},"network":{"events":559,"success":559},"process":{"events":283,"success":283},"process_summary":{"events":43,"success":43},"uptime":{"events":44,"success":44}}},"system":{"cpu":{"cores":8},"load":{"1":2.0508,"15":2.1055,"5":2.2373,"norm":{"1":0.2563,"15":0.2632,"5":0.2797}}}}}}
2018-06-19T22:07:04.156+0200 INFO [monitoring] log/log.go:133 Uptime: 7m1.022626497s
2018-06-19T22:07:04.156+0200 INFO [monitoring] log/log.go:110 Stopping metrics logging.
Example messages in Graylog
docker-compose.yml
version: '2'
services:
# MongoDB: https://hub.docker.com/_/mongo/
mongo:
image: mongo:3
# Elasticsearch: https://www.elastic.co/guide/en/elasticsearch/reference/5.5/docker.html
elasticsearch:
image: docker.elastic.co/elasticsearch/elasticsearch:5.6.10
environment:
- http.host=0.0.0.0
- discovery.type=single-node
# Disable X-Pack security: https://www.elastic.co/guide/en/elasticsearch/reference/5.6/security-settings.html#general-security-settings
- xpack.security.enabled=false
- bootstrap.memory_lock=true
- "ES_JAVA_OPTS=-Xms512m -Xmx512m"
ulimits:
memlock:
soft: -1
hard: -1
mem_limit: 1g
# Graylog: https://hub.docker.com/r/graylog/graylog/
graylog:
image: graylog/graylog:2.4.5-1
environment:
# CHANGE ME!
- GRAYLOG_PASSWORD_SECRET=somepasswordpepper
# Password: admin
- GRAYLOG_ROOT_PASSWORD_SHA2=8c6976e5b5410415bde908bd4dee15dfb167a9c873fc4bb8a81f6f2ab448a918
- GRAYLOG_WEB_ENDPOINT_URI=http://127.0.0.1:9000/api
- GRAYLOG_MESSAGE_JOURNAL_MAX_SIZE=10gb
links:
- mongo
- elasticsearch
ports:
# Graylog web interface and REST API
- 9000:9000
# Beats input
- 5044:5044
Thanks for attempting to reproduce the issue. I will take this info and work with my team. I really appreciate your time.
This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.