Shipping Java Logs to GrayLog


(Charles ) #1

Hi

How do i ship logs from IBM Was to Graylog
i tried different scenarios , i need help

Kind Regards
Charles


(Jochen) #2

There are various GELF log appender for most of the Java logging frameworks on the Graylog Marketplace:


(Charles ) #3

Thank you , i wanted to use the log shippers without affecting the already deployed applications
please send a guide i can use to configure.Preformatted text


(Jochen) #4

http://docs.graylog.org/en/2.4/pages/collector_sidecar.html


(Charles ) #5

Thank you

please see below

server_url: http://127.0.0.1:9000/api/
update_interval: 10
tls_skip_verify: false
send_status: true
list_log_files:
 - /jira/log/SystemOut.log       -  **is this where i put the source logs?**
node_id: graylog-collector-sidecar
collector_id: file:/etc/graylog/collector-sidecar/collector-id
cache_path: /var/cache/graylog/collector-sidecar
log_path: /var/log/graylog/collector-sidecar
log_rotation_time: 86400
log_max_age: 604800
tags:
    - linux
    - apache
backends:
    - name: nxlog
      enabled: false
      binary_path: /usr/bin/nxlog
      configuration_path: /etc/graylog/collector-sidecar/generated/nxlog.conf
    - name: filebeat
      enabled: true
      binary_path: /usr/bin/filebeat
      configuration_path: /etc/graylog/collector-sidecar/generated/filebeat.yml

(Jochen) #6

The configuration settings of the Graylog Collector Sidecar are described in detail at http://docs.graylog.org/en/2.4/pages/collector_sidecar.html#configuration.

list_log_files
Send a directory listing to Graylog and display it on the host status page, e.g. /var/log. This can also be a list of directories


(Charles ) #7

Thank you,

please see my directory , i want to send one file in the directory


(Jochen) #8

You can select the file from the directory in the Graylog web interface when configuring the collector.

The list_log_files configuration setting only tells Graylog in which directories to look for log files to present you in the web interface.


(Charles ) #9

Thank you very much
where does it read tags from ?


(Jochen) #10

You can define tags in the web interface and set one or more of them in the tags configuration setting in the Graylog Collector Sidecar.


(Charles ) #11

I followed the tutorial, still dont see messages





(Jochen) #12

(Charles ) #13

i am a bit confused

Do i need to install filebeat on the log sending machines?
Or there is filebeat that is embedded in GrayLog

i saw also that Filebeat generated mentions Logstash, should i setup standalone Logstash or is it already packaged with GrayLog?

is Elasticsearch packaged with GrayLog as Well?

i have previously used ELK , please clarify


(Jochen) #14

Yes, although the Graylog Collector Sidecar comes with a version of Filebeat, so that you don’t have to install it manually.

No, “logstash” is the name of the output using the Beats (Lumberjack) protocol which is supported by Graylog.

No, unless you’re using the OVA (virtual appliance).


(Charles ) #15
    file /usr/bin/filebeat from install of collector-sidecar-0.1.5-1.x86_64 conflicts with file from package filebeat-6.2.3-1.x86_64

looks like you cant have collector-sidecar and filebeat in one machine


(Jochen) #16

The Graylog Collector Sidecar comes with Filebeat included.


(Charles ) #17

i can see logs , but where is the messages, dont see the log messages under All Messages


(Charles ) #18

there are too many paths and they confusing and also on the filebeat input on the attached , please clarify

filebeat:
  prospectors:
  - encoding: plain
    exclude_files: []
    fields:
      collector_node_id: graylog-collector-sidecar
      gl2_source_collector: 43a7ac5c-67a9-4505-88b3-250d23200cfc
      type: log
    ignore_older: 0
    paths:
    **- /waslogs/bmglogs             ---path**
    scan_frequency: 10s
    tail_files: true
    type: log
output:
  logstash:
    hosts:
    - localhost:5044
path:
  data: /var/cache/graylog/collector-sidecar/filebeat/data
  logs: /var/log/graylog/collector-sidecar
tags:
- linux
- apache


server_url: http://localhost:9000/api/
update_interval: 10
tls_skip_verify: false
send_status: true
list_log_files:
  **- /waslogs/bmglogs               ---path**
node_id: graylog-collector-sidecar
collector_id: file:/etc/graylog/collector-sidecar/collector-id
cache_path: /var/cache/graylog/collector-sidecar
log_path: /var/log/graylog/collector-sidecar
log_rotation_time: 86400
log_max_age: 604800
tags:
    - linux
    - apache
backends:
    - name: nxlog
      enabled: false
      binary_path: /usr/bin/nxlog
      configuration_path: /etc/graylog/collector-sidecar/generated/nxlog.conf
    - name: filebeat
      enabled: true
      binary_path: /usr/bin/filebeat
      configuration_path: /etc/graylog/collector-sidecar/generated/filebeat.yml


(Charles ) #19

Hi, I am now getting Logs using Filebeat, when creating Dashboard ,
Adding widgets - i cant see the features as detailed don the documentation


(Charles ) #20

managed to get indexing happening, i am experiencing an issue with message chart, quick values, statistics and World Map