Installing Graylog Collector Sidecar


#1

Hello,
I am installing Graylog and its agent in order to gather windows event logs
I am having issues starting the service. I get the following error:

Failed service action: Failed to start Graylog collector sidecar: The service did not respond to the start or control request in a timely fashion

Besides this error, can I just use winlogbeat instead of Graylog collector sidecar?

Thank you


(Jan Doberstein) #2

He @markinhuszn

the sidecar is just a helper to configure a collector like winlogbeat from within the Graylog UI - so yes you can just use a vanilla winlogbeat to send in windows event logs.

Please keep in mind that you need to configure a ‘logstash’ output in winlogbeat to send data to the beats input in Graylog.


#3

Thanks for the swift reply,
Can I use any version of Logstash/winlogbeat or does it have to be below 6.x like Elasticsearch?
Thanks


(Jan Doberstein) #4

as long as the protocoll does not change all versions should work - but I did not test every release by elastic with Graylog.


#5

Thanks, I decided to stick with Graylog Collector Sidecar.
Would you happen to know why I get this error after installing it and trying to start the service:

Failed service action: Failed to start Graylog collector sidecar: The service did not respond to the start or control request in a timely fashion

So I found out that I had the wrong version of the sidecar collector and went with the 0.1x one. Everything worked. However, I am not getting winlogbeat to work. This is the display on the logs:

winlogbeat2018/08/10 15:37:11.152097 beat.go:635: CRIT Exiting: error loading config file: open C:\Program Files\graylog\collector-sidecar\generated\winlogbeat.yml: The system cannot find the file specified.
Exiting: error loading config file: open C:\Program Files\graylog\collector-sidecar\generated\winlogbeat.yml: The system cannot find the file specified.

if I try to go to the generated folder and access winlogbeat.yml - there is nothing there. Any clues?


(Jan Doberstein) #6

Did you configure via Graylog UI the Winlogbeat and tag it with a tag that is configured on your windows host?

http://docs.graylog.org/en/2.4/pages/collector_sidecar.html#step-by-step-guide


#8

Ok I got the log file to be created.
Quick question: do I need logstash or is that not needed?
because winlogbeat is running but I am not getting any logs, when I check the winlogbeat.yml I see there is an ouput for logstash… hence my question
This is my issue:

Could not load field information

Loading field information failed with status: cannot GET http://IP:9000/api/system/fields (500)

this is the configuration I used on Graylog.


the highlighted black is the IP I used, first I used it on port 5044 which has Beats(logstash) listening to it, then I put 9000 so that it would go to elasticsearch, but I guess that didnt work either.

If I go to the address to where the error says, it says this:

{“message”:“Couldn’t read cluster state for indices graylog_*”,“details”:[]}


#9

New update to that:
I have fixed the previous issue by increase the VM memory and CPU.
Now when I try to log in to Graylog web interface, I am greeted with this messaged:

{"error":{"root_cause":[{"type":"index_not_found_exception","reason":"no such index","resource.type":"index_or_alias","resource.id":"api","index_uuid":"_na_","index":"api"}],"type":"index_not_found_exception","reason":"no such index","resource.type":"index_or_alias","resource.id":"api","index_uuid":"_na_","index":"api"},"status":404}

This is the elasticsearch. If I change the url to not include /api in the url. I get this:

{
  "name" : "EdZNbVO",
  "cluster_name" : "graylog",
  "cluster_uuid" : "qJBgeCrMRxqQQ1WumGD5kw",
  "version" : {
    "number" : "5.6.10",
    "build_hash" : "b727a60",
    "build_date" : "2018-06-06T15:48:34.860Z",
    "build_snapshot" : false,
    "lucene_version" : "6.6.1"
  },
  "tagline" : "You Know, for Search"
}

my initial thoughts was that graylog wasnt running… So I checked it and this is what I get:

● graylog-server.service - Graylog server
   Loaded: loaded (/usr/lib/systemd/system/graylog-server.service; enabled; vend
   Active: activating (auto-restart) (Result: exit-code) since Mon 2018-08-13 15
     Docs: http://docs.graylog.org/
  Process: 3248 ExecStart=/usr/share/graylog-server/bin/graylog-server (code=exited, status=255)
 Main PID: 3248 (code=exited, status=255)

Any clues?

Fixed the Graylog issue by downloading Java 8 again, but still seeing the same thing on Elasticsearch
The elasticsearch issue was because both graylog and elasticsearch was connecting on the same Port, so I just changed it and it worked.


#10

The only issue now is getting these event logs. I am currently getting the logs to the VM where graylog is installed. I can see the logs coming through. But I can not send it to Graylog, maybe I have the wrong logstash config? can someone provide me a graylog logstash config?
Thanks

EDIT: I definitely believe its a config issue

[2018-08-13T16:47:37,207][ERROR][logstash.pipeline        ] Error registering plugin {:pipeline_id=>"main", :plugin=>"#<LogStash::OutputDelegator:0x705b8c04>", :error=>"Unexpected character ('<' (code 60)): expected a valid value (number, String, array, object, 'true', 'false' or 'null')\n at [Source: (byte[])\"<!DOCTYPE html>\n<html>\n  <head>\n    <meta http-equiv=\"X-UA-Compatible\" content=\"IE=edge\">\n    <meta name=\"robots\" content=\"noindex, nofollow\">\n    <meta charset=\"UTF-8\">\n    <title>Graylog Web Interface</title>\n    <link rel=\"shortcut icon\" href=\"/assets/favicon.png\">\n    \n  </head>\n  <body>\n    <script src=\"/config.js\"></script>\n    \n    <script src=\"/assets/vendor.552834c48b86209e305c.js\"></script>\n    \n    <script src=\"/assets/polyfill.0cbba45d8aad71248f6d.js\"></script>\n    \n    <script src=\"\"[truncated 1140 bytes]; line: 1, column: 2]", :thread=>"#<Thread:0x6b15e8df run>"}
[2018-08-13T16:47:37,222][ERROR][logstash.pipeline        ] Pipeline aborted due to error {:pipeline_id=>"main", :exception=>#<LogStash::Json::ParserError: Unexpected character ('<' (code 60)): expected a valid value (number, String, array, object, 'true', 'false' or 'null')

(Jan Doberstein) #11

He, as you drift and described different problems for me it is not clear what your current issue is and what you have finally running now.

As you can see in the Docs ( http://docs.graylog.org/en/2.4/pages/collector_sidecar.html ) no logstash is needed.

If you still have problems, please describe what you have done. Your current configuration and what is not working.

thanks


#12

Yeah, my bad - my issue is that logs are not getting displayed in Graylog.
I am receiving logs in the VM where Graylog is installed, its just not sending to Graylog.
This is the config winlogbeat created:

fields:
  collector_node_id: graylog-collector-sidecar
  gl2_source_collector: c94e4d9c-c6d4-4b62-a5a7-f2a99bec1a8d
output:
  logstash:
    hosts:
    - VM_IP:5044
path:
  data: C:\Program Files\graylog\collector-sidecar\cache\winlogbeat\data
  logs: C:\Program Files\graylog\collector-sidecar\logs
tags:
- windows
- iis
winlogbeat:
  event_logs:
  - name: Application
  - name: System
  - name: Security
  - name: ForwardedEvents

This is what it’s displayed in the Graylog collector:




This is the status for elasticsearch and graylog, they are both running.

this is whats in the server.log for graylog:

this is the elasticsearch graylog.log

And this is what the status of the machine in graylog:

Also, I am forwarding the logs to port 5044, how does graylog know to get logs from that port? would I have to send the logs to port 9000 where I have graylog?

Got it to work. Had to mess with configuration and add input.


(Jan Doberstein) #13

Did you configured a BEATS Input? ( System > Inputs ) that is listening on Port 5044?


#14

no I hadn’t, its fully working now.
Is there a way to ingest CISCO ASA logs?


(Jan Doberstein) #15

sure - use the power of search in this community.

You can send Syslog messages and need to parse that.


#16

Alright, because I have no idea how it works.
I looked at the marketplace and saw some cisco asa extractors, is that all I really need?


#17

I managed to get it to work I think:


is this correct for CISCO ASA logs?
doesn’t look too detailed


(system) #18

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.