# Windows Log ingestion into graylog server

**URL:** <https://community.graylog.org/t/windows-log-ingestion-into-graylog-server/810>\
**Category:** Graylog Central (peer support)\
**Tags:** sidecar, filebeat-windows, basic-configuration\
**Created:** [April 13, 2017, 9:49am UTC](https://community.graylog.org/t/windows-log-ingestion-into-graylog-server/810 "2017-04-13T09:49:51Z")\
**Posts on this page:** 20\
**Page:** 1

<div class="post-metadata">

**Author:** ![v\_2nas](https://sea2.discourse-cdn.com/flex016/user_avatar/community.graylog.org/v_2nas/32/268_2.png) [@v\_2nas](https://community.graylog.org/u/v_2nas)\
**Post date:** [April 13, 2017, 9:49am UTC](https://community.graylog.org/t/windows-log-ingestion-into-graylog-server/810/1 "2017-04-13T09:49:51Z")

</div>

I am absolutely newbee to graylog. I have downloaded the latest 2.2.x OVA appliance of graylog, installed it.  
I have also installed side car to server 2012 r2 x64 bit machine

I have 2 questions.

1. How do i push windows extended logs (microsoft-printservice-operational) logs (specific events, 307 and 805, xml data) into gray log server.

2. How do i push csv files on windows share into graylog server.

any blogpost, pointers, help will be great.

Thanks  
Nav

---

<div class="post-metadata">

**Author:** ![jan](https://sea2.discourse-cdn.com/flex016/user_avatar/community.graylog.org/jan/32/11_2.png) [@jan](https://community.graylog.org/u/jan)\
**Post date:** [April 13, 2017, 11:39am UTC](https://community.graylog.org/t/windows-log-ingestion-into-graylog-server/810/2 "2017-04-13T11:39:00Z")

</div>

Hej Nav,

maybe [this blog posting](https://www.graylog.org/blog/83-back-to-basics-enhance-windows-security-with-sysmon-and-graylog) will give you an idea how to get the different event logs.

The filebeat Input is what you are looking for to get some files, line by line into Graylog.

You might find [the documentation](http://docs.graylog.org/en/2.2/pages/collector_sidecar.html) useful.

/jd

---

<div class="post-metadata">

**Author:** ![v\_2nas](https://sea2.discourse-cdn.com/flex016/user_avatar/community.graylog.org/v_2nas/32/268_2.png) [@v\_2nas](https://community.graylog.org/u/v_2nas)\
**Post date:** [April 25, 2017, 4:39am UTC](https://community.graylog.org/t/windows-log-ingestion-into-graylog-server/810/3 "2017-04-25T04:39:48Z")

</div>

I tried to follow the post, it’s difficult to follow as too much [basic building block] info is on the external links.

I currently focusing on getting CSV file data [generated everday] into the gray. Then i will look into the windows log which are more tricky, at least at my level.

I have managed to install sidecar and the service graylog collector is running now.  
I don’t see default filebeat.yml file under generated folder and error is logged

I have searched for a sample filebeat.yml file and configured it to get started. After i restart the service, i get the following.  
Exiting: error initializing publisher: No outputs are defined. Please define one under the output section.

The folder contains csv files in 2 subfolders, like csv/FolderA/ and csv/FolderB/

filebeat.yml  
prospectors:  
- C:\csv\*\*  
encoding: utf-8  
exclude\_lines: ["^#"]  
exclude\_files: [".zip"]  
ignore\_older: 240h  
registry\_file: “C:/ProgramData/filebeat/registry”

Regards,  
Navdeep

---

<div class="post-metadata">

**Author:** ![billmurrin](https://sea2.discourse-cdn.com/flex016/user_avatar/community.graylog.org/billmurrin/32/42_2.png) [@billmurrin](https://community.graylog.org/u/billmurrin)\
**Post date:** [April 25, 2017, 5:31am UTC](https://community.graylog.org/t/windows-log-ingestion-into-graylog-server/810/4 "2017-04-25T05:31:21Z")

</div>

Configure the output.logstash section of your filebeat.yml to send data to your graylog server IP address. Setup a beats input on your Graylog server. Ensure that you use the same port on both. Beats typically uses TCP port 5044 by default.

---

<div class="post-metadata">

**Author:** ![v\_2nas](https://sea2.discourse-cdn.com/flex016/user_avatar/community.graylog.org/v_2nas/32/268_2.png) [@v\_2nas](https://community.graylog.org/u/v_2nas)\
**Post date:** [April 25, 2017, 7:00am UTC](https://community.graylog.org/t/windows-log-ingestion-into-graylog-server/810/5 "2017-04-25T07:00:37Z")

</div>

I have updated the filebeat.yml with the following  
filebeat:  
prospectors:

- input\_type: log
  - paths:
    - C:/csv/DocumentPrinted/_/_  
encoding: utf-8  
ignore\_older: 240h

output:  
logstash:  
hosts: [“172.30.48.145:5044”]

and now i get this in the logs … last few lines  
Exiting: Error in initing prospector: Invalid input type: log - C:\csv\DocumentPrinted\*  
Exiting: Error in initing prospector: Invalid input type: log - C:\csv\DocumentPrinted\*\*  
Exiting: Error in initing prospector: Invalid input type: log - C:\csv\DocumentPrinted\*\*  
Exiting: Error in initing prospector: Invalid input type: log - C:\csv\DocumentPrinted\*\*  
Exiting: Error in initing prospector: Invalid input type: log - C:\csv\DocumentPrinted\*\*  
Exiting: error loading config file: yaml: line 4: did not find expected   
Exiting: error loading config file: yaml: line 4: did not find expected   
Exiting: error loading config file: yaml: line 4: did not find expected   
Exiting: error loading config file: yaml: line 4: did not find expected   
Exiting: error loading config file: yaml: line 3: did not find expected key  
Exiting: error loading config file: yaml: line 3: did not find expected key

---

<div class="post-metadata">

**Author:** ![v\_2nas](https://sea2.discourse-cdn.com/flex016/user_avatar/community.graylog.org/v_2nas/32/268_2.png) [@v\_2nas](https://community.graylog.org/u/v_2nas)\
**Post date:** [April 25, 2017, 7:37am UTC](https://community.graylog.org/t/windows-log-ingestion-into-graylog-server/810/6 "2017-04-25T07:37:42Z")

</div>

looks like there was some whitespace inthe filebeat.yml file, now the config looks like

filebeat:  
prospectors:

- paths:
  - c:\csv\RenderJobDiag\*.csv
  - c:\csv\DocumentPrinted\*.csv  
input\_type: log  
encoding: utf-8  
ignore\_older: 240h

output:  
logstash:  
hosts: [“172.30.48.145:5044”]  
compression\_level: 3

and i don’t get any error in the logs

so what is the next step from here on. I still don’t see anything on the graylog server. I have configure filebeat collector running on port 5044 on graylog server.

---

<div class="post-metadata">

**Author:** ![v\_2nas](https://sea2.discourse-cdn.com/flex016/user_avatar/community.graylog.org/v_2nas/32/268_2.png) [@v\_2nas](https://community.graylog.org/u/v_2nas)\
**Post date:** [April 25, 2017, 7:53am UTC](https://community.graylog.org/t/windows-log-ingestion-into-graylog-server/810/7 "2017-04-25T07:53:21Z")

</div>

On graylog server, under collectors, i see graylog-sidecar-collector with status failing  
It says  
Status: No configuration found for configured tags!

on client, collector\_sidecar.log states  
time=“2017-04-25T15:49:41+08:00” level=info msg="[RequestConfiguration] No configuration found for configured tags!"

looks i am missing some configuration here.

---

<div class="post-metadata">

**Author:** ![billmurrin](https://sea2.discourse-cdn.com/flex016/user_avatar/community.graylog.org/billmurrin/32/42_2.png) [@billmurrin](https://community.graylog.org/u/billmurrin)\
**Post date:** [April 25, 2017, 7:54am UTC](https://community.graylog.org/t/windows-log-ingestion-into-graylog-server/810/8 "2017-04-25T07:54:21Z")

</div>

Is your beats input in Graylog started?

If you look in your filebeat mybeat logfile on the server shipping the logs, do you see any events being sent (should see that x registry updated) xxxx events sent or something along those lines.

Can you see the connection between the two systems using netstat on the graylog server using something like netstat.

```auto
netstat -antp | grep 5044

```

---

<div class="post-metadata">

**Author:** ![v\_2nas](https://sea2.discourse-cdn.com/flex016/user_avatar/community.graylog.org/v_2nas/32/268_2.png) [@v\_2nas](https://community.graylog.org/u/v_2nas)\
**Post date:** [April 25, 2017, 7:55am UTC](https://community.graylog.org/t/windows-log-ingestion-into-graylog-server/810/9 "2017-04-25T07:55:48Z")

</div>

I checked from the webinterface, it seems to be started  
[![Imgur](https://i.imgur.com/n4D2M3V.png?fb "Imgur") ](https://imgur.com/n4D2M3V)

---

<div class="post-metadata">

**Author:** ![billmurrin](https://sea2.discourse-cdn.com/flex016/user_avatar/community.graylog.org/billmurrin/32/42_2.png) [@billmurrin](https://community.graylog.org/u/billmurrin)\
**Post date:** [April 25, 2017, 7:57am UTC](https://community.graylog.org/t/windows-log-ingestion-into-graylog-server/810/10 "2017-04-25T07:57:15Z")

</div>

> [@v\_2nas](#):
>
> ent, collector\_sidecar.log states  
> time=“2017-04-25T15:49:41+08:00” level=info msg=“[RequestConfiguration] No configuration found for configured tags!”
> 
> looks i am missing some configuration here.

It looks like it has received 183.5kb so far. Can you see the data show up under All Messages? Also try clicking on the Show Received Messages.

---

<div class="post-metadata">

**Author:** ![v\_2nas](https://sea2.discourse-cdn.com/flex016/user_avatar/community.graylog.org/v_2nas/32/268_2.png) [@v\_2nas](https://community.graylog.org/u/v_2nas)\
**Post date:** [April 25, 2017, 8:38am UTC](https://community.graylog.org/t/windows-log-ingestion-into-graylog-server/810/11 "2017-04-25T08:38:04Z")

</div>

It’s empty, do i need to configure collector configurations, i suspect problem is somewhere here.  
[![Imgur](https://i.imgur.com/Z8kzIQ3.png?fb "Imgur") ](https://imgur.com/Z8kzIQ3)

I tried to follow this article, [http://docs.graylog.org/en/2.2/pages/collector\_sidecar.html#sidecar-step-by-step](http://docs.graylog.org/en/2.2/pages/collector_sidecar.html#sidecar-step-by-step)  
however i am not sure if i have got output part right.

---

<div class="post-metadata">

**Author:** ![billmurrin](https://sea2.discourse-cdn.com/flex016/user_avatar/community.graylog.org/billmurrin/32/42_2.png) [@billmurrin](https://community.graylog.org/u/billmurrin)\
**Post date:** [April 25, 2017, 8:59am UTC](https://community.graylog.org/t/windows-log-ingestion-into-graylog-server/810/12 "2017-04-25T08:59:58Z")

</div>

I haven’t used SideCar Collector so I can’t say, but you should be able to get filebeat sending to Graylog without Collector even being involved. Try to minimize the number of variables involved to just get it working, Once it is working, add collector so you can manage filebeat on your server remotely.

By default, filebeat will send its logs in Windows to C:\ProgramData\<beat-name\>\Logs. Check your mybeat file to see if it shows anything being sent.

---

<div class="post-metadata">

**Author:** ![v\_2nas](https://sea2.discourse-cdn.com/flex016/user_avatar/community.graylog.org/v_2nas/32/268_2.png) [@v\_2nas](https://community.graylog.org/u/v_2nas)\
**Post date:** [April 25, 2017, 9:13am UTC](https://community.graylog.org/t/windows-log-ingestion-into-graylog-server/810/13 "2017-04-25T09:13:08Z")

</div>

i think i am near to the fix but not so near. Do you know how can we specify multiple paths in input under configuration

I tried this but getting error.  
[‘C:\csv\DocumentPrinted\*.csv’], [‘C:\csv\RenderJobDiag\*.csv’]

---

<div class="post-metadata">

**Author:** ![billmurrin](https://sea2.discourse-cdn.com/flex016/user_avatar/community.graylog.org/billmurrin/32/42_2.png) [@billmurrin](https://community.graylog.org/u/billmurrin)\
**Post date:** [April 25, 2017, 9:16am UTC](https://community.graylog.org/t/windows-log-ingestion-into-graylog-server/810/14 "2017-04-25T09:16:17Z")

</div>

> [@v\_2nas](#):
>
> - paths:
> - c:\csv\RenderJobDiag\*.csv
> - c:\csv\DocumentPrinted\*.csv

Check this out. [Configure inputs | Filebeat Reference [8.11] | Elastic](https://www.elastic.co/guide/en/beats/filebeat/current/configuration-filebeat-options.html)

---

<div class="post-metadata">

**Author:** ![v\_2nas](https://sea2.discourse-cdn.com/flex016/user_avatar/community.graylog.org/v_2nas/32/268_2.png) [@v\_2nas](https://community.graylog.org/u/v_2nas)\
**Post date:** [April 25, 2017, 9:18am UTC](https://community.graylog.org/t/windows-log-ingestion-into-graylog-server/810/15 "2017-04-25T09:18:05Z")

</div>

i checked that, but now the filebeat.yml file is being updated via graylog server through webinterface.  
i suspect the problem lies somwhere between inputs and outputs.

---

<div class="post-metadata">

**Author:** ![v\_2nas](https://sea2.discourse-cdn.com/flex016/user_avatar/community.graylog.org/v_2nas/32/268_2.png) [@v\_2nas](https://community.graylog.org/u/v_2nas)\
**Post date:** [April 25, 2017, 9:46am UTC](https://community.graylog.org/t/windows-log-ingestion-into-graylog-server/810/16 "2017-04-25T09:46:30Z")

</div>

does this filebeat.yml content looks fine to you?  
filebeat:  
prospectors:

- document\_type: log  
encoding: plain  
fields:  
gl2\_source\_collector: 25e87973-306f-4169-9dea-2144b5448206  
ignore\_older: 4800h  
input\_type: log  
paths:
  - C:\csv\DocumentPrinted\*.csv  
scan\_frequency: 10s  
tail\_files: true  
output:  
logstash:  
hosts:
  - 172.30.48.145:5044  
path:  
data: C:\Program Files\graylog\collector-sidecar\cache\filebeat\data  
logs: C:\Program Files\graylog\collector-sidecar\logs  
tags:

- windows
- csv

it’s auto updated.

---

<div class="post-metadata">

**Author:** ![v\_2nas](https://sea2.discourse-cdn.com/flex016/user_avatar/community.graylog.org/v_2nas/32/268_2.png) [@v\_2nas](https://community.graylog.org/u/v_2nas)\
**Post date:** [April 25, 2017, 10:11am UTC](https://community.graylog.org/t/windows-log-ingestion-into-graylog-server/810/17 "2017-04-25T10:11:19Z")

</div>

do i need to create system\>outputs type gelf ?  
I have create inputs and outputs under system\>collector\> manage configuration.

output is set to filebeat with graylog server ip and 5044 port  
input is set to filebeat with output create above

---

<div class="post-metadata">

**Author:** ![billmurrin](https://sea2.discourse-cdn.com/flex016/user_avatar/community.graylog.org/billmurrin/32/42_2.png) [@billmurrin](https://community.graylog.org/u/billmurrin)\
**Post date:** [April 25, 2017, 10:16am UTC](https://community.graylog.org/t/windows-log-ingestion-into-graylog-server/810/18 "2017-04-25T10:16:52Z")

</div>

> [@v\_2nas](#):
>
> eate system\>outputs type gelf ?  
> I have create inputs and outputs under system\>collector\> manage configuration.
> 
> output is set to filebeat with graylog server ip and 5044 port

No, an output would be if you wanted to send data from Graylog to somewhere else. You are working on getting an input working. Can you try to post a chunk of your filebeat log file. Please encapsulate in ``` code ``` so it gets rendered properly. Your filebeat logs might be under C:\Program Files\graylog\collector-sidecar\logs

---

<div class="post-metadata">

**Author:** ![v\_2nas](https://sea2.discourse-cdn.com/flex016/user_avatar/community.graylog.org/v_2nas/32/268_2.png) [@v\_2nas](https://community.graylog.org/u/v_2nas)\
**Post date:** [April 25, 2017, 10:22am UTC](https://community.graylog.org/t/windows-log-ingestion-into-graylog-server/810/19 "2017-04-25T10:22:13Z")

</div>

File beat log is empty, i was getting too many errors when troubleshooting, so i cleared them after success. I haven’t seen any logs populating in the logs file after that. I have restarted the service few times.  
[![Imgur](https://i.imgur.com/OfMMxUe.png?fb "Imgur") ](https://imgur.com/OfMMxUe)

---

<div class="post-metadata">

**Author:** ![billmurrin](https://sea2.discourse-cdn.com/flex016/user_avatar/community.graylog.org/billmurrin/32/42_2.png) [@billmurrin](https://community.graylog.org/u/billmurrin)\
**Post date:** [April 25, 2017, 11:02am UTC](https://community.graylog.org/t/windows-log-ingestion-into-graylog-server/810/20 "2017-04-25T11:02:26Z")

</div>

Is that the only log you have for filebeat?

[Next page](https://community.graylog.org/t/windows-log-ingestion-into-graylog-server/810.md?page=2)