# Graylog - Sidecar question

**URL:** <https://community.graylog.org/t/graylog-sidecar-question/21199>\
**Category:** Graylog Central (peer support)\
**Tags:** sidecar\
**Created:** [September 15, 2021, 1:55pm UTC](https://community.graylog.org/t/graylog-sidecar-question/21199 "2021-09-15T13:55:57Z")\
**Posts on this page:** 8\
**Page:** 1

<div class="post-metadata">

**Author:** ![Oirbsiu](https://sea2.discourse-cdn.com/flex016/user_avatar/community.graylog.org/oirbsiu/32/8712_2.png) [@Oirbsiu](https://community.graylog.org/u/Oirbsiu)\
**Post date:** [September 15, 2021, 1:55pm UTC](https://community.graylog.org/t/graylog-sidecar-question/21199/1 "2021-09-15T13:55:57Z")

</div>

I have been using Graylog with a locally installed sidecar - to ingest logfiles taken from an server that is no longer in use.  
The files were ingested as raw messages, however I have since found out that I can process the file as a CSV (decode\_csv).

I wanted to clear the data this sidecar has harvested,  
stopped Filebeat,  
removed the files in /var/lib/graylog-sidecar/collectors/filebeat/  
rotated and removed old indexes

The messages are no longer visible in graylog but when I access system / sidecars  
and click on my sidecar to view the status, the screen Im shown takes ages to load - the files are still referenced.

 ![sidecar](https://us1.discourse-cdn.com/flex016/uploads/graylog/original/2X/7/70bf2a9f6ab922a486fe805d361694e6e1df2d25.png)

How do I clear this and how do I reduce list of files shown?  
Mark

---

<div class="post-metadata">

**Author:** ![gsmith](https://sea2.discourse-cdn.com/flex016/user_avatar/community.graylog.org/gsmith/32/1222_2.png) [@gsmith](https://community.graylog.org/u/gsmith)\
**Post date:** [September 16, 2021, 1:49am UTC](https://community.graylog.org/t/graylog-sidecar-question/21199/3 "2021-09-16T01:49:36Z")

</div>

Hello,

> [@Oirbsiu](#):
>
> How do I clear this and how do I reduce list of files shown?  
> Mark

Navigate to `System/sidecar/Adminstator` highlight then Clear selection.

 ![image](https://us1.discourse-cdn.com/flex016/uploads/graylog/original/2X/2/23d0e064914af3e2b9f98947785bf95da36bced2.png)

You will be greeted with the following

 ![image](https://us1.discourse-cdn.com/flex016/uploads/graylog/original/2X/c/cd51c66f3ebe7febeb0f534625aa64feea7ed896.png)

Should be good, Might have to refresh your browser.

Hope that helps

---

<div class="post-metadata">

**Author:** ![Oirbsiu](https://sea2.discourse-cdn.com/flex016/user_avatar/community.graylog.org/oirbsiu/32/8712_2.png) [@Oirbsiu](https://community.graylog.org/u/Oirbsiu)\
**Post date:** [September 16, 2021, 1:57pm UTC](https://community.graylog.org/t/graylog-sidecar-question/21199/4 "2021-09-16T13:57:12Z")

</div>

Thanks for your reply,  
I managed to work out what was wrong.  
In my /etc/graylog/sidecar/sidecar.yml file I had uncommented list\_log\_files and put some entries in between the square brackets.

```auto
#
# Default: empty list
#list_log_files: []

```

I am new to Graylog and this was where I thought you tell the sidecar to harvest logs from - i know now that this was wrong, I had forgotten that I had made this change and once I removed it, the lists have gone.

---

<div class="post-metadata">

**Author:** ![Oirbsiu](https://sea2.discourse-cdn.com/flex016/user_avatar/community.graylog.org/oirbsiu/32/8712_2.png) [@Oirbsiu](https://community.graylog.org/u/Oirbsiu)\
**Post date:** [September 16, 2021, 2:09pm UTC](https://community.graylog.org/t/graylog-sidecar-question/21199/5 "2021-09-16T14:09:41Z")

</div>

I thought everything was working until i spotted that the dates from the raw messages are being parsed.

my raw messages

```auto
Interim-Update,user5@realm,2021/09/16 14:42:32,0,HOST-1,3174790000001AA1431119,10.10.110.112,Framed-User,PPP,,,10.20.126.116,255.255.255.255,,,,122800,,124584,,1350,,1344,PPPoEoQinQ,180056165,lag-10:999.101,Radius,

```

all messages are pushed though “decode\_csv\_fields”  
e.g.

```auto
processors:
  - decode_csv_fields:
      fields:
        message: csv
      overwrite_keys: true

  - extract_array:
      field: csv
      overwrite_keys: true
      omit_empty: true
      mappings:
        Acct-Status-Type: 0
        User-Name: 1
        Event_Timestamp: 2
        Acct-Delay-Time: 3
        NAS-Identifier: 4
        Acct-Session-Id: 5
        NAS-IP-Address: 6
        Service-Type: 7
        Framed-Protocol: 8
        Framed-Compression: 9
        Unisphere-PPPoE-Description: 10
        Framed-IP-Address: 11
        Framed-IP-Netmask: 12
        Unisphere-Ingress-Policy-Name: 13
        Calling-Station-Id: 14
        Acct-Input-Gigawords: 15
        Acct-Input-Octets: 16
        Acct-Output-Gigawords: 17
        Acct-Output-Octets: 18
        Unisphere-Input-Gigapackets: 19
        Acct-Input-Packets: 20
        Unisphere-Output-Gigapackets: 21
        Acct-Output-Packets: 22
        NAS-Port-Type: 23
        NAS-Port: 24
        NAS-Port-Id: 25
        Acct-Authentic: 26
        Acct-Session-Time: 27
        Delegated-IPv6-Prefix: 28

```

and these are split nicely in to the fields that I can search

what I was not expecting is that the field Event\_Timestamp is being parsed and instead of showing as

```auto
2021-09-16 14:42:32.000 +01:00

```

its

```auto
2021-09-16 15:42:32.000 +01:00

```

how can I prevent this and at the same time, how can I overwrite timestamp with this field value?  
thanks

---

<div class="post-metadata">

**Author:** ![gsmith](https://sea2.discourse-cdn.com/flex016/user_avatar/community.graylog.org/gsmith/32/1222_2.png) [@gsmith](https://community.graylog.org/u/gsmith)\
**Post date:** [September 16, 2021, 10:19pm UTC](https://community.graylog.org/t/graylog-sidecar-question/21199/6 "2021-09-16T22:19:17Z")

</div>

Hello,  
This is kind of a broad question. what I would do is check the following first.

- Does the device sending logs have the correct Date/Time.
- Graylog server have the correct time (NTP) and configuration file `root_timezone =` is correct.
- Insure that the user logged in has the correct Time settings.

 ![image](https://us1.discourse-cdn.com/flex016/uploads/graylog/original/2X/e/ec7ee03c2c95a0213d73ecd94f06d7b4ec6f96e4.png)

- check under **System/Overview** in the Web UI, these should be the same.

 ![image](https://us1.discourse-cdn.com/flex016/uploads/graylog/original/2X/3/37cca9b255ff992c59f742854926e835cd4c4952.png)

EDIT:

> [@Oirbsiu](#):
>
> In my /etc/graylog/sidecar/sidecar.yml file I had uncommented list\_log\_files and put some entries in

My apologies, I missed read your statement. I though you want to remove the collector configuration.

Hope that helps.

---

<div class="post-metadata">

**Author:** ![Oirbsiu](https://sea2.discourse-cdn.com/flex016/user_avatar/community.graylog.org/oirbsiu/32/8712_2.png) [@Oirbsiu](https://community.graylog.org/u/Oirbsiu)\
**Post date:** [September 17, 2021, 8:34am UTC](https://community.graylog.org/t/graylog-sidecar-question/21199/7 "2021-09-17T08:34:00Z")

</div>

So this has me puzzled -

I have tried this with and without a timezone set in my profile  
with I use isle\_of\_man which is basically London BST (GMT+1)

![Inkedtime_LI](https://us1.discourse-cdn.com/flex016/uploads/graylog/original/2X/2/22c24dfabb4c3795208281de280c50bfd06bdf0e.jpeg)

If I view any entry from either as

`/messages/graylog_7/a8758d31-1788-11ec-8577-ac162d8befa8`  
or  
`/api/messages/graylog_7/a8758d31-1788-11ec-8577-ac162d8befa8`

the value I see presented in the Event\_timestamps as it should be

e.g.

 ![Capture1](https://us1.discourse-cdn.com/flex016/uploads/graylog/original/2X/c/cd3c6da95a270b76acff837b7ede5d9a62ad4502.png)

yet the same entry when viewed from the default dashboard, the time that is shown for Event\_timestamp is 1hour ahead…

 ![Capture2](https://us1.discourse-cdn.com/flex016/uploads/graylog/original/2X/a/a769d42056cb069e014f53a35cd82943b1bddb01.png)

and what is even more confusing, is that the @ timestamp field is correct when viewing it from the default dashboard but and hour behind when viewing just that message.

is this a bug?

---

<div class="post-metadata">

**Author:** ![gsmith](https://sea2.discourse-cdn.com/flex016/user_avatar/community.graylog.org/gsmith/32/1222_2.png) [@gsmith](https://community.graylog.org/u/gsmith)\
**Post date:** [September 18, 2021, 12:16am UTC](https://community.graylog.org/t/graylog-sidecar-question/21199/8 "2021-09-18T00:16:06Z")

</div>

> [@Oirbsiu](#):
>
> is this a bug?

Not that I Know of. Elasticsearch stores all “time” fields in UTC/Epoch. I would double check your times/Date on each device and to make sure there not a problem.

```auto
TimeStamp 2021-09-17 08:27;05.496
@timestamp 2021-09-17T07:27:04.088Z
Event_Timestamp 2021/09/17/ 08:26:54

```

You have something crazy going on, not sure what.

Maybe some of these post will help

> [@Diferent Timestamp Webpage and server](https://community.graylog.org/t/diferent-timestamp-webpage-and-server/13511/2):
>
> Graylog saves all dates in UTC format, and shows it in GUI in timezone configured for particular user. For user admin, it uses timezone defined in /etc/graylog/server/server.conf file, parameter root\_timezone. root\_timezone = Europe/Bratislava Change Europe/Bratislava to your real timezone, and restart graylog server. [https://docs.graylog.org/en/3.1/pages/configuration/server.conf.html#general](https://docs.graylog.org/en/3.1/pages/configuration/server.conf.html#general) For other user accounts other than admin, configuration is done in GUI. Every user can change t…

[https://community.graylog.org/search?q=Timestamp](https://community.graylog.org/search?q=Timestamp)

> **[Filebeat incorrect timestamp](https://discuss.elastic.co/t/filebeat-incorrect-timestamp/189009/3)**
>
> Thanks for the answer, the raw logs received in to ES were in my local timezone, but ES what I think was happening was ES assumed they were UTC and added +2 hours when storing them which meant I could not see any data unless I moved the time scale...

> <https://github.com/Graylog2/collector-sidecar/issues/231>
>
> \### Problem description
> When logs are sent to graylog, they are not sorted acco…rding to time, but have errors.
> \### Steps to reproduce the problem
> 
> 1. look
> !\[graylog\_time\](https://user-images.githubusercontent.com/6148257/38787874-36c80b44-4163-11e8-91fa-174e90b57f44.png)
> 
> 
> who can help me,thanks very mach!!!
> \### Environment
> 
> \* Sidecar Version:collector-sidecar-0.1.4-1.x86\_64
> \* Graylog Version:Graylog 2.4.3+2c41897 
> \* Operating System:Centos7
> \* Elasticsearch Version:elasticsearch-5.6.3
> \* MongoDB Version:

**EDIT** : I read over this post trying to figure out an awswer for you about the hour difference.  
Just realize something. If Elasticsearch stores all “time” fields in UTC/Epoch ‎and UTC · ‎Greenwich Mean Time (GMT) . Then your statement was

> [@Oirbsiu](#):
>
> I use isle\_of\_man which is basically London BST (GMT+1)

There is your hour difference. I’m not 100% sure but a pipeline might be in your future to set these timestamps correct.

 ![image](https://us1.discourse-cdn.com/flex016/uploads/graylog/original/2X/7/7b81b258069f32a0eaae91ff8da81b169adb7cfa.png)

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/flex016/uploads/graylog/original/3X/c/7/c7c09c6b5099570133d6502b83f50ba4430de5b6.png) [@system](https://community.graylog.org/u/system)\
**Post date:** [October 2, 2021, 12:16am UTC](https://community.graylog.org/t/graylog-sidecar-question/21199/9 "2021-10-02T00:16:59Z")

</div>

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.