# Receiving all Docker traffic to Graylog Docker Instance . . .?

**URL:** <https://community.graylog.org/t/receiving-all-docker-traffic-to-graylog-docker-instance/27645>\
**Category:** Graylog Central (peer support)\
**Created:** [February 8, 2023, 7:36pm UTC](https://community.graylog.org/t/receiving-all-docker-traffic-to-graylog-docker-instance/27645 "2023-02-08T19:36:54Z")\
**Posts on this page:** 14\
**Page:** 1

<div class="post-metadata">

**Author:** ![coder](https://avatars.discourse-cdn.com/v4/letter/c/9de0a6/32.png) [@coder](https://community.graylog.org/u/coder)\
**Post date:** [February 8, 2023, 7:36pm UTC](https://community.graylog.org/t/receiving-all-docker-traffic-to-graylog-docker-instance/27645/1 "2023-02-08T19:36:54Z")

</div>

Hi All,

So, I have GrayLog running in a Docker instance. I’ve successfully tested the ability to send to GELF endpoints over HTTP. Now, I’d like to have it pull all Docker log traffic from other instances (in unrelated containers). I’ve used SigNoz and it does this by default, out of the box.

Can someone point me in the direction of documentation that clearly outlines how to achieve `Any/All Docker Containers` → `Graylog Docker Instance`? I’d like to do this without excluding other sources from reading logs.

Also, I’m going to be using this for a few Node JS applications, so if someone has quality configurations that they want to recommend (e.g. Winston/Pino/Bunyan transports to Graylog), I’m definitely interested).

Best

---

<div class="post-metadata">

**Author:** ![gsmith](https://sea2.discourse-cdn.com/flex016/user_avatar/community.graylog.org/gsmith/32/1222_2.png) [@gsmith](https://community.graylog.org/u/gsmith)\
**Post date:** [February 8, 2023, 11:39pm UTC](https://community.graylog.org/t/receiving-all-docker-traffic-to-graylog-docker-instance/27645/2 "2023-02-08T23:39:10Z")

</div>

Hello && Welcome @coder

I have by using Filebeat /w Graylog Sidecar. Just filebeat would be fine but I perfer to adjust my setting on the Web UI becuase Im lazy 😆

```auto
# Needed for Graylog
fields_under_root: true
fields.collector_node_id: ${sidecar.nodeName}
fields.gl2_source_collector: ${sidecar.nodeId}

filebeat.inputs:
- type: docker
  containers.ids: 
    - '*'
  type: log
output.logstash:
   hosts: ["192.168.1.100:5044"]
path:
  data: /var/lib/graylog-sidecar/collectors/filebeat/data
  logs: /var/lib/graylog-sidecar/collectors/filebeat/log

```

> **[Container input | Filebeat Reference \[8.11\] | Elastic](https://www.elastic.co/guide/en/beats/filebeat/current/filebeat-input-container.html)**

Also logs are normally located here.  
`/var/lib/docker/containers/<container_id>/<container_id>-json.log`

EDIT:  
Not sure about this statement

> [@coder](#):
>
> achieve `Any/All Docker Containers` → `Graylog Docker Instance`?

By Archive are you refering to long term backups of the whole container? If so, I just create a checkpoit or backup from the node the containers are on , normally through Veeam free version.

---

<div class="post-metadata">

**Author:** ![coder](https://avatars.discourse-cdn.com/v4/letter/c/9de0a6/32.png) [@coder](https://community.graylog.org/u/coder)\
**Post date:** [February 9, 2023, 12:00am UTC](https://community.graylog.org/t/receiving-all-docker-traffic-to-graylog-docker-instance/27645/3 "2023-02-09T00:00:18Z")

</div>

Not “archive” – achieve (successfully accomplish 😆 ).

One other caveat – I’m using MacOS, so log files are . . .well, I have no idea where, and I’m honestly not sure how Signoz is reading the logs (other than perhaps accessing `/var/run/docker.sock/` directly).

Would certainly be nice if this process was a bit more straightforward. 🫠

Thanks for the input btw!

---

<div class="post-metadata">

**Author:** ![gsmith](https://sea2.discourse-cdn.com/flex016/user_avatar/community.graylog.org/gsmith/32/1222_2.png) [@gsmith](https://community.graylog.org/u/gsmith)\
**Post date:** [February 9, 2023, 12:02am UTC](https://community.graylog.org/t/receiving-all-docker-traffic-to-graylog-docker-instance/27645/4 "2023-02-09T00:02:41Z")

</div>

Hey

> [@coder](#):
>
> Not “archive” – achieve (successfully accomplish 😆 ).

Sorry HAHA I just woke up

> [@coder](#):
>
> One other caveat – I’m using MacOS, so log files are . .

Oh sorry , I dont touch Mac stuff, tbh apple /Docker here you may or may not get an anwser. But ill ask around for ya

---

<div class="post-metadata">

**Author:** ![gsmith](https://sea2.discourse-cdn.com/flex016/user_avatar/community.graylog.org/gsmith/32/1222_2.png) [@gsmith](https://community.graylog.org/u/gsmith)\
**Post date:** [February 9, 2023, 12:17am UTC](https://community.graylog.org/t/receiving-all-docker-traffic-to-graylog-docker-instance/27645/5 "2023-02-09T00:17:27Z")

</div>

hey

Just an FYI, I found this.

 ![image](https://us1.discourse-cdn.com/flex016/uploads/graylog/original/2X/4/4996cf3a8710b7cc4ef04ff557b67e7ccae34cc4.png)

Here.

> **[Filebeat quick start: installation and configuration | Filebeat Reference...](https://www.elastic.co/guide/en/beats/filebeat/current/filebeat-installation-configuration.html)**

---

<div class="post-metadata">

**Author:** ![coder](https://avatars.discourse-cdn.com/v4/letter/c/9de0a6/32.png) [@coder](https://community.graylog.org/u/coder)\
**Post date:** [February 9, 2023, 12:24am UTC](https://community.graylog.org/t/receiving-all-docker-traffic-to-graylog-docker-instance/27645/6 "2023-02-09T00:24:10Z")

</div>

All good. Yeah, I’m also on M1 ARM, so there’s an extra 🔧

😆

Definitely don’t lose any sleep over it, but any input is appreciated. Thanks again.

---

<div class="post-metadata">

**Author:** ![gsmith](https://sea2.discourse-cdn.com/flex016/user_avatar/community.graylog.org/gsmith/32/1222_2.png) [@gsmith](https://community.graylog.org/u/gsmith)\
**Post date:** [February 9, 2023, 12:26am UTC](https://community.graylog.org/t/receiving-all-docker-traffic-to-graylog-docker-instance/27645/7 "2023-02-09T00:26:48Z")

</div>

Holy cow man 😆 is there anything else we should know?  
Nah, I’m all good challenge accepted 👍

---

<div class="post-metadata">

**Author:** ![gsmith](https://sea2.discourse-cdn.com/flex016/user_avatar/community.graylog.org/gsmith/32/1222_2.png) [@gsmith](https://community.graylog.org/u/gsmith)\
**Post date:** [February 9, 2023, 1:24am UTC](https://community.graylog.org/t/receiving-all-docker-traffic-to-graylog-docker-instance/27645/8 "2023-02-09T01:24:07Z")

</div>

Hey,

> [@coder](#):
>
> SigNoz

So I was digging around on the internet, I did see Nxlog and Filebeat is able to be installed on MacOS M1 Arm chip, but looks like there were some issues (i.e., 2020).

So I looked here…

> **[Collecting Docker container logs | SigNoz](https://signoz.io/docs/userguide/collect_docker_logs/#steps-for-collecting-logs-if-signoz-is-running-on-a-different-host)**
>
> With SigNoz you can collect all your docker container logs and perform different queries on top of it.

> If you have a signoz running on a different host then you will have to run a otel-collector to export logs from your host to the host where SigNoz is running.

Actually researching SigNoz , very simialer in the configurations need like Rsyslog/ Filebeat config.

> **[Logs | SigNoz](https://signoz.io/docs/userguide/logs/#operators-for-parsing-and-manipulating-logs)**
>
> Logs management in SigNoz

After reading over there documention for SigNoz this can be **achieve** NOT **archived** 😆

By using another service to send logs to SigNoz (otel-collector) then shipped out from there to Graylog.

Just different names but the same principle like the rest of the log shippers. TBH SigNoz reminds me of Rsyslog/Rsyslog server.

---

<div class="post-metadata">

**Author:** ![coder](https://avatars.discourse-cdn.com/v4/letter/c/9de0a6/32.png) [@coder](https://community.graylog.org/u/coder)\
**Post date:** [February 9, 2023, 1:46am UTC](https://community.graylog.org/t/receiving-all-docker-traffic-to-graylog-docker-instance/27645/9 "2023-02-09T01:46:12Z")

</div>

Eeeeeeesh! Yeah, I don’t necessarily care to go from Graylog to Signoz. Just need to pick one or the other.

Man, who would have thought the idea of streaming/parsing logs could get so convoluted? I could have written my own logging/metrics dashboard app in the few days I’ve spent looking over these options. 😆

Appreciate those links. I’ll take a closer look.

---

<div class="post-metadata">

**Author:** ![gsmith](https://sea2.discourse-cdn.com/flex016/user_avatar/community.graylog.org/gsmith/32/1222_2.png) [@gsmith](https://community.graylog.org/u/gsmith)\
**Post date:** [February 9, 2023, 1:55am UTC](https://community.graylog.org/t/receiving-all-docker-traffic-to-graylog-docker-instance/27645/10 "2023-02-09T01:55:33Z")

</div>

> [@coder](#):
>
> I don’t necessarily care to go from Graylog to Signoz

I was refering to from Signox to graylog, By using another service to send logs to SigNoz (otel-collector) then to Graylog Im awake now LOL.

`otel-collector --> SigNoz --> Graylog`

Or

`another_log_shipper/s --> Graylog`

> [@coder](#):
>
> Appreciate those links. I’ll take a closer look.

Of cource, 👍 sorry I cant be more help.

---

<div class="post-metadata">

**Author:** ![coder](https://avatars.discourse-cdn.com/v4/letter/c/9de0a6/32.png) [@coder](https://community.graylog.org/u/coder)\
**Post date:** [February 9, 2023, 3:20am UTC](https://community.graylog.org/t/receiving-all-docker-traffic-to-graylog-docker-instance/27645/11 "2023-02-09T03:20:28Z")

</div>

Wow! So I found this [GH post about Vector](https://github.com/techno-tim/techno-tim.github.io/discussions/97).

I’ve actually got it successfully pulling data via Loki atm. And I can use `/var/run/docker.sock` on Linux or MacOS. No need for additional drivers or otherwise!

Nah, you were definitely helpful. Sometimes just having a springboard is all you need! Thanks again!

---

<div class="post-metadata">

**Author:** ![gsmith](https://sea2.discourse-cdn.com/flex016/user_avatar/community.graylog.org/gsmith/32/1222_2.png) [@gsmith](https://community.graylog.org/u/gsmith)\
**Post date:** [February 9, 2023, 5:49am UTC](https://community.graylog.org/t/receiving-all-docker-traffic-to-graylog-docker-instance/27645/12 "2023-02-09T05:49:22Z")

</div>

I use that also 😂

 ![image](https://us1.discourse-cdn.com/flex016/uploads/graylog/original/2X/4/4cf385aa4f553f18626bc87093f564ef0194fdb0.png)

Not only for Graylog metrics, Opensearch, Zabbix, Nextcloud, Bookstack, etc… pretty much with everthing. Good stuff

---

<div class="post-metadata">

**Author:** ![coder](https://avatars.discourse-cdn.com/v4/letter/c/9de0a6/32.png) [@coder](https://community.graylog.org/u/coder)\
**Post date:** [February 9, 2023, 6:41am UTC](https://community.graylog.org/t/receiving-all-docker-traffic-to-graylog-docker-instance/27645/13 "2023-02-09T06:41:14Z")

</div>

Aaaaaaah, very nice! If you have any quality `Vector` “remap” or “transform” techniques you’d like to share, I’m all ears.

I’m in the fortunate position that I can generate logs in any “shape” that I like. I just need to be able to plot data points from them at the end of the day. Currently, I’m submitting nested JSON objects, but they’re getting pushed through as strings. I’d like to be able to pipe an entire field (not label) back to JSON and query on the object if at all possible. 🤔

Anyway, thanks again, and feel free to spam ideas. 🤣

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/flex016/uploads/graylog/original/3X/c/7/c7c09c6b5099570133d6502b83f50ba4430de5b6.png) [@system](https://community.graylog.org/u/system)\
**Post date:** [February 23, 2023, 6:41am UTC](https://community.graylog.org/t/receiving-all-docker-traffic-to-graylog-docker-instance/27645/14 "2023-02-23T06:41:37Z")

</div>

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.