# Sidecar Collector Connection to Graylog Cluster

**URL:** <https://community.graylog.org/t/sidecar-collector-connection-to-graylog-cluster/1184>\
**Category:** Graylog Central (peer support)\
**Tags:** sidecar\
**Created:** [May 19, 2017, 6:39am UTC](https://community.graylog.org/t/sidecar-collector-connection-to-graylog-cluster/1184 "2017-05-19T06:39:57Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![drstaind](https://avatars.discourse-cdn.com/v4/letter/d/919ad9/32.png) [@drstaind](https://community.graylog.org/u/drstaind)\
**Post date:** [May 19, 2017, 6:39am UTC](https://community.graylog.org/t/sidecar-collector-connection-to-graylog-cluster/1184/1 "2017-05-19T06:39:57Z")

</div>

Hi All,

Experiencing some difficulty in gaining a successful connection from the Sidecar Collector to a Graylog Cluster.  
The cluster itself is working fine. No firewalld or iptables have been enabled.

**Brief overview of the setup**

10.40.1.230 _- ClusterControl server + Graylog web UI + HAProxy_  
10.40.1.231 _- Graylog server + MongoDB Replica Set + ElasticSearch_  
10.40.1.232 _- Graylog server + MongoDB Replica Set + ElasticSearch_  
10.40.1.233 _- Graylog server + MongoDB Replica Set + ElasticSearch_

Graylog Version = _2.2.3_  
Sidecar Collector Version = _0.1.1-1_  
Graylog OS = _centos-release-7-3.1611.el7.centos.x86\_64_  
Sidecar Collector OS = _WinServer2k16r2_

**Configuration of each graylog server** _(/etc/graylog/server/server.conf)_

```
rest_listen_uri = http://0.0.0.0:12900/
#rest_transport_uri = http://0.0.0.0:12900/

```

**Configuration of the graylog web server** _(/etc/graylog/web/web.conf)_

```
graylog2-server.uris="http://10.40.1.231:12900/,http://10.40.1.232:12900/,http://10.40.1.233:12900/"

```

Web URL = [http://10.40.1.230:9000/](http://10.40.1.230:9000/)

I’ve previously had single instance Graylog nodes (non-cluster) working with the sidecar collector without issue, however my inability to connect to the api to establish a Collector has me scratching my head a little. After doing some research, most people were able to resolve by modifying their respective configurations between :9000 and :12900, with appending /api and without. I thought i’d do the same thing.

**Attempting to send to the graylog server instance:**

**Attempt 1:**

Configuration of Sidecar-Collector (C:\Program Files\Graylog\collector-sidecar\collector\_sidecar.yml)

```
server_url: http://10.40.1.231:12900/

```

Result:

```
time="2017-05-19T16:15:12+10:00" level=error msg="[RequestConfiguration] Bad response status from Graylog server: 404 Not Found" 
time="2017-05-19T16:15:12+10:00" level=error msg="Can't fetch configuration from Graylog API: GET http://10.40.1.231:12900/plugins/org.graylog.plugins.collector/925b5261-6142-44ca-b96c-a292d0503a04?tags=%5B%22windows%22%2C%22iis%22%5D: 404 HTTP 404 Not Found" 
time="2017-05-19T16:15:12+10:00" level=error msg="[filebeat] Unable to start collector after 3 tries, giving up!" 
time="2017-05-19T16:15:13+10:00" level=error msg="[UpdateRegistration] Bad response from Graylog server: 404 Not Found" 
time="2017-05-19T16:15:22+10:00" level=error msg="[RequestConfiguration] Bad response status from Graylog server: 404 Not Found" 
time="2017-05-19T16:15:22+10:00" level=error msg="Can't fetch configuration from Graylog API: GET http://10.40.1.231:12900/plugins/org.graylog.plugins.collector/925b5261-6142-44ca-b96c-a292d0503a04?tags=%5B%22windows%22%2C%22iis%22%5D: 404 HTTP 404 Not Found" 
time="2017-05-19T16:15:24+10:00" level=error msg="[UpdateRegistration] Bad response from Graylog server: 404 Not Found" 

```

**Attempt 2:**

Configuration of Sidecar-Collector (C:\Program Files\Graylog\collector-sidecar\collector\_sidecar.yml)

```
server_url: http://10.40.1.231:12900/api

```

Result:

```
time="2017-05-19T16:17:02+10:00" level=error msg="[UpdateRegistration] Bad response from Graylog server: 404 Not Found" 
time="2017-05-19T16:17:12+10:00" level=error msg="[RequestConfiguration] Bad response status from Graylog server: 404 Not Found" 
time="2017-05-19T16:17:12+10:00" level=error msg="Can't fetch configuration from Graylog API: GET http://10.40.1.231:12900/api/plugins/org.graylog.plugins.collector/925b5261-6142-44ca-b96c-a292d0503a04?tags=%5B%22windows%22%2C%22iis%22%5D: 404 HTTP 404 Not Found" 
time="2017-05-19T16:17:13+10:00" level=error msg="[UpdateRegistration] Bad response from Graylog server: 404 Not Found" 

```

**Attempt 3:**

Configuration of Sidecar-Collector (C:\Program Files\Graylog\collector-sidecar\collector\_sidecar.yml)

```
server_url: http://10.40.1.231:9000/

```

Result:

```
time="2017-05-19T16:19:04+10:00" level=error msg="[RequestConfiguration] Fetching configuration failed: Get http://10.40.1.231:9000/plugins/org.graylog.plugins.collector/925b5261-6142-44ca-b96c-a292d0503a04?tags=%5B%22windows%22%2C%22iis%22%5D: dial tcp 10.40.1.231:9000: connectex: No connection could be made because the target machine actively refused it." 
time="2017-05-19T16:19:04+10:00" level=error msg="[UpdateRegistration] Failed to report collector status to server: Put http://10.40.1.231:9000/plugins/org.graylog.plugins.collector/collectors/925b5261-6142-44ca-b96c-a292d0503a04: dial tcp 10.40.1.231:9000: connectex: No connection could be made because the target machine actively refused it." 
time="2017-05-19T16:19:15+10:00" level=error msg="[RequestConfiguration] Fetching configuration failed: Get http://10.40.1.231:9000/plugins/org.graylog.plugins.collector/925b5261-6142-44ca-b96c-a292d0503a04?tags=%5B%22windows%22%2C%22iis%22%5D: dial tcp 10.40.1.231:9000: connectex: No connection could be made because the target machine actively refused it." 
time="2017-05-19T16:19:16+10:00" level=error msg="[UpdateRegistration] Failed to report collector status to server: Put http://10.40.1.231:9000/plugins/org.graylog.plugins.collector/collectors/925b5261-6142-44ca-b96c-a292d0503a04: dial tcp 10.40.1.231:9000: connectex: No connection could be made because the target machine actively refused it." 

```

**Attempt 4:**

Configuration of Sidecar-Collector (C:\Program Files\Graylog\collector-sidecar\collector\_sidecar.yml)

```
server_url: http://10.40.1.231:9000/api

```

Result:

```
time="2017-05-19T16:20:51+10:00" level=error msg="[RequestConfiguration] Fetching configuration failed: Get http://10.40.1.231:9000/api/plugins/org.graylog.plugins.collector/925b5261-6142-44ca-b96c-a292d0503a04?tags=%5B%22windows%22%2C%22iis%22%5D: dial tcp 10.40.1.231:9000: connectex: No connection could be made because the target machine actively refused it." 
time="2017-05-19T16:20:52+10:00" level=error msg="[UpdateRegistration] Failed to report collector status to server: Put http://10.40.1.231:9000/api/plugins/org.graylog.plugins.collector/collectors/925b5261-6142-44ca-b96c-a292d0503a04: dial tcp 10.40.1.231:9000: connectex: No connection could be made because the target machine actively refused it." 
time="2017-05-19T16:21:02+10:00" level=error msg="[RequestConfiguration] Fetching configuration failed: Get http://10.40.1.231:9000/api/plugins/org.graylog.plugins.collector/925b5261-6142-44ca-b96c-a292d0503a04?tags=%5B%22windows%22%2C%22iis%22%5D: dial tcp 10.40.1.231:9000: connectex: No connection could be made because the target machine actively refused it." 
time="2017-05-19T16:21:03+10:00" level=error msg="[UpdateRegistration] Failed to report collector status to server: Put http://10.40.1.231:9000/api/plugins/org.graylog.plugins.collector/collectors/925b5261-6142-44ca-b96c-a292d0503a04: dial tcp 10.40.1.231:9000: connectex: No connection could be made because the target machine actively refused it." 

```

**Attempting to connect to the webserver instance:** (already considered this unlikely)

**Attempt 1:**

Configuration of Sidecar-Collector (C:\Program Files\Graylog\collector-sidecar\collector\_sidecar.yml)

```
server_url: http://10.40.1.230:12900/

```

Result:

```
time="2017-05-19T16:06:30+10:00" level=error msg="[RequestConfiguration] Fetching configuration failed: Get http://10.40.1.230:12900/plugins/org.graylog.plugins.collector/925b5261-6142-44ca-b96c-a292d0503a04?tags=%5B%22windows%22%2C%22iis%22%5D: dial tcp 10.40.1.230:12900: connectex: No connection could be made because the target machine actively refused it." 
time="2017-05-19T16:06:31+10:00" level=error msg="[UpdateRegistration] Failed to report collector status to server: Put http://10.40.1.230:12900/plugins/org.graylog.plugins.collector/collectors/925b5261-6142-44ca-b96c-a292d0503a04: dial tcp 10.40.1.230:12900: connectex: No connection could be made because the target machine actively refused it." 
time="2017-05-19T16:06:41+10:00" level=error msg="[RequestConfiguration] Fetching configuration failed: Get http://10.40.1.230:12900/plugins/org.graylog.plugins.collector/925b5261-6142-44ca-b96c-a292d0503a04?tags=%5B%22windows%22%2C%22iis%22%5D: dial tcp 10.40.1.230:12900: connectex: No connection could be made because the target machine actively refused it." 
time="2017-05-19T16:06:43+10:00" level=error msg="[UpdateRegistration] Failed to report collector status to server: Put http://10.40.1.230:12900/plugins/org.graylog.plugins.collector/collectors/925b5261-6142-44ca-b96c-a292d0503a04: dial tcp 10.40.1.230:12900: connectex: No connection could be made because the target machine actively refused it." 
time="2017-05-19T16:06:52+10:00" level=error msg="[RequestConfiguration] Fetching configuration failed: Get http://10.40.1.230:12900/plugins/org.graylog.plugins.collector/925b5261-6142-44ca-b96c-a292d0503a04?tags=%5B%22windows%22%2C%22iis%22%5D: dial tcp 10.40.1.230:12900: connectex: No connection could be made because the target machine actively refused it." 
time="2017-05-19T16:06:55+10:00" level=error msg="[UpdateRegistration] Failed to report collector status to server: Put http://10.40.1.230:12900/plugins/org.graylog.plugins.collector/collectors/925b5261-6142-44ca-b96c-a292d0503a04: dial tcp 10.40.1.230:12900: connectex: No connection could be made because the target machine actively refused it." 
time="2017-05-19T16:07:03+10:00" level=error msg="[RequestConfiguration] Fetching configuration failed: Get http://10.40.1.230:12900/plugins/org.graylog.plugins.collector/925b5261-6142-44ca-b96c-a292d0503a04?tags=%5B%22windows%22%2C%22iis%22%5D: dial tcp 10.40.1.230:12900: connectex: No connection could be made because the target machine actively refused it." 
time="2017-05-19T16:07:07+10:00" level=error msg="[UpdateRegistration] Failed to report collector status to server: Put http://10.40.1.230:12900/plugins/org.graylog.plugins.collector/collectors/925b5261-6142-44ca-b96c-a292d0503a04: dial tcp 10.40.1.230:12900: connectex: No connection could be made because the target machine actively refused it." 

```

**Attempt 2:**

Configuration of Sidecar-Collector (C:\Program Files\Graylog\collector-sidecar\collector\_sidecar.yml)

```
server_url: http://10.40.1.230:12900/api

```

Result:

```
time="2017-05-19T16:08:25+10:00" level=error msg="[RequestConfiguration] Fetching configuration failed: Get http://10.40.1.230:12900/api/plugins/org.graylog.plugins.collector/925b5261-6142-44ca-b96c-a292d0503a04?tags=%5B%22windows%22%2C%22iis%22%5D: dial tcp 10.40.1.230:12900: connectex: No connection could be made because the target machine actively refused it." 
time="2017-05-19T16:08:26+10:00" level=error msg="[UpdateRegistration] Failed to report collector status to server: Put http://10.40.1.230:12900/api/plugins/org.graylog.plugins.collector/collectors/925b5261-6142-44ca-b96c-a292d0503a04: dial tcp 10.40.1.230:12900: connectex: No connection could be made because the target machine actively refused it." 
time="2017-05-19T16:08:36+10:00" level=error msg="[RequestConfiguration] Fetching configuration failed: Get http://10.40.1.230:12900/api/plugins/org.graylog.plugins.collector/925b5261-6142-44ca-b96c-a292d0503a04?tags=%5B%22windows%22%2C%22iis%22%5D: dial tcp 10.40.1.230:12900: connectex: No connection could be made because the target machine actively refused it." 
time="2017-05-19T16:08:39+10:00" level=error msg="[UpdateRegistration] Failed to report collector status to server: Put http://10.40.1.230:12900/api/plugins/org.graylog.plugins.collector/collectors/925b5261-6142-44ca-b96c-a292d0503a04: dial tcp 10.40.1.230:12900: connectex: No connection could be made because the target machine actively refused it." 
time="2017-05-19T16:08:47+10:00" level=error msg="[RequestConfiguration] Fetching configuration failed: Get http://10.40.1.230:12900/api/plugins/org.graylog.plugins.collector/925b5261-6142-44ca-b96c-a292d0503a04?tags=%5B%22windows%22%2C%22iis%22%5D: dial tcp 10.40.1.230:12900: connectex: No connection could be made because the target machine actively refused it." 

```

**Attempt 3:**

Configuration of Sidecar-Collector (C:\Program Files\Graylog\collector-sidecar\collector\_sidecar.yml)

```
server_url: http://10.40.1.230:9000/

```

Result:

```
time="2017-05-19T16:10:23+10:00" level=error msg="[RequestConfiguration] Bad response status from Graylog server: 404 Not Found" 
time="2017-05-19T16:10:23+10:00" level=error msg="Can't fetch configuration from Graylog API: invalid character '<' looking for beginning of value" 
time="2017-05-19T16:10:23+10:00" level=error msg="[filebeat] Unable to start collector after 3 tries, giving up!" 
time="2017-05-19T16:10:24+10:00" level=error msg="[UpdateRegistration] Bad response from Graylog server: 404 Not Found" 
time="2017-05-19T16:10:33+10:00" level=error msg="[RequestConfiguration] Bad response status from Graylog server: 404 Not Found" 
time="2017-05-19T16:10:33+10:00" level=error msg="Can't fetch configuration from Graylog API: invalid character '<' looking for beginning of value" 
time="2017-05-19T16:10:34+10:00" level=error msg="[UpdateRegistration] Bad response from Graylog server: 404 Not Found" 

```

**Attempt 4:**

Configuration of Sidecar-Collector (C:\Program Files\Graylog\collector-sidecar\collector\_sidecar.yml)

```
server_url: http://10.40.1.230:9000/api

```

Result:

```
time="2017-05-19T16:12:12+10:00" level=error msg="[RequestConfiguration] Bad response status from Graylog server: 404 Not Found" 
time="2017-05-19T16:12:12+10:00" level=error msg="Can't fetch configuration from Graylog API: invalid character '<' looking for beginning of value" 
time="2017-05-19T16:12:12+10:00" level=error msg="[filebeat] Unable to start collector after 3 tries, giving up!" 
time="2017-05-19T16:12:13+10:00" level=error msg="[UpdateRegistration] Bad response from Graylog server: 404 Not Found" 
time="2017-05-19T16:12:22+10:00" level=error msg="[RequestConfiguration] Bad response status from Graylog server: 404 Not Found" 
time="2017-05-19T16:12:22+10:00" level=error msg="Can't fetch configuration from Graylog API: invalid character '<' looking for beginning of value" 
time="2017-05-19T16:12:24+10:00" level=error msg="[UpdateRegistration] Bad response from Graylog server: 404 Not Found" 

```

If anyone that has been able to successfully deploy a sidecar-collector to a graylog cluster would be kind enough to point out where i’m going wrong, i’d be very grateful.

---

<div class="post-metadata">

**Author:** ![jochen](https://sea2.discourse-cdn.com/flex016/user_avatar/community.graylog.org/jochen/32/8_2.png) [@jochen](https://community.graylog.org/u/jochen)\
**Post date:** [May 19, 2017, 9:19am UTC](https://community.graylog.org/t/sidecar-collector-connection-to-graylog-cluster/1184/2 "2017-05-19T09:19:23Z")

</div>

> [@drstaind](#):
>
> Configuration of the graylog web server (/etc/graylog/web/web.conf)

The old standalone Graylog web interface **is not compatible** with Graylog 2.0.0 or higher.

In order to solve your issues:

- Make sure that you’re running Graylog 2.2.3.
- Make sure that the Graylog Collector plugin is installed (check on the _System_ / _Nodes_ / _Details_ pages in the Graylog web interface).
- Make sure that the Graylog Collector Sidecars can communicate with the Graylog REST API (either directly or proxied by HAProxy).

---

<div class="post-metadata">

**Author:** ![drstaind](https://avatars.discourse-cdn.com/v4/letter/d/919ad9/32.png) [@drstaind](https://community.graylog.org/u/drstaind)\
**Post date:** [May 22, 2017, 2:31am UTC](https://community.graylog.org/t/sidecar-collector-connection-to-graylog-cluster/1184/3 "2017-05-22T02:31:11Z")

</div>

Downloaded graylog-plugin-collector-2.3.0-alpha.2.jar to /usr/share/graylog-server/plugin.

Result after graylog-server restart:

```
May 22 12:27:30 graylog1 systemd: Starting Graylog server...
May 22 12:27:30 graylog1 graylog-server: OpenJDK 64-Bit Server VM warning: ignoring option PermSize=128m; support was removed in 8.0
May 22 12:27:30 graylog1 graylog-server: OpenJDK 64-Bit Server VM warning: ignoring option MaxPermSize=256m; support was removed in 8.0
May 22 12:27:31 graylog1 graylog-server: Exception in thread "main" java.lang.NoSuchMethodError: org.graylog2.plugin.Version.from(IIILjava/lang/String;)Lorg/graylog2/plugin/Version;
May 22 12:27:31 graylog1 graylog-server: at org.graylog.plugins.collector.CollectorMetaData.getVersion(CollectorMetaData.java:55)
May 22 12:27:31 graylog1 graylog-server: at org.graylog2.shared.plugins.PluginLoader$PluginComparator.compare(PluginLoader.java:112)
May 22 12:27:31 graylog1 graylog-server: at org.graylog2.shared.plugins.PluginLoader$PluginComparator.compare(PluginLoader.java:103)
May 22 12:27:31 graylog1 graylog-server: at java.util.TimSort.countRunAndMakeAscending(TimSort.java:355)
May 22 12:27:31 graylog1 graylog-server: at java.util.TimSort.sort(TimSort.java:220)
May 22 12:27:31 graylog1 graylog-server: at java.util.Arrays.sort(Arrays.java:1512)
May 22 12:27:31 graylog1 graylog-server: at com.google.common.collect.ImmutableSortedSet.construct(ImmutableSortedSet.java:428)
May 22 12:27:31 graylog1 graylog-server: at com.google.common.collect.ImmutableSortedSet$Builder.build(ImmutableSortedSet.java:562)
May 22 12:27:31 graylog1 graylog-server: at org.graylog2.shared.plugins.PluginLoader.loadPlugins(PluginLoader.java:56)
May 22 12:27:31 graylog1 graylog-server: at org.graylog2.bootstrap.CmdLineTool.loadPlugins(CmdLineTool.java:264)
May 22 12:27:31 graylog1 graylog-server: at org.graylog2.bootstrap.CmdLineTool.installPluginConfigAndBindings(CmdLineTool.java:229)
May 22 12:27:31 graylog1 graylog-server: at org.graylog2.bootstrap.CmdLineTool.run(CmdLineTool.java:151)
May 22 12:27:31 graylog1 graylog-server: at org.graylog2.bootstrap.Main.main(Main.java:44)
May 22 12:27:31 graylog1 systemd: graylog-server.service: main process exited, code=exited, status=1/FAILURE
May 22 12:27:31 graylog1 systemd: Unit graylog-server.service entered failed state.
May 22 12:27:31 graylog1 systemd: graylog-server.service failed.
```

---

<div class="post-metadata">

**Author:** ![jan](https://sea2.discourse-cdn.com/flex016/user_avatar/community.graylog.org/jan/32/11_2.png) [@jan](https://community.graylog.org/u/jan)\
**Post date:** [May 22, 2017, 6:02am UTC](https://community.graylog.org/t/sidecar-collector-connection-to-graylog-cluster/1184/4 "2017-05-22T06:02:59Z")

</div>

@drstaind

you have now the alpha version installed - that might have some issues with newer plugins. If you want to run a stable version of Graylog you should use Graylog **2.2.3**.

Complete remove the Web Interface as this is part of the Graylog server. Take care of what @jochen had written.

---

<div class="post-metadata">

**Author:** ![drstaind](https://avatars.discourse-cdn.com/v4/letter/d/919ad9/32.png) [@drstaind](https://community.graylog.org/u/drstaind)\
**Post date:** [May 23, 2017, 3:36am UTC](https://community.graylog.org/t/sidecar-collector-connection-to-graylog-cluster/1184/5 "2017-05-23T03:36:33Z")

</div>

The issue was indeed related to the old version of graylog\_web installed.

Now running three current instances with the correct plugins for the collector. Working well.

Many thanks for your help.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/flex016/uploads/graylog/original/3X/c/7/c7c09c6b5099570133d6502b83f50ba4430de5b6.png) [@system](https://community.graylog.org/u/system)\
**Post date:** [June 6, 2017, 3:36am UTC](https://community.graylog.org/t/sidecar-collector-connection-to-graylog-cluster/1184/6 "2017-06-06T03:36:35Z")

</div>

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.