Your Elasticsearch service isn’t working - Post logs from Graylog as @gsmith suggests as well as results of:
$ journalctl -eu elasticsearch
Which will show more detail on why Elasticsearch is failing. You can also post results of:
$ cat /etc/elasticsearch/elasticsearch.yml | egrep -v "^\s*(#|$)"
Which shows your ElasticSearch settings.
ubuntu@ubuntu:~$ journalctl -eu elasticsearch
Sep 22 21:42:26 ubuntu systemd-entrypoint[1151]: at org.elasticsearch.common.settings.Settings$Builder.loadFromStream(Settings.java:1113)
Sep 22 21:42:26 ubuntu systemd-entrypoint[1151]: at org.elasticsearch.common.settings.Settings$Builder.loadFromPath(Settings.java:1086)
Sep 22 21:42:26 ubuntu systemd-entrypoint[1151]: at org.elasticsearch.node.InternalSettingsPreparer.prepareEnvironment(InternalSettingsPreparer.java:83)
Sep 22 21:42:26 ubuntu systemd-entrypoint[1151]: at org.elasticsearch.cli.EnvironmentAwareCommand.createEnv(EnvironmentAwareCommand.java:100)
Sep 22 21:42:26 ubuntu systemd-entrypoint[1151]: at org.elasticsearch.cli.EnvironmentAwareCommand.createEnv(EnvironmentAwareCommand.java:91)
Sep 22 21:42:26 ubuntu systemd-entrypoint[1151]: at org.elasticsearch.cli.EnvironmentAwareCommand.execute(EnvironmentAwareCommand.java:86)
Sep 22 21:42:26 ubuntu systemd-entrypoint[1151]: at org.elasticsearch.cli.Command.mainWithoutErrorHandling(Command.java:127)
Sep 22 21:42:26 ubuntu systemd-entrypoint[1151]: at org.elasticsearch.cli.Command.main(Command.java:90)
Sep 22 21:42:26 ubuntu systemd-entrypoint[1151]: at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:126)
Sep 22 21:42:26 ubuntu systemd-entrypoint[1151]: at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:92)
Sep 22 21:42:26 ubuntu systemd-entrypoint[1151]: Caused by: com.fasterxml.jackson.core.JsonParseException: Duplicate field 'cluster.name'
Sep 22 21:42:26 ubuntu systemd-entrypoint[1151]: at [Source: (sun.nio.ch.ChannelInputStream); line: 89, column: 13]
Sep 22 21:42:26 ubuntu systemd-entrypoint[1151]: at com.fasterxml.jackson.core.json.JsonReadContext._checkDup(JsonReadContext.java:204)
Sep 22 21:42:26 ubuntu systemd-entrypoint[1151]: at com.fasterxml.jackson.core.json.JsonReadContext.setCurrentName(JsonReadContext.java:198)
Sep 22 21:42:26 ubuntu systemd-entrypoint[1151]: at com.fasterxml.jackson.dataformat.yaml.YAMLParser.nextToken(YAMLParser.java:399)
Sep 22 21:42:26 ubuntu systemd-entrypoint[1151]: at org.elasticsearch.common.xcontent.json.JsonXContentParser.nextToken(JsonXContentParser.java:52)
Sep 22 21:42:26 ubuntu systemd-entrypoint[1151]: at org.elasticsearch.common.settings.Settings.fromXContent(Settings.java:645)
Sep 22 21:42:26 ubuntu systemd-entrypoint[1151]: at org.elasticsearch.common.settings.Settings.fromXContent(Settings.java:620)
Sep 22 21:42:26 ubuntu systemd-entrypoint[1151]: at org.elasticsearch.common.settings.Settings.access$400(Settings.java:82)
Sep 22 21:42:26 ubuntu systemd-entrypoint[1151]: at org.elasticsearch.common.settings.Settings$Builder.loadFromStream(Settings.java:1109)
Sep 22 21:42:26 ubuntu systemd-entrypoint[1151]: ... 9 more
Sep 22 21:42:26 ubuntu systemd[1]: elasticsearch.service: Main process exited, code=exited, status=1/FAILURE
Sep 22 21:42:26 ubuntu systemd[1]: elasticsearch.service: Failed with result 'exit-code'.
Sep 22 21:42:26 ubuntu systemd[1]: Failed to start Elasticsearch.
-- Reboot --
Sep 22 21:56:50 ubuntu systemd[1]: Starting Elasticsearch...
Sep 22 21:57:07 ubuntu systemd-entrypoint[1415]: Exception in thread "main" SettingsException[Failed to load settings from [elasticsearch.yml]]; nested: JsonParseException[Duplicate field 'cluster.name'
Sep 22 21:57:07 ubuntu systemd-entrypoint[1415]: at [Source: (sun.nio.ch.ChannelInputStream); line: 89, column: 13]];
Sep 22 21:57:07 ubuntu systemd-entrypoint[1415]: at org.elasticsearch.common.settings.Settings$Builder.loadFromStream(Settings.java:1113)
Sep 22 21:57:07 ubuntu systemd-entrypoint[1415]: at org.elasticsearch.common.settings.Settings$Builder.loadFromPath(Settings.java:1086)
Sep 22 21:57:07 ubuntu systemd-entrypoint[1415]: at org.elasticsearch.node.InternalSettingsPreparer.prepareEnvironment(InternalSettingsPreparer.java:83)
Sep 22 21:57:07 ubuntu systemd-entrypoint[1415]: at org.elasticsearch.cli.EnvironmentAwareCommand.createEnv(EnvironmentAwareCommand.java:100)
Sep 22 21:57:07 ubuntu systemd-entrypoint[1415]: at org.elasticsearch.cli.EnvironmentAwareCommand.createEnv(EnvironmentAwareCommand.java:91)
Sep 22 21:57:07 ubuntu systemd-entrypoint[1415]: at org.elasticsearch.cli.EnvironmentAwareCommand.execute(EnvironmentAwareCommand.java:86)
Sep 22 21:57:07 ubuntu systemd-entrypoint[1415]: at org.elasticsearch.cli.Command.mainWithoutErrorHandling(Command.java:127)
Sep 22 21:57:07 ubuntu systemd-entrypoint[1415]: at org.elasticsearch.cli.MultiCommand.execute(MultiCommand.java:91)
Sep 22 21:57:07 ubuntu systemd-entrypoint[1415]: at org.elasticsearch.cli.Command.mainWithoutErrorHandling(Command.java:127)
Sep 22 21:57:07 ubuntu systemd-entrypoint[1415]: at org.elasticsearch.cli.Command.main(Command.java:90)
Sep 22 21:57:07 ubuntu systemd-entrypoint[1415]: at org.elasticsearch.common.settings.KeyStoreCli.main(KeyStoreCli.java:43)
Sep 22 21:57:07 ubuntu systemd-entrypoint[1415]: Caused by: com.fasterxml.jackson.core.JsonParseException: Duplicate field 'cluster.name'
Sep 22 21:57:07 ubuntu systemd-entrypoint[1415]: at [Source: (sun.nio.ch.ChannelInputStream); line: 89, column: 13]
Sep 22 21:57:07 ubuntu systemd-entrypoint[1415]: at com.fasterxml.jackson.core.json.JsonReadContext._checkDup(JsonReadContext.java:204)
Sep 22 21:57:07 ubuntu systemd-entrypoint[1415]: at com.fasterxml.jackson.core.json.JsonReadContext.setCurrentName(JsonReadContext.java:198)
Sep 22 21:57:07 ubuntu systemd-entrypoint[1415]: at com.fasterxml.jackson.dataformat.yaml.YAMLParser.nextToken(YAMLParser.java:399)
Sep 22 21:57:07 ubuntu systemd-entrypoint[1415]: at org.elasticsearch.common.xcontent.json.JsonXContentParser.nextToken(JsonXContentParser.java:52)
Sep 22 21:57:07 ubuntu systemd-entrypoint[1415]: at org.elasticsearch.common.settings.Settings.fromXContent(Settings.java:645)
Sep 22 21:57:07 ubuntu systemd-entrypoint[1415]: at org.elasticsearch.common.settings.Settings.fromXContent(Settings.java:620)
Sep 22 21:57:07 ubuntu systemd-entrypoint[1415]: at org.elasticsearch.common.settings.Settings.access$400(Settings.java:82)
Sep 22 21:57:07 ubuntu systemd-entrypoint[1415]: at org.elasticsearch.common.settings.Settings$Builder.loadFromStream(Settings.java:1109)
Sep 22 21:57:07 ubuntu systemd-entrypoint[1415]: ... 10 more
Sep 22 21:57:10 ubuntu systemd-entrypoint[1151]: Exception in thread "main" 2022-09-22 21:57:10,440 main ERROR No Log4j 2 configuration file found. Using default configuration (logging only errors to the console), or user programmatical>
Sep 22 21:57:10 ubuntu systemd-entrypoint[1151]: SettingsException[Failed to load settings from [elasticsearch.yml]]; nested: JsonParseException[Duplicate field 'cluster.name'
Sep 22 21:57:10 ubuntu systemd-entrypoint[1151]: at [Source: (sun.nio.ch.ChannelInputStream); line: 89, column: 13]];
Sep 22 21:57:10 ubuntu systemd-entrypoint[1151]: at org.elasticsearch.common.settings.Settings$Builder.loadFromStream(Settings.java:1113)
Sep 22 21:57:10 ubuntu systemd-entrypoint[1151]: at org.elasticsearch.common.settings.Settings$Builder.loadFromPath(Settings.java:1086)
Sep 22 21:57:10 ubuntu systemd-entrypoint[1151]: at org.elasticsearch.node.InternalSettingsPreparer.prepareEnvironment(InternalSettingsPreparer.java:83)
Sep 22 21:57:10 ubuntu systemd-entrypoint[1151]: at org.elasticsearch.cli.EnvironmentAwareCommand.createEnv(EnvironmentAwareCommand.java:100)
Sep 22 21:57:10 ubuntu systemd-entrypoint[1151]: at org.elasticsearch.cli.EnvironmentAwareCommand.createEnv(EnvironmentAwareCommand.java:91)
Sep 22 21:57:10 ubuntu systemd-entrypoint[1151]: at org.elasticsearch.cli.EnvironmentAwareCommand.execute(EnvironmentAwareCommand.java:86)
Sep 22 21:57:10 ubuntu systemd-entrypoint[1151]: at org.elasticsearch.cli.Command.mainWithoutErrorHandling(Command.java:127)
Sep 22 21:57:10 ubuntu systemd-entrypoint[1151]: at org.elasticsearch.cli.Command.main(Command.java:90)
Sep 22 21:57:10 ubuntu systemd-entrypoint[1151]: at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:126)
Sep 22 21:57:10 ubuntu systemd-entrypoint[1151]: at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:92)
Sep 22 21:57:10 ubuntu systemd-entrypoint[1151]: Caused by: com.fasterxml.jackson.core.JsonParseException: Duplicate field 'cluster.name'
Sep 22 21:57:10 ubuntu systemd-entrypoint[1151]: at [Source: (sun.nio.ch.ChannelInputStream); line: 89, column: 13]
Sep 22 21:57:10 ubuntu systemd-entrypoint[1151]: at com.fasterxml.jackson.core.json.JsonReadContext._checkDup(JsonReadContext.java:204)
Sep 22 21:57:10 ubuntu systemd-entrypoint[1151]: at com.fasterxml.jackson.core.json.JsonReadContext.setCurrentName(JsonReadContext.java:198)
Sep 22 21:57:10 ubuntu systemd-entrypoint[1151]: at com.fasterxml.jackson.dataformat.yaml.YAMLParser.nextToken(YAMLParser.java:399)
Sep 22 21:57:10 ubuntu systemd-entrypoint[1151]: at org.elasticsearch.common.xcontent.json.JsonXContentParser.nextToken(JsonXContentParser.java:52)
Sep 22 21:57:10 ubuntu systemd-entrypoint[1151]: at org.elasticsearch.common.settings.Settings.fromXContent(Settings.java:645)
Sep 22 21:57:10 ubuntu systemd-entrypoint[1151]: at org.elasticsearch.common.settings.Settings.fromXContent(Settings.java:620)
Sep 22 21:57:10 ubuntu systemd-entrypoint[1151]: at org.elasticsearch.common.settings.Settings.access$400(Settings.java:82)
Sep 22 21:57:10 ubuntu systemd-entrypoint[1151]: at org.elasticsearch.common.settings.Settings$Builder.loadFromStream(Settings.java:1109)
Sep 22 21:57:10 ubuntu systemd-entrypoint[1151]: ... 9 more
Sep 22 21:57:10 ubuntu systemd[1]: elasticsearch.service: Main process exited, code=exited, status=1/FAILURE
Sep 22 21:57:10 ubuntu systemd[1]: elasticsearch.service: Failed with result 'exit-code'.
Sep 22 21:57:10 ubuntu systemd[1]: Failed to start Elasticsearch.
-- Reboot --
Oct 06 12:42:36 ubuntu systemd[1]: Starting Elasticsearch...
-- Reboot --
Oct 06 12:49:50 ubuntu systemd[1]: Starting Elasticsearch...
ubuntu@ubuntu:~$ cat /etc/elasticsearch/elasticsearch.yml | egrep -v “^\s*(#|$)”
cluster.name: graylog
path.data: /var/lib/elasticsearch
path.logs: /var/log/elasticsearch
cluster.name: graylog
action.auto_create_index: false
`ubuntu@ubuntu:/usr/local/bin$ tail -f /var/log/graylog-server/server.log
2022-10-06T22:16:16.334-03:00 ERROR [VersionProbe] Unable to retrieve version from Elasticsearch node: Failed to connect to /127.0.0.1:9200. - Connection refused (Connection refused).
2022-10-06T22:16:16.334-03:00 INFO [VersionProbe] Elasticsearch is not available. Retry #6792
2022-10-06T22:16:21.335-03:00 ERROR [VersionProbe] Unable to retrieve version from Elasticsearch node: Failed to connect to /127.0.0.1:9200. - Connection refused (Connection refused).
2022-10-06T22:16:21.335-03:00 INFO [VersionProbe] Elasticsearch is not available. Retry #6793
2022-10-06T22:16:26.335-03:00 ERROR [VersionProbe] Unable to retrieve version from Elasticsearch node: Failed to connect to /127.0.0.1:9200. - Connection refused (Connection refused).
2022-10-06T22:16:26.335-03:00 INFO [VersionProbe] Elasticsearch is not available. Retry #6794
2022-10-06T22:16:31.336-03:00 ERROR [VersionProbe] Unable to retrieve version from Elasticsearch node: Failed to connect to /127.0.0.1:9200. - Connection refused (Connection refused).
2022-10-06T22:16:31.336-03:00 INFO [VersionProbe] Elasticsearch is not available. Retry #6795
2022-10-06T22:16:36.336-03:00 ERROR [VersionProbe] Unable to retrieve version from Elasticsearch node: Failed to connect to /127.0.0.1:9200. - Connection refused (Connection refused).
2022-10-06T22:16:36.337-03:00 INFO [VersionProbe] Elasticsearch is not available. Retry #6796
2022-10-06T22:16:41.337-03:00 ERROR [VersionProbe] Unable to retrieve version from Elasticsearch node: Failed to connect to /127.0.0.1:9200. - Connection refused (Connection refused).
2022-10-06T22:16:41.337-03:00 INFO [VersionProbe] Elasticsearch is not available. Retry #6797
2022-10-06T22:16:46.338-03:00 ERROR [VersionProbe] Unable to retrieve version from Elasticsearch node: Failed to connect to /127.0.0.1:9200. - Connection refused (Connection refused).
2022-10-06T22:16:46.338-03:00 INFO [VersionProbe] Elasticsearch is not available. Retry #6798
2022-10-06T22:16:51.339-03:00 ERROR [VersionProbe] Unable to retrieve version from Elasticsearch node: Failed to connect to /127.0.0.1:9200. - Connection refused (Connection refused).
2022-10-06T22:16:51.339-03:00 INFO [VersionProbe] Elasticsearch is not available. Retry #6799
2022-10-06T22:16:56.339-03:00 ERROR [VersionProbe] Unable to retrieve version from Elasticsearch node: Failed to connect to /127.0.0.1:9200. - Connection refused (Connection refused).
2022-10-06T22:16:56.339-03:00 INFO [VersionProbe] Elasticsearch is not available. Retry #6800
2022-10-06T22:17:01.340-03:00 ERROR [VersionProbe] Unable to retrieve version from Elasticsearch node: Failed to connect to /127.0.0.1:9200. - Connection refused (Connection refused).
2022-10-06T22:17:01.340-03:00 INFO [VersionProbe] Elasticsearch is not available. Retry #6801
2022-10-06T22:17:06.341-03:00 ERROR [VersionProbe] Unable to retrieve version from Elasticsearch node: Failed to connect to /127.0.0.1:9200. - Connection refused (Connection refused).
2022-10-06T22:17:06.341-03:00 INFO [VersionProbe] Elasticsearch is not available. Retry #6802
Thanks in advance,````
Hello @emgonzalez59
Noticed in your logs it showed this…
Exception in thread "main" SettingsException[Failed to load settings from [elasticsearch.yml]]; nested: JsonParseException[Duplicate field 'cluster.name'
As you shown above…
Something wrong with Elasticsearch yaml file.
It should look something like this default setting
[root@graylog graylog_user]# cat /etc/elasticsearch/elasticsearch.yml | egrep -v "^\s*(#|$)"
cluster.name: graylog
path.data: /var/lib/elasticsearch
path.logs: /var/log/elasticsearch
network.host: 127.0.0.1
http.port: 9200
action.auto_create_index: false
discovery.type: single-node
[root@graylog graylog_user]#
EDIT: But this thing is, you should still be able to login Graylog Without Elasticsearch, so I would double check for duplicate configurations, Permissions on files and folders, firewall and/or check Selinux/Apparmor enabled
I have deleted the duplicated lines in ElasticSearch config file and corercted the IP address to localhost 127.0.0.1 , and now I can access the Graylog WebUI thru http://IP_Address:9000
FYI here is the output of cat command for elasticsearch,yml file:
ubuntu@ubuntu:/etc/selinux$ cat /etc/elasticsearch/elasticsearch.yml | egrep -v "^\s*(#|$)"
cluster.name: graylog
path.data: /var/lib/elasticsearch
path.logs: /var/log/elasticsearch
network.host: 127.0.0.1
http.port: 9200
action.auto_create_index: false
On the other hand, I run this command :
ubuntu@ubuntu:/etc/selinux$ curl -X GET http://127.0.0.1:9200
{
“name” : “ubuntu”,
“cluster_name” : “graylog”,
“cluster_uuid” : “VZTpvLCiQZi_gdD03knI6A”,
“version” : {
“number” : “7.10.2”,
“build_flavor” : “oss”,
“build_type” : “deb”,
“build_hash” : “747e1cc71def077253878a59143c1f785afa92b9”,
“build_date” : “2021-01-13T00:42:12.435326Z”,
“build_snapshot” : false,
“lucene_version” : “8.7.0”,
“minimum_wire_compatibility_version” : “6.8.0”,
“minimum_index_compatibility_version” : “6.0.0-beta1”
},
“tagline” : “You Know, for Search”
}
What do you think about ElasticSearch? do you think it is running?
Another question (sorry for that…) It is the first time i installed a SIEM like Graylog, I am trying to integrate it with ClearPass Policy Manager NAC. Can I get documentation about it? Thanks again and sorry for this long issue…
@emgonzalez59 Awesome glad your resolved your issue…
As for
Not sure, are you referring to send logs from that type of device?
just to clarify another thing… I have checked that my Ubuntu is NOT running SeLinux (output returns 1) >>
ubuntu@ubuntu:/etc/selinux$ selinuxenabled; echo $?
1
From what I can see looks like you have everything running.
Also @emgonzalez59
You can use this command to check the status of elasticsearch, If its in “Green” as shown below you should be good 
curl -XGET http://127.0.0.1:9200/_cluster/health?pretty
[root@graylog graylog_user]# curl -XGET http://10.10.10.10:9200/_cluster/health?pretty
{
"cluster_name" : "graylog",
"status" : "green",
"timed_out" : false,
"number_of_nodes" : 1,
"number_of_data_nodes" : 1,
"active_primary_shards" : 436,
"active_shards" : 436,
"relocating_shards" : 0,
"initializing_shards" : 0,
"unassigned_shards" : 0,
"delayed_unassigned_shards" : 0,
"number_of_pending_tasks" : 0,
"number_of_in_flight_fetch" : 0,
"task_max_waiting_in_queue_millis" : 0,
"active_shards_percent_as_number" : 100.0
}
[root@graylog graylog_user]# ^C
hi there,
It seems that ElasticSearch is not ok.
ubuntu@ubuntu:~$ curl -X GET http://192.168.1.2:9200/_cluster/health?pretty
curl: (7) Failed to connect to 192.168.1.2 port 9200: Connection refused
You were able to see elasticsearch via loopback (127.0.0.1) so put loopback in your cluster check…
ubuntu@ubuntu:~$ curl -X GET http://127.0.0.1:9200/_cluster/health?pretty
Yes, with 127.0.0.1 the output shows Green status.
Then I suppose all is running
Thanks,
Enrique
Obtener Outlook para Android
Great!! Post any follow on questions as New Topics, we’ll try our best to help!
This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.