Elasticsearch configuration for graylog action.auto_create_index: false throwing error

Graylog Central (peer support)
1

Before you post: Your responses to these questions will help the community help you. Please complete this template if you’re asking a support question.
Don’t forget to select tags to help index your topic!

1. Describe your incident:

I am trying to install elasticsearch and add graylog specific configuration to it.
I am following below official document.
https://docs.graylog.org/v1/docs/centos

sudo tee -a /etc/elasticsearch/elasticsearch.yml > /dev/null <<EOT
cluster.name: graylog
action.auto_create_index: false
EOT

But action.auto_create_index: false this configuration is not working and elasticsearch fails with the error thats its too restrictive.
Its throwing error.

When I am removing action.auto_create_index: false the elasticsearch is starting normally.

2. Describe your environment:

  • OS Information: CentOS Linux release 8.4.2105

  • Package Version: 4.2

  • Service logs, configurations, and environment variables:
    Elastic search 7.10.0
    Mongo 4.2
    Java 1.8.0

3. What steps have you already taken to try and solve the problem?

I removed action.auto_create_index: false from configuration file and elasticsearch started running fine.

4. How can the community help?
Why elasticsearch not working with autocreateindex false? How to solve this?
Is it ok to remove index parameters and run graylog?

Helpful Posting Tips: Tips for Posting Questions that Get Answers [Hold down CTRL and link on link to open tips documents in a separate tab]

2

Hello && Welcome @spandey

I might be able to help.

We would need to see this error in its full glory.

Can you show Elasticsearch/Graylog log files when this issue occurs?

/var/log/elasticsearch/
/var/log/graylog-server

If possible show us elasticsearch.yml file configuration?

cat /etc/elasticsearch/elasticsearch.yml | egrep -v "^\s*(#|$)"

Just an FYI instead of using that command to configure the elasticsearch YAML file, try manually configuring it like so…

vi /etc/elasticsearch/elasticsearch.yml

Then type in the following and notice the spacing.

action.auto_create_index: false

Results:

[root@graylog graylog]# cat /etc/elasticsearch/elasticsearch.yml | egrep -v "^\s*(#|$)"
cluster.name: graylog
path.data: /var/lib/elasticsearch
path.logs: /var/log/elasticsearch
network.host: 8.8.8.8
http.port: 9200
action.auto_create_index: false
discovery.type: single-node
[root@graylog graylog]#

Remember to restart Elasticsearch when done.

systemctl restart elasticsearch

action.auto_create_index is there to prevent Elasticsearch from automatically creating indices when asked to index documents into an index that doesn’t currently exist. Graylog wants to be in control of creating indices so it can set up the deflector aliases properly, so if you have disabled that setting at some point a message will be indexed to the deflector alias, which may not exist yet, and Elasticsearch will create it as an index which will stop the index rotation cold.

Elasticsearch documentation.

Automatic index creation is controlled by the action.auto_create_index setting. This setting defaults to true, which allows any index to be created automatically. You can modify this setting to explicitly allow or block automatic creation of indices that match specified patterns, or set it to false to disable automatic index creation entirely

5

I am editing the elasticsearch.yml file using vim.
I didnt use cat .

I am not able to send java exception logs here ., my post is getting hidden.

Below is part of the error.

Caused by: java.lang.IllegalArgumentException: the [action.auto_create_index] setting value [false] is too restrictive. disable [action.auto_create_index] or set it to [.watches,.triggered_watches,.watcher-history-*]

6

The following works in my lab, so I’m not sure why your having issues.

action.auto_create_index: false

I also Found these post. it may help understand why this is happening, Unfortunately I can not reproduce this issue in my lab.

EDIT:

Not sure what going on there. What I was asking was to see you Elasticsearch file. and post it here using this command as I showed above.

I would need to see full logs. With out more data, I cant give you a direct answer.

7 
[root@j3chyvmsysl01 tmp]# cat /etc/elasticsearch/elasticsearch.yml | egrep -v "^\s*(#|$)"
path.data: /var/lib/elasticsearch
path.logs: /var/log/elasticsearch
cluster.name: graylog


JAVA error

> [2022-04-16T03:12:55,869][INFO ][o.e.n.Node               ] [j3chyvmsysl01] version[7.10.0], pid[1458884], build[default/rpm/51e9d6f22758d0374a0f3f5c6e8f3a7997850f96/2020-11-09T21:30:33.964949Z], OS[Linux/4.18.0-305.3.1.el8.x86_64/amd64], JVM[AdoptOpenJDK/OpenJDK 64-Bit Server VM/15.0.1/15.0.1+9]
> [2022-04-16T03:12:55,872][INFO ][o.e.n.Node               ] [j3chyvmsysl01] JVM home [/usr/share/elasticsearch/jdk], using bundled JDK [true]
> [2022-04-16T03:12:55,873][INFO ][o.e.n.Node               ] [j3chyvmsysl01] JVM arguments [-Xshare:auto, -Des.networkaddress.cache.ttl=60, -Des.networkaddress.cache.negative.ttl=10, -XX:+AlwaysPreTouch, -Xss1m, -Djava.awt.headless=true, -Dfile.encoding=UTF-8, -Djna.nosys=true, -XX:-OmitStackTraceInFastThrow, -XX:+ShowCodeDetailsInExceptionMessages, -Dio.netty.noUnsafe=true, -Dio.netty.noKeySetOptimization=true, -Dio.netty.recycler.maxCapacityPerThread=0, -Dio.netty.allocator.numDirectArenas=0, -Dlog4j.shutdownHookEnabled=false, -Dlog4j2.disable.jmx=true, -Djava.locale.providers=SPI,COMPAT, -Xms1g, -Xmx1g, -XX:+UseG1GC, -XX:G1ReservePercent=25, -XX:InitiatingHeapOccupancyPercent=30, -Djava.io.tmpdir=/tmp/elasticsearch-18171107793140923621, -XX:+HeapDumpOnOutOfMemoryError, -XX:HeapDumpPath=/var/lib/elasticsearch, -XX:ErrorFile=/var/log/elasticsearch/hs_err_pid%p.log, -Xlog:gc*,gc+age=trace,safepoint:file=/var/log/elasticsearch/gc.log:utctime,pid,tags:filecount=32,filesize=64m, -XX:MaxDirectMemorySize=536870912, -Des.path.home=/usr/share/elasticsearch, -Des.path.conf=/etc/elasticsearch, -Des.distribution.flavor=default, -Des.distribution.type=rpm, -Des.bundled_jdk=true]
> [2022-04-16T03:12:58,805][ERROR][o.e.b.Bootstrap          ] [j3chyvmsysl01] Exception
> java.lang.IllegalStateException: failed to load plugin class [org.elasticsearch.xpack.watcher.Watcher]
>         at org.elasticsearch.plugins.PluginsService.loadPlugin(PluginsService.java:722) ~[elasticsearch-7.10.0.jar:7.10.0]
>         at org.elasticsearch.plugins.PluginsService.loadBundle(PluginsService.java:658) ~[elasticsearch-7.10.0.jar:7.10.0]
>         at org.elasticsearch.plugins.PluginsService.loadBundles(PluginsService.java:479) ~[elasticsearch-7.10.0.jar:7.10.0]
>         at org.elasticsearch.plugins.PluginsService.<init>(PluginsService.java:168) ~[elasticsearch-7.10.0.jar:7.10.0]
>         at org.elasticsearch.node.Node.<init>(Node.java:346) ~[elasticsearch-7.10.0.jar:7.10.0]
>         at org.elasticsearch.node.Node.<init>(Node.java:289) ~[elasticsearch-7.10.0.jar:7.10.0]
>         at org.elasticsearch.bootstrap.Bootstrap$5.<init>(Bootstrap.java:227) ~[elasticsearch-7.10.0.jar:7.10.0]
>         at org.elasticsearch.bootstrap.Bootstrap.setup(Bootstrap.java:227) ~[elasticsearch-7.10.0.jar:7.10.0]
>         at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:393) [elasticsearch-7.10.0.jar:7.10.0]
>         at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:170) [elasticsearch-7.10.0.jar:7.10.0]
>         at org.elasticsearch.bootstrap.Elasticsearch.execute(Elasticsearch.java:161) [elasticsearch-7.10.0.jar:7.10.0]
>         at org.elasticsearch.cli.EnvironmentAwareCommand.execute(EnvironmentAwareCommand.java:86) [elasticsearch-7.10.0.jar:7.10.0]
>         at org.elasticsearch.cli.Command.mainWithoutErrorHandling(Command.java:127) [elasticsearch-cli-7.10.0.jar:7.10.0]
>         at org.elasticsearch.cli.Command.main(Command.java:90) [elasticsearch-cli-7.10.0.jar:7.10.0]
>         at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:126) [elasticsearch-7.10.0.jar:7.10.0]
>         at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:92) [elasticsearch-7.10.0.jar:7.10.0]
> Caused by: java.lang.reflect.InvocationTargetException
>         at jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) ~[?:?]
>         at jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:64) ~[?:?]
>         at jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) ~[?:?]
>         at java.lang.reflect.Constructor.newInstanceWithCaller(Constructor.java:500) ~[?:?]
>         at java.lang.reflect.Constructor.newInstance(Constructor.java:481) ~[?:?]
>         at org.elasticsearch.plugins.PluginsService.loadPlugin(PluginsService.java:715) ~[elasticsearch-7.10.0.jar:7.10.0]
>         ... 15 more
> Caused by: java.lang.IllegalArgumentException: the [action.auto_create_index] setting value [false] is too restrictive. disable [action.auto_create_index] or set it to [.watches,.triggered_watches,.watcher-history-*]
>         at org.elasticsearch.xpack.watcher.Watcher.validAutoCreateIndex(Watcher.java:616) ~[?:?]
>         at org.elasticsearch.xpack.watcher.Watcher.<init>(Watcher.java:248) ~[?:?]
>         at jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) ~[?:?]
>         at jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:64) ~[?:?]
>         at jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) ~[?:?]
>         at java.lang.reflect.Constructor.newInstanceWithCaller(Constructor.java:500) ~[?:?]
>         at java.lang.reflect.Constructor.newInstance(Constructor.java:481) ~[?:?]
>         at org.elasticsearch.plugins.PluginsService.loadPlugin(PluginsService.java:715) ~[elasticsearch-7.10.0.jar:7.10.0]
>         ... 15 more
8

image
image1248×818 96.3 KB

I have uploaded a snippet of the error.

Below is elastic configuration.

[root@j3chyvmsysl01 tmp]# cat /etc/elasticsearch/elasticsearch.yml | egrep -v "^\s*(#|$)"
path.data: /var/lib/elasticsearch
path.logs: /var/log/elasticsearch
cluster.name: graylog
9

This is the issue I am also facing.

index too restrictive.
I tried adding java error but it is keep on getting hidden.

Below is elastic configuration.

[root@j3chyvmsysl01 tmp]# cat /etc/elasticsearch/elasticsearch.yml | egrep -v "^\s*(#|$)"
path.data: /var/lib/elasticsearch
path.logs: /var/log/elasticsearch
cluster.name: graylog
[root@j3chyvmsysl01 tmp]#
10 
[root@j3chyvmsysl01 tmp]# cat /etc/elasticsearch/elasticsearch.yml | egrep -v "^\s*(#|$)"
path.data: /var/lib/elasticsearch
path.logs: /var/log/elasticsearch
cluster.name: graylog
[root@j3chyvmsysl01 tmp]#
11

[root@j3chyvmsysl01 tmp]# cat /etc/elasticsearch/elasticsearch.yml | egrep -v “^\s*(#|$)”
path.data: /var/lib/elasticsearch
path.logs: /var/log/elasticsearch
cluster.name: graylog
[root@j3chyvmsysl01 tmp]#

12

I am not able to post any comment. Every logs or configuration is going for hidden.

13

Hello,

Can you post this command here?

curl -XGET http://localhost:9200/

I see there are double post perhaps its a Network/Browser issue?

14

Hi,
Yes multiple post has been done because the portal was not accepting my messages.
I will check the localhost and will get back to you.
Thanks!

15

[root@j3chyvmsysl01 certs]# curl -XGET http://localhost:9200/
{
“name” : “j3chyvmsysl01”,
“cluster_name” : “graylog”,
“cluster_uuid” : “EtTRfDsCRfSj-qDKrGWWgw”,
“version” : {
“number” : “7.10.0”,
“build_flavor” : “default”,
“build_type” : “rpm”,
“build_hash” : “51e9d6f22758d0374a0f3f5c6e8f3a7997850f96”,
“build_date” : “2020-11-09T21:30:33.964949Z”,
“build_snapshot” : false,
“lucene_version” : “8.7.0”,
“minimum_wire_compatibility_version” : “6.8.0”,
“minimum_index_compatibility_version” : “6.0.0-beta1”
},
“tagline” : “You Know, for Search”
}

16

I see you not use elasticsearch-oss, is this correct?

What do you get when you run this command.

curl -XGET http://localhost:9200/_cluster/health?pretty

17

I will look into it @gsmith.
Currently looking into troubleshooting a separate issue related to HTTPS configuration.
Thanks!

18

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.