# New request for custom index template

**URL:** <https://community.graylog.org/t/new-request-for-custom-index-template/24030>\
**Category:** Graylog Central (peer support)\
**Tags:** basic-configuration\
**Created:** [May 26, 2022, 12:45pm UTC](https://community.graylog.org/t/new-request-for-custom-index-template/24030 "2022-05-26T12:45:25Z")\
**Posts on this page:** 12\
**Page:** 1

<div class="post-metadata">

**Author:** ![Willy](https://avatars.discourse-cdn.com/v4/letter/w/c2a13f/32.png) [@Willy](https://community.graylog.org/u/Willy)\
**Post date:** [May 26, 2022, 12:45pm UTC](https://community.graylog.org/t/new-request-for-custom-index-template/24030/1 "2022-05-26T12:45:25Z")

</div>

Dear friends,  
I have been using graylog 4.0 for a long time but now I have an error that I need to fix.

To log messages from windows systems I use a specific input channel and a stream that  
forward all of these messages in a particular index (windows).

Unfortunately, as you can see from the message below, the type value is not enough to record everything and it needs to be increased

windows\_0 239e0f53-dc23-11ec-9172-001a4a16018f ElasticsearchException[Elasticsearch exception [type=illegal\_argument\_exception, reason=Limit of total fields [1000] has been exceeded]]

I see some guides and threads without any improvements:

[https://docs.graylog.org/docs/elasticsearch](https://docs.graylog.org/docs/elasticsearch)

I need to change the default value, 1000, for “type” value using a custom template  
for this index.  
Can anybody help me to solve this case ?  
Thanks a lot  
Bye  
Willy

---

<div class="post-metadata">

**Author:** ![tmacgbay](https://sea2.discourse-cdn.com/flex016/user_avatar/community.graylog.org/tmacgbay/32/3878_2.png) [@tmacgbay](https://community.graylog.org/u/tmacgbay)\
**Post date:** [May 26, 2022, 3:12pm UTC](https://community.graylog.org/t/new-request-for-custom-index-template/24030/2 "2022-05-26T15:12:03Z")

</div>

You can raise the limit but there are potential drawbacks to that - [here](https://www.graylog.org/post/what-to-do-when-you-have-1000-fields) is a good Graylog article on it

Also - [Here](https://community.graylog.org/t/elasticsearchexception-limit-of-total-fields-1000-has-been-exceeded/21179/9) is a previous post where we talked through finding and reducing the number of fields.

Finally of note - I believe the 1000 fields limit is per index set - if you can split out what you are doing into another index set (or more) that may address the issue.

---

<div class="post-metadata">

**Author:** ![Willy](https://avatars.discourse-cdn.com/v4/letter/w/c2a13f/32.png) [@Willy](https://community.graylog.org/u/Willy)\
**Post date:** [May 27, 2022, 8:26am UTC](https://community.graylog.org/t/new-request-for-custom-index-template/24030/3 "2022-05-27T08:26:26Z")

</div>

I read this guide [What to Do When You Have 1000+ Fields? | Graylog](https://www.graylog.org/post/what-to-do-when-you-have-1000-fields) and I change the value for the index that are in use at the moment.

[root@graylog enrico]# curl -XGET localhost:9200/windows\_0/\_settings/?pretty  
{  
“windows\_0” : {  
“settings” : {  
“index” : {  
“mapping” : {  
“total\_fields” : {  
“limit” : “2000”  
}  
},  
“number\_of\_shards” : “8”,  
“provided\_name” : “windows\_0”,  
“creation\_date” : “1653398764948”,  
“analysis” : {  
“analyzer” : {  
“analyzer\_keyword” : {  
“filter” : “lowercase”,  
“tokenizer” : “keyword”  
}  
}  
},  
“number\_of\_replicas” : “0”,  
“uuid” : “XA-XmGRwRd2u3Uwg3DVGug”,  
“version” : {  
“created” : “7100299”  
}  
}  
}  
}  
}

now I’d like to know how I can change the default value for new windows\_\* index  
that will be created when graylog make a rotation. I need to set type to 2000.

Thanks

---

<div class="post-metadata">

**Author:** ![Willy](https://avatars.discourse-cdn.com/v4/letter/w/c2a13f/32.png) [@Willy](https://community.graylog.org/u/Willy)\
**Post date:** [June 3, 2022, 9:51am UTC](https://community.graylog.org/t/new-request-for-custom-index-template/24030/4 "2022-06-03T09:51:38Z")

</div>

anybody can help me ?

---

<div class="post-metadata">

**Author:** ![tmacgbay](https://sea2.discourse-cdn.com/flex016/user_avatar/community.graylog.org/tmacgbay/32/3878_2.png) [@tmacgbay](https://community.graylog.org/u/tmacgbay)\
**Post date:** [June 3, 2022, 1:10pm UTC](https://community.graylog.org/t/new-request-for-custom-index-template/24030/5 "2022-06-03T13:10:03Z")

</div>

I still think you need to look at **why** you are ending up with over 1000 fields - [here is an example post of syslog Input with Fortigate](https://community.graylog.org/t/graylog-es-log-limit-of-total-fields-1000-in-index-graylog-519-has-been-exceeded-and-no-longer-processing/2009) where the input needed to be changed to RAW since the Syslog input was creating new fields for each message because it was parsing improperly. Alternatively you can split incoming to different indexes so that you reduce the number of fields in the index.

---

<div class="post-metadata">

**Author:** ![Willy](https://avatars.discourse-cdn.com/v4/letter/w/c2a13f/32.png) [@Willy](https://community.graylog.org/u/Willy)\
**Post date:** [June 7, 2022, 2:28pm UTC](https://community.graylog.org/t/new-request-for-custom-index-template/24030/6 "2022-06-07T14:28:26Z")

</div>

Dear all,  
I’m using some index , for example one for cisco one for fortigate and so on. The problem is that only inside window.index and I think that the limit that I need to be set is about 1400. The problem is occurr after rotation because when the new index was spawn it has got a default value, 1000. I set a new value only for windows\_\* index inside elastisearch.yml config file ? Are there any other possibilities ?  
Thanks again

---

<div class="post-metadata">

**Author:** ![tmacgbay](https://sea2.discourse-cdn.com/flex016/user_avatar/community.graylog.org/tmacgbay/32/3878_2.png) [@tmacgbay](https://community.graylog.org/u/tmacgbay)\
**Post date:** [June 7, 2022, 2:52pm UTC](https://community.graylog.org/t/new-request-for-custom-index-template/24030/7 "2022-06-07T14:52:39Z")

</div>

You want something like this:

```auto
curl -XPUT -H "Content-Type: application/json" \ '<ElasticServer>:9200/<Index_name>/_settings' \ -d '{"index.mapping.total_fields.limit": 2000 }

```

NOTE: Pulled from [here](https://www.elastic.co/guide/en/elasticsearch/reference/current/mapping-settings-limit.html) and [here](https://stackoverflow.com/questions/43136878/how-to-set-elasticsearch-index-mapping-total-fields-limit-to-unlimited).

---

<div class="post-metadata">

**Author:** ![Willy](https://avatars.discourse-cdn.com/v4/letter/w/c2a13f/32.png) [@Willy](https://community.graylog.org/u/Willy)\
**Post date:** [June 8, 2022, 9:57am UTC](https://community.graylog.org/t/new-request-for-custom-index-template/24030/8 "2022-06-08T09:57:40Z")

</div>

Dear Tmcgbay  
if you have more time for this case… in the following you’ll see my index “windows” settings:

## [Windows-index](https://graylog.pg.infn.it/system/index_sets/628cdcea999e5d54583b3c82) 2 indices, 63,759,830 documents, 13.9GiB

Windows systems logs  
Index prefix: windows  
Shards: 8  
Replicas: 0  
Field type refresh interval:5 seconds

Index rotation strategy: Index Time  
Rotation period: P1M (1 month, a month)  
Index retention strategy: Delete  
Max number of indices: 12

now I’ve got two index and your command works fine , but it can be applied only to the windows\_0  
and windows\_1 index, it doesn’t change the default value for the index windows\_2,3,4,… which will be generated when the rotation mechanism deems it appropriate.

I hope I have explained my case well.

Thanks  
Willy

---

<div class="post-metadata">

**Author:** ![tmacgbay](https://sea2.discourse-cdn.com/flex016/user_avatar/community.graylog.org/tmacgbay/32/3878_2.png) [@tmacgbay](https://community.graylog.org/u/tmacgbay)\
**Post date:** [June 8, 2022, 3:34pm UTC](https://community.graylog.org/t/new-request-for-custom-index-template/24030/9 "2022-06-08T15:34:54Z")

</div>

OK - what you really want is a [custom mapping](https://docs.graylog.org/docs/elasticsearch#custom-index-mappings).

Current settings of current index with:

```auto
curl -GET -H "Content-Type: application/json" '<elasticsearch>:9200/<index_name>/_settings?pretty'

```

Create a file that has a JSON settings for your index that contains the following

- index name is winbloat\_\* (covers all versions for future index rotations)
- file name is winbloat\_custom\_index.json
- Jump from 1000 default to 2000 total fields

```auto
{
  "template": "winbloat_*",
     "settings" : {
      "index" : {
        "mapping" : {
          "total_fields" : {
            "limit" : "2000"
          }
        }
      }
   }
}

```

Apply the custom template with:  
`curl -X PUT -H 'Content-Type: application/json' -d @'winbloat_custom_index.json' 'http://<ElasticSearch>:9200/_template/winbloat_custom_index?pretty'`

Remove it if you don’t like it with:

```auto
curl -X DELETE -H 'Content-Type: application/json' 'http://<ElasticSearch>:9200/_template/winbloat_custom_index?pretty'

```

---

<div class="post-metadata">

**Author:** ![Willy](https://avatars.discourse-cdn.com/v4/letter/w/c2a13f/32.png) [@Willy](https://community.graylog.org/u/Willy)\
**Post date:** [June 14, 2022, 12:42pm UTC](https://community.graylog.org/t/new-request-for-custom-index-template/24030/10 "2022-06-14T12:42:15Z")

</div>

> [@tmacgbay](#):
>
> `winbloat_`

Are you sure about winbloat name ? However I’ll try , many thanks again  
Enrico

---

<div class="post-metadata">

**Author:** ![tmacgbay](https://sea2.discourse-cdn.com/flex016/user_avatar/community.graylog.org/tmacgbay/32/3878_2.png) [@tmacgbay](https://community.graylog.org/u/tmacgbay)\
**Post date:** [June 14, 2022, 1:25pm UTC](https://community.graylog.org/t/new-request-for-custom-index-template/24030/11 "2022-06-14T13:25:22Z")

</div>

Didn’t know what your windows index name was so I took a guess… 🤪 🤣

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/flex016/uploads/graylog/original/3X/c/7/c7c09c6b5099570133d6502b83f50ba4430de5b6.png) [@system](https://community.graylog.org/u/system)\
**Post date:** [June 28, 2022, 1:25pm UTC](https://community.graylog.org/t/new-request-for-custom-index-template/24030/12 "2022-06-28T13:25:49Z")

</div>

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.