# Graylog search is slow

**URL:** <https://community.graylog.org/t/graylog-search-is-slow/36172>\
**Category:** Graylog Central (peer support)\
**Created:** [August 6, 2025, 9:01am UTC](https://community.graylog.org/t/graylog-search-is-slow/36172 "2025-08-06T09:01:51Z")\
**Posts on this page:** 20\
**Page:** 1

<div class="post-metadata">

**Author:** ![glennfernandez83](https://sea2.discourse-cdn.com/flex016/user_avatar/community.graylog.org/glennfernandez83/32/11029_2.png) [@glennfernandez83](https://community.graylog.org/u/glennfernandez83)\
**Post date:** [August 6, 2025, 9:01am UTC](https://community.graylog.org/t/graylog-search-is-slow/36172/1 "2025-08-06T09:01:51Z")

</div>

**1. Describe your incident:**

graylog search is slow and returning this error  
**Error!**

Executing search failed: SyntaxError: Unexpected token ‘i’, “timeout” is not valid JSON

**2. Describe your environment:**

we are using graylog 6.3.1 running on AWS ecs fargate with 1 master node and 1 slave node. DB is aws opensearch 2.19 running on 6 r7g.large.search

**3. What steps have you already taken to try and solve the problem?**

initially, we are running aws opensearch 3 r7g.medium.search which we have already upgraded and increased the node to 6 but still slow. we have also updated graylog to latest and opensearch from 2.17 to 2.19 but still getting the same error.

---

<div class="post-metadata">

**Author:** ![konrad](https://sea2.discourse-cdn.com/flex016/user_avatar/community.graylog.org/konrad/32/1661_2.png) [@konrad](https://community.graylog.org/u/konrad)\
**Post date:** [August 7, 2025, 7:47am UTC](https://community.graylog.org/t/graylog-search-is-slow/36172/2 "2025-08-07T07:47:39Z")

</div>

Please provide more context from your server.log. Or where did you see this error?

---

<div class="post-metadata">

**Author:** ![glennfernandez83](https://sea2.discourse-cdn.com/flex016/user_avatar/community.graylog.org/glennfernandez83/32/11029_2.png) [@glennfernandez83](https://community.graylog.org/u/glennfernandez83)\
**Post date:** [August 7, 2025, 9:29am UTC](https://community.graylog.org/t/graylog-search-is-slow/36172/3 "2025-08-07T09:29:52Z")

</div>

came from graylog console.

---

<div class="post-metadata">

**Author:** ![glennfernandez83](https://sea2.discourse-cdn.com/flex016/user_avatar/community.graylog.org/glennfernandez83/32/11029_2.png) [@glennfernandez83](https://community.graylog.org/u/glennfernandez83)\
**Post date:** [August 7, 2025, 9:33am UTC](https://community.graylog.org/t/graylog-search-is-slow/36172/4 "2025-08-07T09:33:15Z")

</div>

![image](https://us1.discourse-cdn.com/flex016/uploads/graylog/original/3X/f/a/fa3ca940c6c6451e8c3bb20ca74c363bc32effae.png)

---

<div class="post-metadata">

**Author:** ![janheise](https://avatars.discourse-cdn.com/v4/letter/j/c89c15/32.png) [@janheise](https://community.graylog.org/u/janheise)\
**Post date:** [August 7, 2025, 1:14pm UTC](https://community.graylog.org/t/graylog-search-is-slow/36172/5 "2025-08-07T13:14:23Z")

</div>

Hi, thank you for supplying more info. You write “graylog search is slow” - but the image shows exceptions regarding bulk indexing - which has nothing to do with searching.  
Also, the errors shown mention a NullPointerException not a Syntax error from your first post.

Can you please clarify your initial question and provide additional info?

---

<div class="post-metadata">

**Author:** ![glennfernandez83](https://sea2.discourse-cdn.com/flex016/user_avatar/community.graylog.org/glennfernandez83/32/11029_2.png) [@glennfernandez83](https://community.graylog.org/u/glennfernandez83)\
**Post date:** [August 8, 2025, 2:16am UTC](https://community.graylog.org/t/graylog-search-is-slow/36172/6 "2025-08-08T02:16:39Z")

</div>

before I upgrade aws opensearch from 2.17 to 2.19, the logs are not displaying on the graylog console and it returns this error.  
Executing search failed: SyntaxError: Unexpected token ‘i’, “timeout” is not valid JSON

after upgrading to aws opensearch 2.19, I am now getting the error attached on the screenshot.

 ![image](https://us1.discourse-cdn.com/flex016/uploads/graylog/original/3X/2/4/24bea0a8fd7405d4f2e75e9b797bdef6935edb9f.png)

---

<div class="post-metadata">

**Author:** ![janheise](https://avatars.discourse-cdn.com/v4/letter/j/c89c15/32.png) [@janheise](https://community.graylog.org/u/janheise)\
**Post date:** [August 11, 2025, 8:04am UTC](https://community.graylog.org/t/graylog-search-is-slow/36172/7 "2025-08-11T08:04:20Z")

</div>

Can we go back to your upgrade process? From what version(s) did you start? How long has this been running without problems?  
I assume it worked with your old version of Graylog and OpenSearch 2.17? Am I correct? And after the Graylog update to 6.3.1 the problems started? And then you updated to 2.19?

How does the “search is slow” come into play? That was already with the old, running versions?

---

<div class="post-metadata">

**Author:** ![glennfernandez83](https://sea2.discourse-cdn.com/flex016/user_avatar/community.graylog.org/glennfernandez83/32/11029_2.png) [@glennfernandez83](https://community.graylog.org/u/glennfernandez83)\
**Post date:** [August 11, 2025, 9:34am UTC](https://community.graylog.org/t/graylog-search-is-slow/36172/8 "2025-08-11T09:34:38Z")

</div>

more than 2 years already without any problem. even prior to to upgrade, we already have the slow issue. now even the default search with 5 mins time frame is not displaying.

---

<div class="post-metadata">

**Author:** ![Wine\_Merchant](https://sea2.discourse-cdn.com/flex016/user_avatar/community.graylog.org/wine_merchant/32/14596_2.png) [@Wine\_Merchant](https://community.graylog.org/u/Wine_Merchant)\
**Post date:** [August 11, 2025, 11:51am UTC](https://community.graylog.org/t/graylog-search-is-slow/36172/9 "2025-08-11T11:51:09Z")

</div>

Hello @glennfernandez83,

Another user who experienced [the same error](https://community.graylog.org/t/elasticsearch-exception-reason-key-types-is-not-supported-in-the-metadata-section/27468/6) contained within your screenshot found that removing the below option from the opensearch.yml file fixed the issue.

`compatibility.override_main_response_version: true`

---

<div class="post-metadata">

**Author:** ![glennfernandez83](https://sea2.discourse-cdn.com/flex016/user_avatar/community.graylog.org/glennfernandez83/32/11029_2.png) [@glennfernandez83](https://community.graylog.org/u/glennfernandez83)\
**Post date:** [August 12, 2025, 1:29am UTC](https://community.graylog.org/t/graylog-search-is-slow/36172/10 "2025-08-12T01:29:57Z")

</div>

i am using aws opensearch service

---

<div class="post-metadata">

**Author:** ![Wine\_Merchant](https://sea2.discourse-cdn.com/flex016/user_avatar/community.graylog.org/wine_merchant/32/14596_2.png) [@Wine\_Merchant](https://community.graylog.org/u/Wine_Merchant)\
**Post date:** [August 12, 2025, 8:06am UTC](https://community.graylog.org/t/graylog-search-is-slow/36172/11 "2025-08-12T08:06:19Z")

</div>

When you initially upgraded to OS 2.17, what version were you on? Moving forward you should use [this compatibility matrix](https://go2docs.graylog.org/current/downloading_and_installing_graylog/compatibility_matrix.htm) to make sure you are upgrading to a supported version.

Check the option is enabled by running the below.

```auto
curl -X GET https://user:password@URL:443/_cluster/settings?include_defaults=true

```

If it is you can use this next command to disable

```auto
curl -X PUT https://user:password@URL:443/_cluster/settings -H 'Content-Type: application/json' -d '
{
  "persistent" : {
    "compatibility.override_main_response_version" : false
  }
}'

```

---

<div class="post-metadata">

**Author:** ![glennfernandez83](https://sea2.discourse-cdn.com/flex016/user_avatar/community.graylog.org/glennfernandez83/32/11029_2.png) [@glennfernandez83](https://community.graylog.org/u/glennfernandez83)\
**Post date:** [August 13, 2025, 2:16am UTC](https://community.graylog.org/t/graylog-search-is-slow/36172/12 "2025-08-13T02:16:21Z")

</div>

it was 2.17 before we upgraded to 2.19.

---

<div class="post-metadata">

**Author:** ![glennfernandez83](https://sea2.discourse-cdn.com/flex016/user_avatar/community.graylog.org/glennfernandez83/32/11029_2.png) [@glennfernandez83](https://community.graylog.org/u/glennfernandez83)\
**Post date:** [August 13, 2025, 11:31am UTC](https://community.graylog.org/t/graylog-search-is-slow/36172/13 "2025-08-13T11:31:26Z")

</div>

> [@Wine\_Merchant](#):
>
> ```auto
> curl -X PUT https://user:password@URL:443/_cluster/settings -H 'Content-Type: application/json' -d '
> {
> "persistent" : {
> "compatibility.override_main_response_version" : false
> }
> }'
> 
> ```

this worked and we can now see our logs. however there’s some notifications

 ![image](https://us1.discourse-cdn.com/flex016/uploads/graylog/original/3X/e/e/ee5fae6acd32758dbf4aed2fddd96f13fa654029.png)

---

<div class="post-metadata">

**Author:** ![Wine\_Merchant](https://sea2.discourse-cdn.com/flex016/user_avatar/community.graylog.org/wine_merchant/32/14596_2.png) [@Wine\_Merchant](https://community.graylog.org/u/Wine_Merchant)\
**Post date:** [August 13, 2025, 12:39pm UTC](https://community.graylog.org/t/graylog-search-is-slow/36172/14 "2025-08-13T12:39:53Z")

</div>

Check the server.conf of you Graylog nodes for the is\_leader option, it should be true on one node and false on all others. Make the change and restart the Graylog service.

---

<div class="post-metadata">

**Author:** ![glennfernandez83](https://sea2.discourse-cdn.com/flex016/user_avatar/community.graylog.org/glennfernandez83/32/11029_2.png) [@glennfernandez83](https://community.graylog.org/u/glennfernandez83)\
**Post date:** [August 13, 2025, 1:16pm UTC](https://community.graylog.org/t/graylog-search-is-slow/36172/15 "2025-08-13T13:16:04Z")

</div>

![image](https://us1.discourse-cdn.com/flex016/uploads/graylog/original/3X/a/2/a2e6619ee98d1d4b940929d9c94e094eedbff104.png)  
i have that option on 1 node and false on other node.

---

<div class="post-metadata">

**Author:** ![Wine\_Merchant](https://sea2.discourse-cdn.com/flex016/user_avatar/community.graylog.org/wine_merchant/32/14596_2.png) [@Wine\_Merchant](https://community.graylog.org/u/Wine_Merchant)\
**Post date:** [August 13, 2025, 1:42pm UTC](https://community.graylog.org/t/graylog-search-is-slow/36172/16 "2025-08-13T13:42:18Z")

</div>

That’s good.

If you close the notifications, do they reappear?

---

<div class="post-metadata">

**Author:** ![glennfernandez83](https://sea2.discourse-cdn.com/flex016/user_avatar/community.graylog.org/glennfernandez83/32/11029_2.png) [@glennfernandez83](https://community.graylog.org/u/glennfernandez83)\
**Post date:** [August 14, 2025, 1:53am UTC](https://community.graylog.org/t/graylog-search-is-slow/36172/17 "2025-08-14T01:53:09Z")

</div>

yes, still come back.

---

<div class="post-metadata">

**Author:** ![Wine\_Merchant](https://sea2.discourse-cdn.com/flex016/user_avatar/community.graylog.org/wine_merchant/32/14596_2.png) [@Wine\_Merchant](https://community.graylog.org/u/Wine_Merchant)\
**Post date:** [August 14, 2025, 8:51am UTC](https://community.graylog.org/t/graylog-search-is-slow/36172/18 "2025-08-14T08:51:02Z")

</div>

The No leader node error can occur when resource utilisation is high on the Mongo/Graylog node but the multiple leader issue is less clear.

At this point it would be a good idea to paw through the Graylog server logs to see if that gives a clearer picture of what is occurring.

---

<div class="post-metadata">

**Author:** ![glennfernandez83](https://sea2.discourse-cdn.com/flex016/user_avatar/community.graylog.org/glennfernandez83/32/11029_2.png) [@glennfernandez83](https://community.graylog.org/u/glennfernandez83)\
**Post date:** [August 26, 2025, 10:22am UTC](https://community.graylog.org/t/graylog-search-is-slow/36172/19 "2025-08-26T10:22:54Z")

</div>

I checked this today and can see that opensearch 2.19 is already supported. since we are running aws opensearch 2.19, I decided to set this to true

curl -X PUT [https://user:password@URL:443/\_cluster/settings](https://user:password@URL:443/_cluster/settings) -H ‘Content-Type: application/json’ -d ’  
{  
“persistent” : {  
“compatibility.override\_main\_response\_version” : false  
}  
}’

however, I got this error instead.  
The Indexer version which is currently running (Elasticsearch:7.10.2) has a different major version than the one the leader node was started with (OpenSearch:2.19.0). This will most probably result in errors during indexing or searching. A full restart is required after a major Indexer version upgrade.

I restarted my graylog master and node but still error persist and logs were not showing again.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/flex016/uploads/graylog/original/3X/c/7/c7c09c6b5099570133d6502b83f50ba4430de5b6.png) [@system](https://community.graylog.org/u/system)\
**Post date:** [September 9, 2025, 10:23am UTC](https://community.graylog.org/t/graylog-search-is-slow/36172/20 "2025-09-09T10:23:34Z")

</div>

Closed on September 9, 2025, 10:23am UTC

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.