OpenSearch exception [type=too_many_nested_clauses, reason=Query contains too many nested clauses; maxClauseCount is set to 1024]

royalstar · August 11, 2024, 11:40am

When I try to use simple Query like “openvpn” AND “connected”, I get this error:

OpenSearch exception [type=too_many_nested_clauses, reason=Query contains too many nested clauses; maxClauseCount is set to 1024].

I used to use this query earlier and it worked.

Graylog 6.0.5
Openseach 2.16.0
Lucene 9.11.1

Has anyone had this problem?

gsmith · August 13, 2024, 4:08am

Hey @royalstar

The “easy” way out is to increase the indices.query.bool.max_clause_count limit in the configuration file to a higher value.
I think it used to be 1024 in ES 7 and now in ES 8 it has been raised to 4096.

Just be aware that doing so might harm the performance of your cluster and even bring nodes down depending on your data volume.

github.com/Graylog2/graylog2-server

using `OR` in a search query returns error `too_many_nested_clauses`

opened 10:45PM - 20 Dec 22 UTC

drewmiranda-gl

bug triaged

Executing a search query containing `OR` returns the following error: ``` U…nable to perform search query: OpenSearch exception [type=too_many_nested_clauses, reason=Query contains too many nested clauses; maxClauseCount is set to 1024]. ``` ![image](https://user-images.githubusercontent.com/107503402/208780389-6823f242-7444-494b-a2e7-627e44dc17aa.png) I can somewhat replicate this querying opensearch directly, for example: `/*/_search?q=?q=Allow%20OR%20Deny` . Whats interesting is that some indexes return results without any issue while others return the `too_many_nested_clauses` error. ![image](https://user-images.githubusercontent.com/107503402/208780581-f98fc29c-13df-4daa-8be3-14d87409a176.png) Limiting my graylog query to streams that return results without issue work correctly. Below are all the indexes that opensearch returned exceptions for when using `OR` in a search query: ``` gl_linux_auditbeat_150 too_many_nested_clauses Query contains too many nested clauses; maxClauseCount is set to 1024 gl_linux_auditbeat_151 too_many_nested_clauses Query contains too many nested clauses; maxClauseCount is set to 1024 gl_linux_auditbeat_152 too_many_nested_clauses Query contains too many nested clauses; maxClauseCount is set to 1024 gl_linux_auditbeat_153 too_many_nested_clauses Query contains too many nested clauses; maxClauseCount is set to 1024 gl_linux_auditbeat_154 too_many_nested_clauses Query contains too many nested clauses; maxClauseCount is set to 1024 gl_linux_auditbeat_155 too_many_nested_clauses Query contains too many nested clauses; maxClauseCount is set to 1024 gl_linux_auditbeat_156 too_many_nested_clauses Query contains too many nested clauses; maxClauseCount is set to 1024 gl_linux_auditbeat_157 too_many_nested_clauses Query contains too many nested clauses; maxClauseCount is set to 1024 gl_linux_auditbeat_158 too_many_nested_clauses Query contains too many nested clauses; maxClauseCount is set to 1024 gl_linux_auditbeat_159 too_many_nested_clauses Query contains too many nested clauses; maxClauseCount is set to 1024 gl_windows_common_133 too_many_nested_clauses Query contains too many nested clauses; maxClauseCount is set to 1024 gl_windows_common_138 too_many_nested_clauses Query contains too many nested clauses; maxClauseCount is set to 1024 gl_windows_common_139 too_many_nested_clauses Query contains too many nested clauses; maxClauseCount is set to 1024 gl_windows_common_140 too_many_nested_clauses Query contains too many nested clauses; maxClauseCount is set to 1024 gl_windows_common_143 too_many_nested_clauses Query contains too many nested clauses; maxClauseCount is set to 1024 gl_windows_common_144 too_many_nested_clauses Query contains too many nested clauses; maxClauseCount is set to 1024 gl_windows_common_145 too_many_nested_clauses Query contains too many nested clauses; maxClauseCount is set to 1024 gl_windows_common_146 too_many_nested_clauses Query contains too many nested clauses; maxClauseCount is set to 1024 gl_windows_common_147 too_many_nested_clauses Query contains too many nested clauses; maxClauseCount is set to 1024 gl_windows_common_148 too_many_nested_clauses Query contains too many nested clauses; maxClauseCount is set to 1024 gl_windows_common_150 too_many_nested_clauses Query contains too many nested clauses; maxClauseCount is set to 1024 gl_windows_common_151 too_many_nested_clauses Query contains too many nested clauses; maxClauseCount is set to 1024 gl_windows_common_153 too_many_nested_clauses Query contains too many nested clauses; maxClauseCount is set to 1024 gl_windows_common_154 too_many_nested_clauses Query contains too many nested clauses; maxClauseCount is set to 1024 gl_windows_common_155 too_many_nested_clauses Query contains too many nested clauses; maxClauseCount is set to 1024 gl_windows_common_157 too_many_nested_clauses Query contains too many nested clauses; maxClauseCount is set to 1024 gl_windows_common_158 too_many_nested_clauses Query contains too many nested clauses; maxClauseCount is set to 1024 gl_windows_security_132 too_many_nested_clauses Query contains too many nested clauses; maxClauseCount is set to 1024 gl_windows_security_137 too_many_nested_clauses Query contains too many nested clauses; maxClauseCount is set to 1024 gl_windows_security_143 too_many_nested_clauses Query contains too many nested clauses; maxClauseCount is set to 1024 gl_windows_security_144 too_many_nested_clauses Query contains too many nested clauses; maxClauseCount is set to 1024 gl_windows_security_146 too_many_nested_clauses Query contains too many nested clauses; maxClauseCount is set to 1024 gl_windows_security_149 too_many_nested_clauses Query contains too many nested clauses; maxClauseCount is set to 1024 gl_windows_security_150 too_many_nested_clauses Query contains too many nested clauses; maxClauseCount is set to 1024 gl_windows_security_153 too_many_nested_clauses Query contains too many nested clauses; maxClauseCount is set to 1024 gl_windows_security_154 too_many_nested_clauses Query contains too many nested clauses; maxClauseCount is set to 1024 gl_windows_security_156 too_many_nested_clauses Query contains too many nested clauses; maxClauseCount is set to 1024 gl_windows_security_157 too_many_nested_clauses Query contains too many nested clauses; maxClauseCount is set to 1024 graylog_2 too_many_nested_clauses Query contains too many nested clauses; maxClauseCount is set to 1024 ``` ## Expected Behavior Graylog returns search results when using the `OR` statement. ## Current Behavior Graylog doesn't return search results for some streams/indices. ## Possible Solution Unknown. not clear if this is related to https://github.com/opensearch-project/OpenSearch/issues/3652 but the fact that this can be replicated independent of graylog could mean its an issue with OpenSearch and not graylog. ## Steps to Reproduce (for bugs) 1. Execute a search query for `Deny OR Allow` ## Context I was testing various sigma rules and encountered this when a rule (`Django Framework Exceptions`) generated a query with several `OR` statements and generated the error described above. ## Your Environment * Graylog Version: 5.0.1 * Java Version: 17.0.5 * Elasticsearch Version: OpenSearch 2.4.1 * MongoDB Version: 5.0.14 * Operating System: Ubuntu Server 20.04 LTS * Browser version: Chrome 108.0.5359.124

royalstar · August 14, 2024, 9:38pm

Thank you @gsmith.
What is actual problem here and how to solve it right way?

gsmith · August 14, 2024, 10:02pm

hey @royalstar

OpenSearch is the issue.
The issue was found here

royalstar · August 16, 2024, 8:53am

Will downgrading Opensearch solve the problem?

Topic		Replies	Views
Query contains too many nested clauses Graylog Central (peer support) alert	3	3387	May 3, 2023
Dashboard Error - Unable to perform search query: OpenSearch exception Graylog Central (peer support) elastic , dashboards	3	2930	October 26, 2023
Query parsing error: maxClauseCount is set to 1024 Graylog Central (peer support) elastic	6	1884	December 1, 2022
OpenSearchException Graylog Central (peer support)	15	1880	May 11, 2023
Best Practice: ElasticSearch/OpenSearch? Graylog Central (peer support) elastic	12	2985	April 26, 2023

OpenSearch exception [type=too_many_nested_clauses, reason=Query contains too many nested clauses; maxClauseCount is set to 1024]

Related topics