Query parsing error: maxClauseCount is set to 1024

[schmitz-ens]

Hello,
I’d like to trigger an alert with a large search query : 1500 IP address. Query won’t run, I get " Query parsing error : Cannot parse query, cause: maxClauseCount is set to 1024." in the UI.

• OS Information: docker-compose

• Package Version: Graylog 4.3.7 (single instance), ES 7.10.2 (single instance)

3. What steps have you already taken to try and solve the problem?

I tried to use indices.query.bool.max_clause_count:2048 in my elasticsearch.yml and restart, but I still have the error.

I also tried to find a way to stay under the limit of 1024, but I don’t see any reliable solution.

Any help would be appreciated !

[gsmith]

Hello @schmitz-ens && Welcome

@schmitz-ens

1500 IP address. Query won’t run, I get " Query parsing error : Cannot parse query, cause: maxClauseCount is set to 1024

Now that’s a lot of IP addresses to query.
I think its 150 items. If you want more result items, you’ll have to set the limit query parameter accordingly or page through the result set (see offset query parameter).

Take a look HERE

Add info:
I’m not suggestion this by al means but if you insist on querying 1500 items you might want to look at something like this command.

curl -X PUT localhost:9200/index_id/_settings -H ‘Content-Type: application/json’ -d'{   “index.mapping.total_fields.limit”: 1500}

[schmitz-ens]

Hello, and thx for your help.

@gsmith

Now that’s a lot of IP addresses to query.

Maybe I’m thinking this wrong : I want to setup an alert to check if one of my computers have been in contact with a list of “bad” IP address. So my search query looks like this : fw_dst_ip:(“XXX.XXX.XXX.XXX” OR “XXX.XXX.XXX.XXX” OR "XXX.XXX.XXX.XXX… And so on. It was working fine with a list of 600 IP address but now my “bad” IP address list is around 1500. Maybe I should implement this check another way ?

I tried to raise the index.mapping.total_fields.limit on my current index, but I still have the error “maxClauseCount is set to 1024”

Thx a lot.

[ihe]

I’d use a lookup table for this. Put in your 1500 IPs, or even more into them. You could even mark them with different tags.
In the next step use a pipeline to do a lookup, and if successfull add a field with that tag from the lookup table.
Then search for existence of that tag.

[tmacgbay]

I would also suggest a table - The do take up memory, particularly large ones so be aware of that…

[schmitz-ens]

Thx for your help and advices !!! I will use a lookup table and a pipeline.