Graylog & PaloAlto - Limit of total fields has been exceeded?

[chrihub]

Im fairly new to Graylog and I’m very happy to work with this tool. But so maybe my question is quite easy to solve.

I’m trying to import PAN-OS 9.1 Logs into my Graylog 3.3 Installation. Therefore, I installed the Integrations Plugin and started working with the “Palo Alto Networks Input”.
When configuring my Firewall to send Logs to Graylog, I start receiving the following errormessages:

WARN [Messages] Failed to index message: index= id= error=<{“type”:“illegal_argument_exception”,“reason” :“Limit of total fields [1000] in index [graylog_499] has been exceeded”}>

Those messages do definitely not contain more than arround 120 Fields. In PaloAlto I configured the Logs to be sent alternately as IETF or BSD. There was no change.

Did anybody here faced the a similar Issue so far?

Kind Regards
Christian

[Ponet (Jesse Hills)]

See the below:

Graylog: ES Log: Limit of total fields [1000] in index [graylog_519] has been exceeded and no longer processing

The thread on the Elastic discussion boards you’ve linked to already shows how to increase the maximum number of fields per index/mapping, also see https://www.elastic.co/guide/en/elasticsearch/reference/5.5/mapping.html#mapping-limit-settings for details. This being said, I think having 1000 different fields in one index is a bit excessive and you should think about splitting up your messages in different index sets, so that each index has less than 1000 different fields: http://docs.graylog.o…

[chrihub]

Thanks for your feedback. Your feedback helped me to understand the error message. Mistakenly I interpreted the Errormessage in that way that one incoming Logmessage contains more than 1000 Fields what is not the case. I’m going to split up the indexes as you suggested.