For this small setup running on a study LAB environment i keep on getting ‘limit of total fields has been exceeded’ This with a setup that has not changed profoundly.
As this is a lab setup it sometimes happens machines are disconnected for some time and spew a lot of logs in a short time. Or i enable logs options which generated for more than anticipated.
If with even a small setup this message comes into scope i don’t trust this for a far bigger environment. WIll i be shuffeling index organisation continuously ? Should i consider creating indexes per log type or per device or device group to avoid any messing with number of fields etc ?
As you did not share how you ingest what kind of data the general advice is - check if all data need to be in the same index. Check if you need all populated indices. Collect only what you want/need or plan carefully what you store how.
Can I deduce creating an index per data source/type would be most beneficial ? So one index for say nxlog sending eventlogs and one for nxlog sending syslog to a GELF input would be most sensible instead of simpy collecting all GELF input into one index ?
