We have been using graylog for sorting system logs, but today when i tried to use same for Apache logs it doesn`t seem to work as needed. Issue cropped up when i tried to search Apache logs ( After Grok Pattern applied ) in Graylog UI and filter them based on request size bigger than 1024 ( Graylog search using “byte” parameter). Search output never returns output as intended with results still showing logs of requests below 1024.
Can anyone shed some light if i need to enable it from config or i am making some mistake. I know we are running an outdated version but still it shouldn`t be a problem.
Log Line :192.168.0.155 - - [30/May/2017:01:41:46 -0700] “POST /testing.php HTTP/1.1” 20041 (this value to be searched above 1024)“https://www.----.com/” “Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/58.0.3029.110 Safari/537.36”
Graylog Search : gl2_source_input:3dvbf2eb78crrca5aa26521a AND verb:POST AND bytes:>1000
OR
Graylog Search : gl2_source_input:3dvbf2eb78crrca5aa26521a AND verb:POST AND bytes:[1000 TO 10000]
I’m not seeing anything wrong with your query. However, I too have had some very odd results when doing range searches in the past.
I’d verify your data is consistently being submitted as an Integer for this field. I think this was perhaps part of our problem until we made sure certain fields (http status codes, etc) were always integer when submitted. What I found is that even if the field is of type String, you can still do range searches (I just tested this again with the same odd ball results I was expecting; I did a “fieldname:>100” on a fieldname I know is always String and I had results returned).
The data type of the first value submitted for a field will be that field’s data type for the existence of the Index (e.g. if someone submits “200 OK” for an http_status field instead of using 200 [integer], that field will have the data type of String for that Index’s entire existence; next time the Index rolls over, the roulette wheel is spun again).
One way to check is to go in to Graylog as an Admin and on the System/Overview page, look at the Indexer Failures section. (I think this is also logged in the server.log output file.) You’ll see messages about failed message insertions because of incompatible data types, and the field name in question should be listed.
I’m not sure what else to check, but I’d recommend at least confirming the above. Hope this helps.
