Graylog v3.3 - simple search with numbers (usually) fail

Graylog Central (peer support)
1

I send logs from pfsense to graylog and am trying to find lines which block access to certain port, for example.

Log entries I have look like this:

full_message
<134>Sep 23 12:16:51 filterlog: 5,1000103483,em0,match,block,in,4,0x0,241,54321,0,none,6,tcp,40,[src ip],[dst ip],54615,80,0,SA,1245964053,2966902018,65535,

message
filterlog: 5,1000103483,em0,match,block,in,4,0x0,241,54321,0,none,6,tcp,40,[src ip],[dst ip],54615,80,0,SA,1245964053,2966902018,65535,

But if I try to search for certain things, most of my searches simply don’t return anything. For example:

these work:
source:“filterlog:” AND full_message:(block AND “Sep 23 12:16:51” AND “1000103483”)
source:“filterlog:” AND full_message:(block AND “Sep 23 12:16:51” AND “SA”)

but these don’t work:
source:“filterlog:” AND full_message:(block AND “Sep 23 12:16:51” AND “80”)
source:“filterlog:” AND full_message:(block AND “445”)
source:“filterlog:” AND full_message:(block AND “241”)
source:“filterlog:” AND full_message:(block AND “0x0”)

So if I wanted to search for lines where access to port 445 was blocked, it doesn’t work.

I can’t understand what I’m doing wrong?? I even just updated to v3.3.5 to see if it was a bug that was fixed recently, but no help.

And secondary question related to same entries:
I am sending those logs from pfsense using “remote rsyslog”. Does anybody know what I need to do to fix the “source” from one pfsense to something like “pfsense1” instead of (very annoying) “pfsense-module:”, which doesn’t even show which pfsense is actually sending them?

Thanks!

.mika

2

Best way if you want to search for values like port number and so on is to extract fields from message and search in extracted fields, like dst_port:80

Check this nice article:

To fix source you can you this little pipeline rule, which replace filterlog: to ip address of sending device (uses graylog internal field gl2_remote_ip):

rule "Pfsense replace source by ip"
when
  has_field("source") and contains(to_string($message.source), "filterlog")
then
  set_field("source", to_string($message.gl2_remote_ip));
end
1 Like
3

Ah, thank you! And sorry for delay, your reply is appreciated. I shall read that article, probably useful in other cases as well… :slight_smile:

I ended up setting up individual inputs for pfsenses and overriding the source there. That way I don’t have to update the rules if/when I enable new logging options there…

4

But the original problem remains: if full_message and message are in fact text strings, why searching for numeric values fail?

Is this one of those “it is what it is”-things?? :slight_smile:

5

Graylog uses Elastic search’s standard analyzer to index words, it creates terms by which you can search. I doesn’t mean that all numbers and phases are analyzed as in original text. Analyzer and tokenizer in ES try to simplify them to terms to quick search. So your message with words/numbers separated by , is not analyzed as you expect, so you can’t search in it. For example, if values would be separated by space, it should by searchable easily.



1 Like
6

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.