Cannot aggregate VPN user data where values contain email address

1. Describe your incident:
When building an aggregate for VPN login events with user values containing email addresses and login names the values with email address will not be included. Rows where user value is just the username data will be returned.

2. Describe your environment:

  • OS Information: centos 8

  • Package Version: 4.3.7

3. What steps have you already taken to try and solve the problem?
Removing user value in aggregate returns all rows. Tried increasing limit. Tried limiting returned rows to just the values including email addresses.

4. How can the community help?
Is there a setting I’m missing or do I have to somehow escape the values for email addresses t work correctly in aggregates?

Hello @mrhill741

Can you show an example of your message received?
You have a few option that can get a email address from an event.

  1. GROK
  2. REGEX
  3. Pipeline
  4. Extractor

When using the standard search and use the same filters as my chart I get results so it seems to be only with the grouping in the charts. Only difference that I can see between returned values and skipped is the skipped values have a full email address in the user field.

Example message that is not included:
<190>date=2022-09-18 time=07:44:26 devname=“FIREWALL1234567” devid=“FIREWALL1234567” eventtime=1663512266057608431 tz=“-0700” logid=“0101039425” type=“event” subtype=“vpn” level=“information” vd=“root” logdesc=“SSL VPN tunnel down” action=“tunnel-down” tunneltype=“ssl-web” tunnelid=639187983 remip=10.10.10.10 user="user@domain.com" group=“vpn-group” dst_host=“N/A” reason=“User requested termination of service” duration=260 sentbyte=0 rcvdbyte=0 msg=“SSL tunnel shutdown”

Example Message that is always returned even with grouping:
<190>date=2022-09-18 time=07:44:26 devname=“FIREWALL1234567” devid=“FIREWALL1234567” eventtime=1663512266057608431 tz=“-0700” logid=“0101039425” type=“event” subtype=“vpn” level=“information” vd=“root” logdesc=“SSL VPN tunnel down” action=“tunnel-down” tunneltype=“ssl-web” tunnelid=639187983 remip=10.10.10.10 user=“user” group=“vpn-group” dst_host=“N/A” reason=“User requested termination of service” duration=260 sentbyte=0 rcvdbyte=0 msg=“SSL tunnel shutdown”

Hello,

I have multiply Fortinet/FortiGate Firewall. Done something similar to this.

image

I made a GROK pattern.

firewall_user	(?>(?<!\\)(?>"(?>\\.|[^\\"]+)+"|""|(?>'(?>\\.|[^\\']+)+')|''|(?>`(?>\\.|[^\\`]+)+`)|``))

Then added it to extractor.

you probably could simplify this into a regex pattern only.

hope that helps

I did not think about my Extractor for the Firewall Input. I updated my User extractor with regex:
^.\suser="(.?)"\s+

and I now get both short user and user with email addresses.

Thanks for the hint!

1 Like