Indexing data from syslog RFC 5424

Graylog Central (peer support)
, ,
1

Hello there,
I’m new to Graylog, so excuse me if this is a dumb question, but I’m trying to send syslog from an application called ADAudit Plus.

When I enable it to send syslog messages to Graylog, this is how it looks in Graylog:

"
[ Category = LogonReports ] [ REPORT_PROFILE = All Users Logon ] [ USERNAME = testuser ] [ CLIENT_IP_ADDRESS = 10.x.x.x ] [ CLIENT_HOST_NAME = testcomp.test.com ] [ TIME_GENERATED = 1550222219 ] [ RECORD_NUMBER = 105215901 ] [ EVENT_TYPE = 8 ] [ EVENT_TYPE_TEXT = Success ] [ DOMAIN = test ] [ SOURCE = test.test.com ] [ LOGON_SERVICE = krbtgt ] [ USER_SID = %{S-1-5-21-823518204-1963214169-725345543-15345} ] [ ERROR_CODE = 0x0 ] [ ERROR_CODE_TEXT = - ] [ EVENT_NUMBER = 4768 ] [ REMARKS = A Kerberos authentication ticket (TGT) was requested. ] [ PRE_AUTHENTICATION_TYPE = 2 ] [ TRANSITED_SERVICES = null ] [ TICKET_OPTIONS = 0x40810010 ] [ TICKET_ENCRYPTION_TYPE = 0x12 ] [ CLIENT_PORT = 53544 ] [ CERTIFICATE_THUMBPRINT = null ] [ CERTIFICATE_SERIAL_NUMBER = null ] [ CERTIFICATE_ISSUER_NAME = null ] [ USER_SAM_ACCOUNT_NAME = null ] [ USER_DISPLAY_NAME = null ] [ USER_PRINCIPAL_NAME = null ] [ USER_GUID = null ] [ USER_DISTINGUISH_NAME = null ] [ USER_OU_GUID = null ] [ USER_DEPARTMENT = null ] [ USER_MANAGER_NAME = null ] [ CLIENT_HOST_DOMAIN_NAME = null ] [ SOURCE_NAME = null ] [ LOG_FILE_NAME = null ] [ KEYWORDS_NAME = null ] [ TASK_CATEGORY_NAME = null ] [ TASK_CATEGORY_ID = null ] [ EXTRA_COLUMN1 = null ] [ EXTRA_COLUMN2 = null ] [ EXTRA_COLUMN3 = null ] [ EXTRA_COLUMN4 = null ] [ EXTRA_COLUMN5 = null ] [ EXTRA_COLUMN6 = null ] [ EXTRA_COLUMN7 = null ] [ EXTRA_COLUMN8 = null ] [ EXTRA_COLUMN9 = null ] [ EXTRA_COLUMN10 = null ] [ CONFIGURED_DOMAIN_NAME = null ]
"
(I’ve gotten it to work with nxlog and the GELF format, but I would prefer to just forward the message from our current AD audit software).

I have the option to change the “[” “=” “]” in the software, but I don’t know what to change to get Graylog to automatically index this information? I also have the option of sending the data with TFC3164, but the issue is the same.
I’ve used the UDP syslog input in Graylog.

Hope you can help

Thanks in advance.

2

so you try to send structured data - what is good - but you would need to tell Graylog that you have structured data and that it should extract the key-value pairs.

If possible you should configure your software to send something like:

Category=LogonReports, REPORT_PROFILE=All Users Logon, USERNAME=testuser, ....

This way you might be able to just check “expand structured data” in the UDP Syslog Input and the magic happens.

But if not, you would need to extract the data with a key-value processing pipeline or extractor and the above will be way easier to parse than your first suggested.

1 Like
3

Hi Jan,

Thank you for the reply.
I converted the input, now it looks like this:

Category = LogonReports,REPORT_PROFILE = All Users Logon,USERNAME = test,CLIENT_IP_ADDRESS = 10.x.x.x,CLIENT_HOST_NAME = testuser,TIME_GENERATED = 1550227373,RECORD_NUMBER = 105228498,EVENT_TYPE = 8,EVENT_TYPE_TEXT = Success,DOMAIN = TEST,SOURCE = test,LOGON_SERVICE = krbtgt,USER_SID = %{S-1-5-21-823518204-1965331169-725332543-10873},ERROR_CODE = 0x0,ERROR_CODE_TEXT = -,EVENT_NUMBER = 4768,REMARKS = A Kerberos authentication ticket (TGT) was requested.,PRE_AUTHENTICATION_TYPE = 2,TRANSITED_SERVICES = null,TICKET_OPTIONS = 0x40810010,TICKET_ENCRYPTION_TYPE = 0x12,CLIENT_PORT = 64311,CERTIFICATE_THUMBPRINT = null,CERTIFICATE_SERIAL_NUMBER = null,CERTIFICATE_ISSUER_NAME = null,USER_SAM_ACCOUNT_NAME = null,USER_DISPLAY_NAME = null,USER_PRINCIPAL_NAME = null,USER_GUID = null,USER_DISTINGUISH_NAME = null,USER_OU_GUID = null,USER_DEPARTMENT = null,USER_MANAGER_NAME = null,CLIENT_HOST_DOMAIN_NAME = null,SOURCE_NAME = null,LOG_FILE_NAME = null,KEYWORDS_NAME = null,TASK_CATEGORY_NAME = null,TASK_CATEGORY_ID = null,EXTRA_COLUMN1 = null,EXTRA_COLUMN2 = null,EXTRA_COLUMN3 = null,EXTRA_COLUMN4 = null,EXTRA_COLUMN5 = null,EXTRA_COLUMN6 = null,EXTRA_COLUMN7 = null,EXTRA_COLUMN8 = null,EXTRA_COLUMN9 = null,EXTRA_COLUMN10 = null,CONFIGURED_DOMAIN_NAME = null,

However checking “expand structured data” didn’t do the trick.
I’m not that good with programming, do you have an example of a key-value processing pipeline I could have look at to get me started?
I wonder if I used a pipeline to remote all the whitespaces would make “expand structured data” work? Or does “expand structured data” apply before pipeline processing?

Thanks.

1 Like
4

it does not work because of the space - if you can change to KEY=VALUE,KEY=VALUE it should work.

5

Hello again,

I’ve been fiddling with this for a couple of hours now and managed to remove the whitespace. It still doesn’t work though :confused:

6

Okay, I got it to work with a key=value extractor as configured on this page:
http://docs.graylog.org/en/2.4/pages/extractors.html

My only issue at this point is all the key=null values, does anyone have a way I can remove these fields?

1 Like
7

Dude, good job! You stuck with it and finally managed to make it work like you wanted! Sounds like a good start to the weekend!

8

I did!
I still have a bunch of Null values that I would like to get rid of though - any idea how I can make that happen?

9

I would try something with the processing pipelines and regex - try to match if a field is null and then remove that field.

10

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.