Missing event data field

We are using Graylog for (amongst other things) integration with O365 using CAS. It’s very easy to set up:

Microsoft provide a Java agent which connects to your O365 tenancy, downloads events and re-writes them back out as syslog to our Graylog input.

However, we have realised we are missing the IPv6 data field from the activity log.

An example activity log (from the MS page above:)

2017-11-22T17:50:04.000Z CEF:0|MCAS|SIEM_Agent|0.111.85|EVENT_CATEGORY_LOGOUT|Log out|0|externalId=1511373015679_167ae3eb-ed33-454a-b548-c2ed6cea6ef0 rt=1511373004000 start=1511373004000 end=1511373004000 msg=Log out suser=admin@contoso.com destinationServiceName=ServiceNow dvc= requestClientApplication= cs1Label=portalURL cs1=https://contoso.portal.cloudappsecurity.com/#/audits?activity.id=eq(1511373015679_167ae3eb-ed33-454a-b548-c2ed6cea6ef0,) cs2Label=uniqueServiceAppIds cs2=APPID_SERVICENOW cs3Label=targetObjects cs3=admin@contoso.com,admin@contoso.com,admin@contoso.com cs4Label=policyIDs cs4= c6a1Label=“Device IPv6 Address” c6a1=[IPv6 address goes here]

Mostly this parses correctly. However the c6a1 field is completely missing from Graylog. My suspicion is that it is because of the label format:

c6a1Label=“Device IPv6 Address”

It is the only label field with has quotes and whitespace. The field literally does not exist within our Graylog environment, whereas this for example:

cs2Label=uniqueServiceAppIds cs2=APPID_SERVICENOW

parses properly, and we can see uniqueServiceAppIds in searches etc.

I cannot find any errors, so I am guessing Graylog just isn’t recognizing this as a valid syslog field. Has anyone else seen similar and have any suggestions as to how we can preserve this field?



How are you parsing the message? Are you using an extractor or are you doing it in a pipeline?

Post details and use the formatting </> for code and messages so it is easier to read…

We use pipelines to do things like match IP addresses to threat intelligence feeds etc. But I don’t believe that is relevant here - this is a field missing from the input itself. There is nothing for me to run an extractor on or to enrich in a pipeline.

Screenshot 2020-08-18 at 14.21.29

This is a CEF UDP input. We are sending syslog messages in CEF format as I describe above.

This is a brief description of CEF:


That Microsoft O365 event, in CEF format, should be parsed properly. When I look at the input and show messages, I can see it almost is except for the final IPv6 field.

According to the CEF standard I would expect there to be a field called:

"Device IPv6 Address"

with a corresponding data field containing an IPv6 address. But this simply doesn’t exist, even on events where I know that field was present and populated. I suspect it is because the quotes and/or the whitespace are causing an issue somewhere.

Can I force an input to retain a particular field type? Or is there anything else we can do to ensure this field isn’t apparently just discarded?



It may be a bug in the CEF Input or perhaps CEF is not supposed to accept spaces in field data?.. which would seem odd. You could submit it up to Github as a bug. In the meantime I would ingest the message RAW and either use an extractor or pipeline rules to parse out the fields. It would certainly give more control of the results.

Ok, makes sense. I’ll try that. Thanks.

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.