Fields From Palo Syslogs

Graylog Central (peer support)
1

Long story short. We were using Graylog to capture our syslogs from our Palo firewalls. One of our servers died, and the Graylog VM was not being backed up.

I rebuild a new Graylog server to start connecting the syslogs from our Palo firewalls again.

But when I view a syslog message from the Palo in Graylog I am not seeing all the fields I use to. I currently have event_source_product, full_message, message, source, timestamp, and vendor_subtype.

I used to have fields showing the source and destination IP. There were fields showing the username and a bunch of other information. I can see all the data is there in the syslog message, but the data is not being parsed out into fields I can use to search with.

When I first created Graylog this was about a year ago, then our firewalls were running PANOS 10, and I think I used the Palo Alto input in Graylog.

Now our firewalls are running PANOS 11.2.4-h2, I am using the Palo Alto Networks TCP (PAN-OS-v11+) input in Graylog.

But I can’t figure out how to get Graylog to parse the other data into searchable fields.

2

Can you share a sanitized sample log message?
PANOS 11 codec uses Apache CSVParser to extract fields, just like the older versions, so unclear why it would be behaving differently.

3

Here is a sample log.

{
“event_source_product”: “PAN”,
“gl2_accounted_message_size”: 1936,
“gl2_receive_timestamp”: “2025-06-24 11:35:55.140”,
“vendor_subtype”: “THREAT”,
“gl2_remote_ip”: “192.168.6.41”,
“gl2_remote_port”: 49847,
“streams”: [
“6852ece450a4235323b5c02f”
],
“gl2_message_id”: “01JYGVJ9BR00F92CBZ9TDFCYS3”,
“source”: “Firewall.company.local”,
“message”: “1,2025/06/24 07:35:54,023201002425,THREAT,url,2818,2025/06/24 07:35:54,10.1.95.109,17.248.210.7,173.241.232.195,17.248.210.7,Remote Locations to Internet,ssl,vsys1,06 LAN,WAN_Zone,ethernet1/2.101,ethernet1/1,Syslog Forward,2025/06/24 07:35:55,20924,1,56966,443,9438,443,0x40b400,tcp,alert,"gateway-asset.icloud-content.com/",(9999),computer-and-internet-info,informational,client-to-server,7517298228159870177,0x0,10.0.0.0-10.255.255.255,United States,0,0,0,0,0,0,0,CDC-FW-01,0,0,N/A,N/A,AppThreat-0-0,0x0,0,4294967295,"computer-and-internet-info,low-risk",c0986931-2ba1-468b-97ae-3b71d4c5b3c9,0,0,2025-06-24T07:35:55.149-04:00,encrypted-tunnel,networking,browser-based,4,"used-by-malware,able-to-transfer-file,has-known-vulnerability,tunnel-other-application,pervasive-use",ssl,no,no,NonProxyTraffic,false,0,0,0,0”,
“gl2_source_input”: “6852eb0350a4235323b5b7f3”,
“gl2_processing_timestamp”: “2025-06-24 11:35:55.143”,
“full_message”: “<14>Jun 24 07:35:55 Firewall.company.local 1,2025/06/24 07:35:54,023201002425,THREAT,url,2818,2025/06/24 07:35:54,10.1.95.109,17.248.210.7,173.241.232.195,17.248.210.7,Remote Locations to Internet,ssl,vsys1,06 LAN,WAN_Zone,ethernet1/2.161,ethernet1/1,Syslog Forward,2025/06/24 07:35:55,20924,1,56966,443,9438,443,0x40b400,tcp,alert,"gateway-asset.icloud-content.com/",(9999),computer-and-internet-info,informational,client-to-server,7517298228159870177,0x0,10.0.0.0-10.255.255.255,United States,0,0,0,0,0,0,0,Firewall,0,0,N/A,N/A,AppThreat-0-0,0x0,0,4294967295,"computer-and-internet-info,low-risk",c0986931-2ba1-468b-97ae-3b71d4c5b3c9,0,0,2025-06-24T07:35:55.149-04:00,encrypted-tunnel,networking,browser-based,4,"used-by-malware,able-to-transfer-file,has-known-vulnerability,tunnel-other-application,pervasive-use",ssl,no,no,NonProxyTraffic,false,0,0,0,0”,
“gl2_source_node”: “51bb5ab7-7b3a-4b63-8387-e2b9e2a94ca1”,
“_id”: “63deb170-50ef-11f0-8898-0050569e1f4e”,
“gl2_processing_duration_ms”: 3,
“timestamp”: “2025-06-24T11:35:55.000Z”
}

4

Please use a code block when pasting logs. All the quotes got transformed to invalid characters (for JSON).

Perhaps those nested quotes in the message and full_messages fields are causing problems.

5

The 11.x input doesn’t appear to parse the logs beyond the fields listed in the documentation. We are still using the 9.x input until we can take the time to create our own parser.

" The PAN-OS 11 input automatically detects whether the ingested data is from version 11.0 or later and processes the log data using either processing pipelines or Illuminate content. This input does not fully parse the entire message schema, instead, it extracts key fields such as event_source_product and vendor_subtype, which are added to the message."

6

Are you using the 9.x inputs in Graylog? If so what PANOS are you running on your firewalls?

7

We are currently running 11.2.4-h7. We created the 9.x input when we were on PanOS 10.x and the 11.x input didn’t exist yet. The 9.x input parser might be ‘losing’ information as warned in the documentation but it is still suiting our needs.

8

That worked. Using the 9.x input can parse out way more information from the syslog than the 11.x input. It is odd I am running PANOS 11 but the PAN 11.x input does a worse job than the deprecated 9.x input at parsing the data.

9

Glad it worked even though it’s not ideal. Not sure exactly which fields are being dropped but one day I’ll look at it and write my own parser. We set up a new Graylog cluster and I couldn’t figure out why the Palo input wasn’t parsing anything. Went to the old environment and saw the input was still the 9.x one. Not sure why they would add a newer version of the input and not support any of the fields, even basic ones.

10

Here’s the backstory:

With the older (8.0, 9+) Palo inputs, we did all the parsing in the Java code. And every time Palo released a new PAN-OS version that changed their schema and added new fields, we fell behind. When Palo released PAN-OS v11, the schema change was serious enough that it started breaking people using the Palo v9+ input. So for the newest Palo input, we changed our approach. The Java codec now only parses out enough information to identify the log type with the expectation that pipelines have to be built to take over the rest.

For Enterprise customers, Illuminate handles this. We have a lot more flexibility when it comes to delivering new Illuminate content than we do with delivering new Java code and thus the Content Team can respond to Palo changes quicker than our backend devs ever could.

For non-customer users, it is an unfortunate outcome because we have stopped doing something we used to do. But the fact is we just couldn’t keep up. We were consistently multiple versions behind when it came to handling Palo content and our dev team is too small to make keeping up with Palo a priority.

11

Thanks much for the explanation Patrick! I’ll be looking into the illuminate packs once we can get our enterprise license but for now the 9.x input will work for us.

12

Thanks for the back story. Since the 9.x input works way better than the 11x input we will just use that with our firewalls running PANOS11.

What was breaking for people running PANOS11 using the 9.x input? For me using the 11.x input broke stuff.

Since the 9.x is a deprecated input is there any EOL to when GrayLog will remove that input?

13

No guarantees, but it’s likely to remain available for quite a while. As long as somebody is still using it and it’s not generating any maintenance effort, there’s no urgency to remove it.

14

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.