NSX: Empty Dynamic Address Group on Firewalls - Knowledge Base - Palo Alto Networks

NSX: Empty Dynamic Address Group on Firewalls

11108

Created On 03/12/20 23:48 PM - Last Modified 04/04/20 00:49 AM

VMware NSX

8.1

8.0

9.0

9.1

PAN-OS

Panorama

VM-Series

Symptom

Panorama shows IP addresses being populated on Address Groups however the managed firewalls do not have any IP’s registered on Dynamic Address Group.

Run the below CLI command on PA-VM to verify if any IP addresses are being registered on the firewall

> show object registered-ip all

Next run the below CLI command on PA-VM firewall to verify the last “register” event for the IP in question. In this case IP’s were pushed to firewall about 4 hours ago.

> show log iptag datasource_type equal xml-api ip in x.x.x.x

Set logging level on useridd process on PA-VM to “debug” by using below CLI:

> debug user-id on debug

Manually trigger “Synchronize Dynamic Objects” on Panorama Service Manager and monitor useridd on PA-VM firewall to check below events:

useridd Logs:

2020-01-31 10:26:10.146 -0800 clear all registered ip adddresses upon XMLAPI request
2020-01-31 10:26:29.724 -0800 Processing dnld delta : 4, full : 39
2020-01-31 10:26:29.724 -0800 dnld 4 registered ip takes 0 second

Above logs snippet shows a working scenario for registered IP’s being downloaded from Panorama however in this case no such log events were observed.
Verify resource utilization on Panorama. In this case, the management CPU was running high.
Adopt PAN-OS troubleshooting steps to identify the root cause for high CPU. In this case, it was the elasticSearch process consuming high CPU due to heavy log indexing

Environment

Platform: PA-VM
Deployment: Operations Centric

Cause

Log Ingestion rate seen under mp-monitor log was 47K logs/s which is above the system supported limit for log ingestion in hybrid or mixed mode.

https://knowledgebase.paloaltonetworks.com/KCSArticleDetail?id=kA10g000000Clc8CAC

Resolution

[Temp Fix] Kill the elasticSearch 'es' process from root to bring the load down to 26% temporarily and triggered “Synchronize Dynamic Objects” under Panorama > Service Manager to register the IP’s on Firewall
Logs Forwarded should be reduced / re-designed so that log ingestion rate on Panorama lies within the supported limits.

Other users also viewed:

How to download GlobalProtect from the Customer Support Portal

Download and Install the GlobalProtect App for Windows

Resource List: GlobalProtect Configuring and Troubleshooting

PAN-OS Software Updates

Where to find the current preferred software versions? (PAN-OS, GlobalProtect, User-ID Agent, Plugins)