The VM-Series Firewall in OCI environment unexpectedly reboots
12156
Created On 11/25/19 19:09 PM - Last Modified 12/01/20 21:40 PM
Symptom
- System log indicates the following DP restarts:
admin@PA-VM> show log system severity equal critical | match rebooting 2019/10/28 14:51:42 critical general general 0 data_plane: restarts exhausted, rebooting system 2019/10/16 16:57:14 critical general general 0 data_plane: restarts exhausted, rebooting system 2019/10/16 16:44:34 critical general general 0 data_plane: restarts exhausted, rebooting system
- System files indicates crashes consistent with the system DP reboots:
admin@PA-VM> show system files /opt/panlogs/cores/: total 4.0K drwxrwxrwx 2 root root 4.0K Oct 16 16:46 crashinfo /opt/panlogs/cores/crashinfo: total 0 /var/cores/: total 2.1M -rw-r--r-- 1 root root 407K Oct 16 17:00 all_task_4_8.1.9-h4_0.tar.gz -rw-r--r-- 1 root root 408K Oct 16 17:00 all_task_6_8.1.9-h4_0.tar.gz -rw-r--r-- 1 root root 408K Oct 16 17:00 all_task_5_8.1.9-h4_0.tar.gz -rw-r--r-- 1 root root 412K Oct 28 14:45 all_task_1_8.1.9-h4_8.tar.gz drwxr-xr-x 2 root root 4.0K Oct 28 14:50 crashinfo -rw-r--r-- 1 root root 410K Oct 28 15:00 all_task_2_8.1.9-h4_8.tar.gz /var/cores/crashinfo: total 1004K -rw-rw-rw- 1 root root 38K Oct 16 16:37 all_task_1_8.1.9-h4_0.info -rw-rw-rw- 1 root root 38K Oct 16 16:37 all_task_2_8.1.9-h4_0.info -rw-rw-rw- 1 root root 38K Oct 16 16:39 all_task_1_8.1.9-h4_1.info -rw-rw-rw- 1 root root 38K Oct 16 16:39 all_task_2_8.1.9-h4_1.info -rw-rw-rw- 1 root root 38K Oct 16 16:41 all_task_1_8.1.9-h4_2.info -rw-rw-rw- 1 root root 38K Oct 16 16:42 all_task_2_8.1.9-h4_2.info -rw-rw-rw- 1 root root 38K Oct 16 16:43 all_task_1_8.1.9-h4_3.info -rw-rw-rw- 1 root root 29K Oct 16 16:49 all_task_1_8.1.9-h4_4.info -rw-rw-rw- 1 root root 29K Oct 16 16:50 all_task_2_8.1.9-h4_3.info -rw-rw-rw- 1 root root 29K Oct 16 16:50 all_task_4_8.1.9-h4_0.info -rw-rw-rw- 1 root root 29K Oct 16 16:50 all_task_5_8.1.9-h4_0.info -rw-rw-rw- 1 root root 29K Oct 16 16:51 all_task_6_8.1.9-h4_0.info -rw-rw-rw- 1 root root 29K Oct 16 16:51 all_task_1_8.1.9-h4_5.info -rw-rw-rw- 1 root root 29K Oct 16 16:52 all_task_2_8.1.9-h4_4.info -rw-rw-rw- 1 root root 29K Oct 16 16:52 all_task_4_8.1.9-h4_1.info -rw-rw-rw- 1 root root 29K Oct 16 16:53 all_task_5_8.1.9-h4_1.info -rw-rw-rw- 1 root root 29K Oct 16 16:53 all_task_6_8.1.9-h4_1.info -rw-rw-rw- 1 root root 29K Oct 16 16:54 all_task_1_8.1.9-h4_6.info -rw-rw-rw- 1 root root 29K Oct 16 16:54 all_task_2_8.1.9-h4_5.info -rw-rw-rw- 1 root root 29K Oct 16 16:54 all_task_4_8.1.9-h4_2.info -rw-rw-rw- 1 root root 29K Oct 16 16:55 all_task_5_8.1.9-h4_2.info -rw-rw-rw- 1 root root 29K Oct 16 16:55 all_task_6_8.1.9-h4_2.info -rw-rw-rw- 1 root root 29K Oct 16 16:56 all_task_1_8.1.9-h4_7.info -rw-rw-rw- 1 root root 29K Oct 16 16:56 all_task_2_8.1.9-h4_6.info -rw-rw-rw- 1 root root 35K Oct 28 14:44 all_task_1_8.1.9-h4_8.info -rw-rw-rw- 1 root root 35K Oct 28 14:44 all_task_2_8.1.9-h4_7.info -rw-rw-rw- 1 root root 36K Oct 28 14:46 all_task_2_8.1.9-h4_8.info -rw-rw-rw- 1 root root 36K Oct 28 14:49 all_task_2_8.1.9-h4_9.info -rw-rw-rw- 1 root root 36K Oct 28 14:51 all_task_2_8.1.9-h4_10.info
Environment
- Oracle Cloud Infrastructure
- PA-VM Series
- Dataplane for PA-VM Series
- PAN-OS Version 8.1.x
Cause
- The Data Plane Development Kit (DPDK) is a set of data plane libraries and network interface controller drivers for fast packet processing.
- At this moment, the PAN-VM series firewall doesn't support DPDK on Oracle Cloud Infrastructure: Reference Step #13 in the following techdoc Launch the VM-Series Firewall in OCI
Resolution
- From the CLI prompt, enter the following command:
admin@PA-VM> set system setting dpdk-pkt-io off
- Reboot the system:
admin@PA-VM> request restart system
Additional Information
- DPDK is enabled by default in PAN-VM firewall.
- Data Plane Development Kit (DPDK) enhances VM-Series performance by increasing network interface card (NIC) packet processing speed.
- On the VM-Series firewall, DPDK is enabled by default on ESXi. If you disable DPDK, the NIC uses packet mmap instead of DPDK.
- To check the status of DPDK, please reference the following steps:
1. To show the current setting
> show system setting dpdk-pkt-io Device current Packet IO mode: DPDK Device DPDK Packet IO capable: yes Device default Packet IO mode: DPDK
2. To disable the DPDK. Please take note that reboot is required when turning it off.
> set system setting dpdk-pkt-io off Disable DPDK Packet IO mode on Enable DPDK Packet IO mode > set system setting dpdk-pkt-io off
Device is now in non-DPDK IO mode, please reboot device
3. To verify if its being turned off, it will change IO mode to "Packet NMAP"
> show system setting dpdk-pkt-io Device current Packet IO mode: Packet MMAP Device DPDK Packet IO capable: yes Device default Packet IO mode: DPDK