Root Partition Reaches 100% Due to .nfsxxx Files in CN-Series Firewalls
629
Created On 09/23/25 18:41 PM - Last Modified 09/23/25 22:09 PM
Symptom
- The root partition utilization of a CN-Series firewall shows 100% usage.
- Upon inspection, numerous files with the
.nfsxxxnaming convention are found in the/var/log/pandirectory. - Log files and other data fail to write to the root partition, leading to potential operational issues.
Environment
- Product: CN-Series
- Deployment Environment: OpenShift or Native Kubernetes
- Storage Configuration: NFS shares used for persistent volumes (PVs).
Cause
- The root cause is the use of Network File System (NFS) shares for persistent storage with CN-Series firewalls. This configuration is not qualified and is unsupported by Palo Alto Networks.
- When a file is deleted from an NFS mount while the file is still in use, the NFS client renames it to a temporary
.nfsxxxfile instead of immediately deleting it. This process, known as "silly renaming," is a safety feature to prevent data loss. - In this unsupported setup, the files fail to be properly cleaned up and accumulate over time, consuming all available space in the root partition.
Resolution
- Migrate the CN-Series firewall's persistent volumes (PVs) from the current NFS shares to a local storage solution.
- Make sure the new PVs and persistent volume claims (PVCs) are set up to use a local storage class that's officially supported in your OpenShift or native Kubernetes environment.
- As a temporary workaround, deleting the
.nfsxxxfiles will free up the disk space.
Additional Information
- Using an unsupported NFS solution for Persistent Volumes can cause a range of issues beyond what's expected. For example, you might see problems with file operations, where content updates fail or certain processes crash.
- These issues often arise because of how NFS manages file locking and concurrent access among different users or applications.
- To ensure optimal performance and stability, it's crucial to stick to the officially supported configurations.
- Using storage solutions that aren't officially approved can lead to unpredictable behavior, and any resulting problems will not be covered under a support agreement.