Palo Alto Networks Knowledgebase: How Disk Space is Allocated on Log Collectors

How Disk Space is Allocated on Log Collectors

14486
Created On 06/24/19 14:33 PM - Last Updated 06/24/19 14:47 PM
CLI Log Collector Logs Reporting and Logging Cortex Data Lake Panorama
Symptom

Where does the space go?

A log collector is deployed with 4 1TB disk pairs. The GUI reports 3.23 TB of total space that can be allocated via quota. Various CLI commands show different values from the GUI. What is going on here? How much space do you actually have for logs?



Cause

How space is allocated?

There are several factors that affect how much space on a disk is used for logs in Elasticsearch. The following graphic explains where space goes when a disk or disk pair is added to a log collector.
Graphic.png



Resolution

Total Size

In this example, a 1 TB disk is added to a log collector. Right from the beginning a discrepancy can be seen between the size of the disk (1 TB) and the amount of space that the operating system sees (917 GB). This discrepancy is due to a couple of factors:

 

  1. File system overhead: There is some overhead associated with creation of the file system on the disk, but it is fairly minor.
  2. 1000 vs. 1024 Bytes per Kilobyte: Hard drive manufacturers count 1000 bytes per kilobyte while the operating system uses 1024 bytes per kilobyte. This is the biggest source of the discrepancy that we see above. You still actually have the same number of bytes available, hard drive manufacturers just count the kilobytes differently.

 

Logd Formatted Logs

One third (~33%) of the available disk space is allocated to logd formatted logs. The logd format is what was used prior to the introduction of Elasticsearch. Post 8.0, the logd formatted logs are stored to support upgrade and downgrade only. No indexes are generated for these logs.

 

Elasticsearch

Two thirds (~66%) of the available disk space is allocated for use by Elasticsearch. Configured quotas are applied against this allocated space. In the graphic above it can be seen that the default quota percentages are applied against the 66% of the total disk space that is allocated to Elasticsearch.

 

Checking disk usage on the CLI

 

To check overall disk usage:

> show system disk-space

 

This command will show the PAN-OS equivalent to the Unix 'df -h' command. Usage statistics for each mount are included. Logging disks are mounted on /opt/panlogs/:

 

show system disk-space.png

 

Looking at the 'Size' column for the logging disks shows that the operating system sees the disks as 917 GB due to the 1000 vs 1024 bytes per kilobyte discrepancy.

 

To check usage by Elasticsearch

> show system search-engine-quota

 

This command will show the status of Elasticsearch's disk allocation (66% of total disk size):

 

show system search-engine-quota.png

 

To check usage by logd

> show system logdb-quota

 

This command will show the space used for storing logdb formatted logs (33% of total disk size):

 

show system logdb-quota.png



Attachments
Actions
  • Print
  • Copy Link

    https://knowledgebase.paloaltonetworks.com/KCSArticleDetail?id=kA10g000000Cm5aCAC&refURL=http%3A%2F%2Fknowledgebase.paloaltonetworks.com%2FKCSArticleDetail

Attachments
Choose Language