How to calculate maximum primary shard supported?
8197
Created On 01/19/22 07:24 AM - Last Modified 06/04/25 09:27 AM
Objective
To calculate maximum primary shard supported by the device when esmonitor reports shards percent 100.
> less mp-log esmonitor.log
.....
-0700 ELASTICSEARCH STATUS: active_primary_shards 1088 active_shards 2176 active_shards_percent 100
-0700 ELASTICSEARCH STATUS: active_primary_shards 1088 active_shards 2176 active_shards_percent 100
-0700 ELASTICSEARCH STATUS: active_primary_shards 1088 active_shards 2176 active_shards_percent 100
......
Environment
- Panorama Log Collector
- Supported PAN-OS.
Procedure
- Run the command "> show log-collector-es-cluster health" and notedown the number of data nodes.
- Run the command "> debug logdb show-heap-size collector-group Log-Collector" and note down the maximum heap size.
- The maximum primary shard supported by the log collector can be calculated using the formula
Number of data nodes X 20 X Maximum Heap size
Example:
> show log-collector-es-cluster health
{
"cluster_name" : "__pan_cluster__",
"status" : "green",
"timed_out" : false,
"number_of_nodes" : 2,
"number_of_data_nodes" : 2, <<<<<<
"active_primary_shards" : 1814,
"active_shards" : 3628,
......
> debug logdb show-heap-size collector-group Log-Collector
Response from logger xxxxxxxxxxxx: Minimum Heap size: 30g Maximum Heap size: 30g
With the above data, the heap size is = 2 X 20 X 30 = 1200 ( Number of data nodes X 20 X Maximum heap size)
Starting from PANOS version 11.1, the cluster shard limit defaults to 1000 per data node.
Additional Information
How many shards should I have in my Elasticsearch cluster?