Classified DoS Protection Drops TCP/UDP/IP Flood Packets Before Reaching Threshold
Symptom
In some cases, it may appear TCP/UDP/IP flood packets get dropped before hitting the configured threshold.
Environment
Classified DoS Protection has been enabled and set to Random Early Drop
Firewalls with multiple dataplanes (DP)
Cause
Assuming there are no dataplanes (DP) in a firewall when the admin sets a flood activate rate, PAN-OS will assign 1/n of this rate to each dataplane.
If the chassis is a PA-7000 series, the rule 1/n depends on the session distribution policy as illustrated:
- fixed (fixed to a single dataplane): n=1
- ingress-slot: N=d (number of DPs per slot)
- random: N=s*d ("s" is the number slots online/available, "d" is the number of DPs per slot )
All other firewalls with multiple dataplanes will evenly distribute the load across all available dataplanes.
This scheme works well when the flood is evenly distributed to all dataplanes. Otherwise, Random Early Drop (RED) may kick-in on one dataplane while the flood rate is well below the 1/n threshold on other DPs, triggering RED prematurely.
How can we check the per dataplane distribution?
admin@PA > show dos-protection rule <rulename> settings
Check for values against the row called "DP"
Example:
Configured rates:
alarm rate: 10000 cps, activate rate: 25000 cps, maximal rate: 60000 cps
Fixed:
admin@PA > set session distribution-policy fixed s1dp0 admin@PA > show dos-protection rule DoS_1 settings DP s1dp0: DP alarm rate: 10000 cps, activate rate: 25000 cps, maximal rate: 60000 cps
Ingress-slot:
admin@PA > set session distribution-policy ingress-slot admin@PA > show dos-protection rule DoS_1 settings DP alarm rate: 5001 cps, activate rate: 12501 cps, maximal rate: 30001 cps <<< gets dived to two.
Random:
admin@PA > set session distribution-policy random admin@PA > show dos-protection rule DoS_1 settings DP alarm rate: 2501 cps, activate rate: 6251 cps, maximal rate: 15001 cps << I have 2 slots, hence 4 DP's
NOTE:
There are other factors that may affect this test. For example, iperf/ testing tools (when testing the DoS profiles in lab) may introduce jitter and burstiness to the generated traffic.
The internal network and distributed systems inside the firewall may also add jitter to the traffic flow.
Jitter/burstiness may trigger RED when the flood rate is close to the threshold. DoS Protection looks at the flood rate not packet counter in a second.
Resolution
Consider the above calculations to prevent dropped packets before reaching the configured thresholds.