Defending from DoS and volumetric DDoS attacks
Symptom
Network Flood attacks can overwhelm the CPU or Memory components, driving the firewall or servers behind it to a halt.
Environment
PAN-OS >= 7.1
Cause
Network Flood
Resolution
The first step to understand how to defend against a DoS attack is to identify what characteristics it has.
By number of attack sources:
- DDoS Attacks: The attack is multi-sourced (distributed).
- DoS Attacks: The attack is single-sourced.
By how traffic is processed by the firewall:
- Slow path: The attack packets result in the creation of new sessions. This means that packets follow the Slow Path, where each packet needs to run policy lookup and session installation.
- Fast path: The attack packets only create one session and the successive packets continue to match that single session. This means that packet follow the Fast path, where policy match evaluation is not required.
By what is under attack:
- Host bound: The attack's target is an IP address owned by the firewall
- Server behind the firewall: The attack's target is an IP address owned by a device behind the firewall.
What Protections should be used
Volumetric DDoS Attacks
The Palo Alto Networks firewall is not positioned to defend against volumetric DDoS attacks, however, Zone Protection can help safeguard the firewall resources.DoS Policies track connection-per-second rate by source-ip, and in distributed attacks, the sources are many, where each source-ip may not generate enough volume to trigger connection-per-second based rules.
With Zone Protection, the whole connection-per-second rate incoming to interfaces in the Zone can be aggregated regardless of the source-ip's sending the traffic, and the internet Zone can be sacrificed to continue to allow traffic between other firewall zones. Zone Protection will shut down the protocol being leveraged for the attack, meaning that the DDoS network flood by the attacker will be successful in bringing down the internet connectivity.
Given that Zone Protection will bring the protocol used by the packets down, this means that if the attack is ICMP based, then Zone Protection could help alleviate ICMP DDoS attacks, while continuing to allow TCP, UDP and Other traffic from the internet.
| The reason why Zone Protection should not be used to defend servers behind the firewall is that it would simply lower the success threshold of the attack - it will cause denial of service at a lower connection-per-second rate than the application server may be able to whit-stand, so it would just help the attacker reach their objective with lesser effort. |
| To properly defend against Volumetric DDoS attacks, a specialized DDoS appliance should be used. As an alternative, the majority of ISP's will offer optional DDoS protection. Check with your ISP if that is a service they offer. |
During an active attack, the connection-per-second rate can be estimated by querying a single second of traffic logs and counting the number of entries. This technique can also be used to determine if the attack is multi or single sourced.
Slow Path DoS Attacks against the firewall
To defend the firewall resources from a Slow Path DoS Attack, use Zone Protection - Flood Protection.
The Palo Alto Networks firewall can keep track of connection-per-second rates to carry out discards through Random Early Drop (RED) or SYN Cookies (if the attack is a SYN Flood).
SYN Cookies is a technique that will help evaluate if the received SYN packet is legitimate, or part of a network flood. The firewall will discard the SYN packet, codify it into the initial sequence number in a crafted SYN-ACK, and only reconstruct the three-way handshake if a valid ACK is received from the source.
Zone Protection - Network Flood tracks connection-per-second rate incoming to a Zone. It aggregates all connection-per-second rate (for each Protocol) coming in on all interfaces tied to the protected Zone. Zone Protection does not apply to packets that do match an active session. The purpose of this protection is to defend the Firewall resources. The idea is to 'sacrifice' a Zone so that other Zones in the firewall can continue to exchange traffic, though if the DDoS attack is coming from the internet, the internet service can be taken offline for the duration of the attack (if you are trying to defend from DDoS so that your internet service is not impacted, Zone Protection is typically self-defeating, since it will simply lower the target threshold for a successful attack). It is important to note that one of the potential advantages of Zone Protection for DDoS mitigation, is that the traffic shutdown happens per-protocol, so if the DDoS is ICMP based, the Zone is shutdown for all ICMP traffic, while continuing to allow all TCP, UDP, ICMPv6 and Other-IP traffic. (In other words, Zone Protection may be helpful in DDoS mitigation for ICMP, ICMPv6 and Other-IP traffic, since these are in most cases, not essential protocols for internet service).
Whenever a Zone Protection Network Flood protection is triggered, the source of the attack is not written to the logs (there isn't a single source). Attribution in DoS attacks is generally not useful, as attackers will typically spoof the source address.
Zone Protection Threat Log entries will indicate "From Zone" and "To Zone" and will both be the same Zone (indicates ingress zone of the flood). The "rule" name will be empty.
| Note: Never use the default threshold values for Alert, Activate and Maximum as this may lead to a self-inflicted outage (settings set too low would result in discarding legitimate traffic). Finding appropriate values is not a trivial task nor a set-and-forget action, and will require trial and error by setting Activate and Maximum values to the highest setting, and leveraging the Alert rate to evaluate the maximum legitimate traffic peaks. There may be special or seasonal network events like holiday sales, network backups, or things of the like that would push legitimate traffic higher, so these values should account for future network traffic growth and legitimate spikes. |
Slow Path DoS Attacks against resources behind the firewall
To defend the resources behind the firewall from a Slow Path DoS Attack, use DoS Policies - Flood Protection.
DoS Policy: Aggregate
Track connection-per-second rate matching a DoS Policy. It aggregates all connection-per-second rates matching the DoS Policy. The purpose of this protection is to offer a more global defense for services hosted *behind* the firewall.
Whenever a DoS Policy: Aggregate protection is triggered, the source of the attack is not written to the logs (there isn't a single source).
Aggregate DoS Policies Threat Log entries do not indicate "From Zone" and "To Zone" and will instead indicate the DoS Policy "rule" name.
DoS Policy: Classified - track by source
Whenever a DoS Policy: Classified protection is triggered, the source of the attack is written to the logs. Keep in mind attribution is not possible since source is typically spoofed, however you can leverage the source IP to either PBF the traffic to Discard, or move it to a more aggressive DoS Policy.
DoS Policy: Classified - track by source-and-destination.
Whenever a DoS Policy: Classified protection is triggered, the source of the attack is written to the logs. Keep in mind attribution is not possible since source is typically spoofed, however you can leverage the source IP to either PBF the traffic to Discard, of move it to a more aggressive DoS Policy.
During an active DoS attack, you can configure a DoS Classified policy to detect offending source IP's, and manually move attackers to a more restrictive policy. In a DDoS attack, this is typically not effective because the source of the attack can come from hundreds of sources, all sending small connection-per-second rates - not crossing the defined thresholds.
Fast Path DoS Attacks
To defend the firewall resources from a Fast Path DoS Attack, use Packet Buffer Protection.In fast path attacks the connection-per-second rate is close to 0, because successive packets match an active connection, and it doesn't deploy new sessions., therefore connection-per-second threshold based protections do not apply.
If it is a single sourced fast path attack you will instead see a single session with a sizable number of Bytes. Just keep in mind that traffic log entries are written when the session ends, so if the attack is active it may not show up in the traffic logs until the session ends. You can clear all sessions (with the customer’s permission) to force it to end and write the log entries. Some long-lived sessions may also give you the false impression that there is a lot of Bytes transited through a cleared-out session and it may be inferred as being related to the attack, so pay attention to the start-date timestamp in the session (maybe add start date as an additional query filter).
If there is packet buffer exhaustion you can implement PBP to defend the firewall if the attack is leveraging one or very few allowed sessions for attack
Some traffic types may exhaust packet descriptors before packet buffers, and in that case, if the firewall runs PAN-OS >= 10.0 you can shift the mode of PBP to leverage packet latency instead of packet buffer utilization, which can result in a more effective mitigation.
Additional Information
Zone Protection and DoS Protection
Baseline CPS Measurements for Setting Flood Thresholds