Firewall dropping RST from Client after Server's Challenge-ACK

Firewall dropping RST from Client after Server's Challenge-ACK

125356
Created On 02/25/19 01:54 AM - Last Updated 03/05/19 18:19 PM


Symptom
  • Firewall dropping RST from Client after Server's "Challenge ACK" preventing client from establishing TCP connections to server.


Environment
  • Any client-server architecture where the Server is configured to mitigate "Blind Reset Attack Using the SYN Bit" and sends "Challenge-ACK"
  • As a response to client's SYN, the Server challenges by sending an ACK to confirm the loss of the previous connection and the request to start a new connection.
  • This challenge ACK has acknowledgement number from previous connection and upon seeing the unexpected ACK, client sends a RST; thus tearing down TCP connection on the server also.
  • RFC5961: https://tools.ietf.org/html/rfc5961#section-4 
  • Sample packet capture explaining such a flow:
        User-added image
        > First ACK from the server has acknowledgement number different than SYN's sequence number.
        > Client responding with RST with sequence number derived from challenge ACK.
        > Upon receiving the RST, Server tears down old TCP connection and relies on the SYN retransmission from the client end to re-establish the connection.


Cause
  • When Palo Alto firewall is placed between such client and server, it doesn't understand such a flow by default.
  • Firewall forwards the ACK received from server to the client however the RST from client is dropped by firewall with following reason in debug logs (How to enable debug):
           == 2019-02-20 13:34:24.975 +0530 ==
           Packet received at fastpath stage, tag 97481, type ATOMIC
           Packet info: len 60 port 19 interface 19 vsys 1
           wqe index 228938 packet 0x0x800000031627d8ce, HA: 0
           Packet decoded dump:
              L2:     00:26:6c:23:81:cd->08:30:6b:a1:e0:13, type 0x0800
              IP:     192.168.0.17->X.X.X.X, protocol 6
               version 4, ihl 5, tos 0x00, len 40,
               id 939, frag_off 0x4000, ttl 128, checksum 22508(0x57ec)
              TCP:    sport 51227, dport 443, seq 972108513, ack 972108513,
               reserved 0, offset 5, window 0, checksum 27948,
               flags 0x04 ( RST), urgent data 0, l4 data len 0
          TCP option:
              Flow fastpath, session 97481 (set work 0x800000030ef51900 exclude_video 0 from sp 0x80000002c393f080 exclude_video 0)
              NAT session, run address/port translation
              handle out of window packet, l7 not done, cutthrough  fin
              Drop invalid RST or FIN packet
  •  Following counters seen for such a flow:
                              tcp_drop_packet                        1        0 warn      tcp       pktproc   packets dropped because of failure in tcp reassembly
              tcp_drop_out_of_wnd                    2        0 warn      tcp       resource  out-of-window packets dropped
  •  As such the TCP connection between both client and server enters into a hung state. In other words, the client keeps on trying to establish a new connection while the server continues to respond with a challenge ACK.


Resolution
  • While dropping the out of window RST is actually an intended behavior, it breaks the Challenge-ACK mechanism.
  • Starting from PanOS 8.0.7 and onward, the following configuration is provisioned to make the firewall aware of "Challenge-ACK" mechanism. The client's RST will not be dropped, thereby letting the mechanism work unhindered

              >configure
              #set deviceconfig setting tcp allow-challenge-ack yes
              #commit
              #exit
              >


Additional Information
Reference: https://tools.ietf.org/html/rfc5961#section-4 (Blind Reset Attack Using the SYN Bit)
 


Attachments
Actions
  • Print
  • Copy Link

    https://knowledgebase.paloaltonetworks.com/KCSArticleDetail?id=kA10g000000boBJCAY&refURL=http%3A%2F%2Fknowledgebase.paloaltonetworks.com%2FKCSArticleDetail

Attachments