When To Use Adjust MSS
Symptom
TCP transfers fail/timeout after a while.
For example, client PC is trying to access a website and the webpage does not load and timeout happens or loads only partially.
Browser might give "ERR_CONNECTION_TIMED_OUT" or load only parts of the web-page.
The web-page might take a very long time to completely load on the browser.
Environment
Any TCP based transfer.
Website access issues have been discussed in this document.
Cause
Initial Check :
The error or state of the browser has to be first checked to see if the issue is not caused by issues with server.
If http error codes of 500 or 400 series are seen then it can be server or authorization related issue.
If there are no such error codes and if the PC is able to resolve the URL, then the issue could be due to MTU issues on the path.
Topology :
Assumptions :
The MTU setting on eth1/1 interface of R1 router is 1400 Bytes.
10.10.40.1 is the web server which hosts the abcd.com website.
abcd.com on the Client PC resolves to 10.10.40.1
The website does not load on the client PC.
Resolution
How Does Lower MTU Affect TCP Communication :
In this case, since the MTU of eth1/1 interface of Router R1 is set to 1400 Bytes, packets of size more that 1400 Bytes will have to be fragmented before they can be sent out of this interface.
If Don't Fragment (DF) flag is set on the IP header of the packet, then the Router R1 will have to drop the packet and send "Fragmentation needed but DF bit set" (type 3, code 4) ICMP error code to the source.
Most TCP based applications will have the DF bit set on the IP header.
Consider a data packet of size 1500 Bytes from server 10.10.40.1 is being sent to Client 10.10.10.1.
Since the DF flag on this packet will be set, Router R1 will drop the packet when trying to send it out of ethernet 1/1 interface.
This will cause all packets of size more that 1400 Bytes to be dropped and thus the webpage will not load on the client.
If SSL (HTTPS) connection is being used, then the packets with the Server Certificate are usually big packets which can get dropped causing the SSL/TLS handshake to fail.
How to find MTU Issue and Optimal MSS :
It is not always possible to have access to all the hosts in the path to check the MTU settings.
A simple ping test from the Client PC or Server can be used to determine if there is an MTU issue in the Path.
The following parameters have to be set while doing the Ping.
1. Do Not Fragment Bit (-f)
2. Size (-l)
Below output is taken from Windows PC. Respective commands will have to be used on Machines running a different OS.
As discussed in the above sections, since the MTU of eth1/1 on R1 router is set to 1400 Bytes, the ping test on the client will show the below results.
C:\Users\skadmin>ping 10.10.40.1 -f -l 1472 Pinging 10.10.40.1 with 1472 bytes of data: Packet needs to be fragmented but DF set. Packet needs to be fragmented but DF set.The reason to chose 1472 Bytes as the size for the first test is because the complete packet will then be 1500 Bytes.
Complete Packet = 20B(IP Header) + 8B(ICMP Header) + 1472B(Payload) = 1500 Bytes.
We can see that ICMP message "Fragmentation needed but DF bit set" message is coming back.
So we perform multiple such Ping tests to find out the Maximum size of packet that is able to reach the server and back without getting fragment.
C:\Users\admin>ping 10.10.40.1 -f -l 1440 Pinging 10.10.40.1 with 1440 bytes of data: Packet needs to be fragmented but DF set. C:\Users\admin>ping 10.10.40.1 -f -l 1350 Pinging 10.10.40.1 with 1350 bytes of data: Reply from 10.10.40.1: bytes=1350 time=447ms TTL=58 C:\Users\skadmin>ping 10.10.40.1 -f -l 1380 Pinging 10.10.40.1 with 1380 bytes of data: Packet needs to be fragmented but DF set. C:\Users\admin>ping 10.10.40.1 -f -l 1372 Pinging 10.10.40.1 with 1372 bytes of data: Reply from 10.10.40.1: bytes=1372 time=438ms TTL=58 C:\Users\admin>ping 10.10.40.1 -f -l 1373 Pinging 10.10.40.1 with 1373 bytes of data: Packet needs to be fragmented but DF set.Divide and conquer method has been used to find the Max value where the ping succeeds.
Based on the above test outputs it can be seen that the ping works when size is 1372 Bytes and fails when it is 1373 Bytes.
This means only a 1400 Bytes packet will be able to reach the server and back without getting fragmented.
1372 Bytes + ICMP header 8 Bytes + IP header 20 Bytes = 1400 Bytes.
To determine the Optimal MSS that can be used, take the result of the above test which is 1400 Bytes and subtract the potential TCP header and IP header value.
1400 Bytes - (IP header 20 Bytes + TCP header 20 Bytes) = 1360 Bytes.
So the Optimal MSS that can be used is 1360 Bytes.
How to Use this Optimal MSS :
Most Firewalls and routers have the capability of adjusting the MSS value on a TCP connection through them.
They can rewrite the MSS value on the SYN and SYN ACK packets exchanged between the Client and Sever.
This feature can be used to set the MSS value to the calculated optimal MSS value 1360 Bytes so that both client and sever will build their segments only to that size.
Once the TCP and IP headers are then added, the size of the complete packet will be equal to 1400 Bytes, which is the Maximum size of packet that can go through without fragmentation.
Segment 1360 Bytes + TCP header 20 Bytes + IP header 20 Bytes = 1400 Bytes.
Thus the fragmentation issue on the path can be avoided for the TCP connection and packet drops can be avoided.
Please note that even though adjusting the MSS value on the PA firewall solves the issue, the issue is not caused by the Firewall.
The issue is caused by other hosts in the path that have lower MTU setting.
On PA firewall to adjust the MSS value to 1360 Bytes, the Adjustment size has to be configured as 140 Bytes.
1500 - 1360 = 140 Bytes
Configure MSS Adjust Size
Additional Information
Additional Useful Links :
How TCP MSS Adjustment automatically happens for traffic over IPSEC tunnels on PA firewalls
HOW THE PALO ALTO NETWORK FIREWALL HANDLES PACKETS THAT EXCEED THE MTU
HOW TO CONFIGURE MTU AND MSS SETTINGS FROM THE CLI