Cisco: How can MSS help to solve issues in VPN communication

Since a week, I’m stretching my brains to solve a communication problem over a VPN connection. The problem was that connections like SSH over VPN were not successfully completed. Imagine site A (Paris – remote end) and site B (Hamburg – local end).

In the back, of this sites, servers and clients. If somebody tried to connect from a client in site A over SSH to a server in site B, the initial authentication protocol was successful, but as soon as a command was typed on the terminal (like ls -la or ps aux) and the server had to return a bunch of results, the SSH console was completely stuck

Immediately I was thinking that this has to do with MTU size (default 1500 bytes on each site) and the DF (Don’t fragment) bit being set. I have tested SSH over Windows and Linux machine and all the time the DF bit was set ON:

I think it is obvious why I did paint over IP addresses. Interesting part is that only SSH was having this DF bit set. I’ve tried also FTP or regular ping, and there, I could not see the DF bit on.

You image what problems I had. MTU being 1500 Bytes and DF bit on, the packet was not fragmented. With the IPsec VPN overhead, the packet could end up to 1604 bytes (together with point-to-point GRE overhead), so most of the packets got dropped.

If I would decrease the MTU size on the client / server interface to let’s say 1300 bytes, then everything was working fine. However, this was not a scalable solution when you have more clients and servers, so to prevent future issues, I keep on looking for another solution. The salvation came from “ip tcp mss-adjust” command.

You probably know what MSS is, but here is a small description. The maximum segment size (MSS) is an option of the TCP protocol that specifies the largest amount of data, specified in bytes, that a computer or communications device can receive in a single, unfragmented piece. It does not count the TCP header or the IP header. For optimum communications, the number of bytes in the data segment and the headers must not add up to more than the number of bytes in the maximum transmission unit (MTU).

This Maximum Segment Size (MSS) announcement (often mistakenly called a negotiation) is sent from the data receiver to the data sender and says “I can accept TCP segments up to size “X bytes”.

Typically a host bases its MSS value on its outgoing interface’s maximum transmission unit (MTU) size because if the MSS value is set too low, the result is an inefficient use of bandwidth; More packets are required to transmit the data. An MSS value that is set too high could result in an IP datagram that is too large to send and that must be fragmented. In my case if the MTU was 1500, the MSS was also 1500 bytes.  This value didn’t help me at all, due to the issues that I’ve mentioned above.

So, instead of changing every device MTU size to a lower value than 1500 bytes, I’ve decided to change the MSS to a reasonable value that would solve my problem. On Cisco devices, you can configure the MSS in a few straightforward steps:

# configure terminal
# interface FastEthernet 0/0
# ip tcp adjust-mss 1300

The 1300 Bytes is just an example in my case. This value can be something between 500 and 1460 bytes.

Next time when you have problems with connections over VPN, try to se the MSS to a lower value and see if it’s working. If it is, then you are saved. If not…maybe your problem is not related to this topic.