Cisco: How can MSS help to solve issues in VPN communication

Since a week, I’m stretching my brains to solve a communication problem over a VPN connection. The problem was that connections like SSH over VPN were not successfully completed. Imagine site A (Paris – remote end) and site B (Hamburg – local end).

In the back, of this sites, servers and clients. If somebody tried to connect from a client in site A over SSH to a server in site B, the initial authentication protocol was successful, but as soon as a command was typed on the terminal (like ls -la or ps aux) and the server had to return a bunch of results, the SSH console was completely stuck

Immediately I was thinking that this has to do with MTU size (default 1500 bytes on each site) and the DF (Don’t fragment) bit being set. I have tested SSH over Windows and Linux machine and all the time the DF bit was set ON:


I think it is obvious why I did paint over IP addresses. Interesting part is that only SSH was having this DF bit set. I’ve tried also FTP or regular ping, and there, I could not see the DF bit on.

You image what problems I had. MTU being 1500 Bytes and DF bit on, the packet was not fragmented. With the IPsec VPN overhead, the packet could end up to 1604 bytes (together with point-to-point GRE overhead), so most of the packets got dropped.

If I would decrease the MTU size on the client / server interface to let’s say 1300 bytes, then everything was working fine. However, this was not a scalable solution when you have more clients and servers, so to prevent future issues, I keep on looking for another solution. The salvation came from “ip tcp mss-adjust” command.

You probably know what MSS is, but here is a small description. The maximum segment size (MSS) is an option of the TCP protocol that specifies the largest amount of data, specified in bytes, that a computer or communications device can receive in a single, unfragmented piece. It does not count the TCP header or the IP header. For optimum communications, the number of bytes in the data segment and the headers must not add up to more than the number of bytes in the maximum transmission unit (MTU).

This Maximum Segment Size (MSS) announcement (often mistakenly called a negotiation) is sent from the data receiver to the data sender and says “I can accept TCP segments up to size “X bytes”.

Typically a host bases its MSS value on its outgoing interface’s maximum transmission unit (MTU) size because if the MSS value is set too low, the result is an inefficient use of bandwidth; More packets are required to transmit the data. An MSS value that is set too high could result in an IP datagram that is too large to send and that must be fragmented. In my case if the MTU was 1500, the MSS was also 1500 bytes.  This value didn’t help me at all, due to the issues that I’ve mentioned above.

So, instead of changing every device MTU size to a lower value than 1500 bytes, I’ve decided to change the MSS to a reasonable value that would solve my problem. On Cisco devices, you can configure the MSS in a few straightforward steps:

# configure terminal
# interface FastEthernet 0/0
# ip tcp adjust-mss 1300

The 1300 Bytes is just an example in my case. This value can be something between 500 and 1460 bytes.

Next time when you have problems with connections over VPN, try to se the MSS to a lower value and see if it’s working. If it is, then you are saved. If not…maybe your problem is not related to this topic.

Published by

Calin

Calin is a network engineer, with more than 20 years of experience in designing, installing, troubleshooting, and maintaining large enterprise WAN and LAN networks.

7 thoughts on “Cisco: How can MSS help to solve issues in VPN communication”

  1. Good article. However, I was wondering what will be the effect of changing MTU (ip MTU x) value on the router’s interface instead of changing the MSS(ip tcp-adjust x). The application can do path MTU discovery and then calculate the MSS value accordingly and send it across to its peer during handshake

    1. Sagar, you’re right, but what do you do if there is a firewall in the way which drops ICMP packets? PMTUD will not work if ICMP packets do not arrive back to host. One of the solutions in this case would be to ajdust MSS.

      I did read lately a very good article by Darren @ mellowd.co.uk, exactly about this topic. I can recommend you to go through it: http://mellowd.co.uk/ccie/?p=5531 I will give you a nice view of PMTUD and MSS adjustments.

      HTH,
      Calin

  2. Ya. If ICMPs are blocked anywhere then PMTU won’t work. I’ll go through the link.
    Thanks again for replying.

  3. Great article:)

    The problem of using p tcp adjust-mss 1300, router will set MSS in every TCP SYNC message (both ingress/egress) for TCP applications if it sees advertised MSS in SYN is > 1300, which puts extra burden on CPU.

    Another we can achieve the same result ( assuming we have an control either on Paris/Hamburg router)
    by modifying MTU on local interface on the router connected to HOSTA/HOST B.

    This is due to the fact, TCP drives its MSS from its directly connected interface ‘s IP MTU and this is what is get sent during TCP ‘s hand shake. We simply need to factor in GRE over head( if you are using GRE based IPSEC), IP SEC overhead( variable depending upon encryption/authentication method negotiated) and 40 bytes ( 20 bytes for IP header, 20 bytes for TCP, assuming no option) .

    Doing this will allow TCP based application to use suitable MSS which will not result in fragmentation, without burdening router to inspect every TCP SYN that passes through its interface where ip tcp adjust-mss is configured.

    Thanks

Any opinion on this post? Please let me know:

This site uses Akismet to reduce spam. Learn how your comment data is processed.