Why Can't I Browse the Internet when Using a GRE Tunnel?

Introduction

Sometimes when traffic goes through a generic routing encapsulation (GRE) tunnel, you can successfully use Ping and Telnet, but you can't download Internet pages or transfer files using FTP. This Tech Note explains a common reason for this problem, and offers several workarounds.

Packet Fragmentation and ICMP Messages

Let's take the following network diagram as an example:

Network Diagram

In the diagram above, when the Client wants to access a page on the Internet, it establishes a TCP session with the Web Server. During this process, the Client and Web Server announce their maximum segment size (MSS), indicating to each other that they can accept TCP segments up to this size. Upon receiving the MSS option, each device calculates the size of the segment that can be sent. This is called the Send Max Segment Size (SMSS), and it equals the smaller of the two MSSs. For more information about TCP Maximum Segment Size, see RFC 879 .

For the sake of argument, let's say the Web Server in the example above determines that it can send packets up to 1500 bytes in length. It therefore sends a 1500 byte packet to the Client, and, in the IP header, it sets the "don't fragment" (DF) bit. When the packet arrives at R2, the router tries encapsulating it into the tunnel packet. In the case of the GRE tunnel interface, the IP maximum transmission unit (MTU) is 24 bytes less than the IP MTU of the real outgoing interface. For an Ethernet outgoing interface that means the IP MTU on the tunnel interface would be 1500 minus 24, or 1476 bytes.

R2 is trying to send a 1500 byte IP packet into a 1476 byte IP MTU interface. Since this isn't possible, R2 needs to fragment the packet, creating one packet of 1476 bytes (data and IP header) and one packet of 44 bytes (24 bytes of data and a new IP header of 20 bytes). R2 then GRE encapsulates both of these packets to get 1500 and 68 byte packets, respectively. These packets can now be sent out the real outbound interface, which has a 1500 byte IP MTU.

However, remember that the packet received by R2 has the DF bit set. Therefore, R2 can't fragment the packet, and instead, it needs to instruct the Web Server to send smaller packets. It does this by sending an Internet Control Message Protocol (ICMP) type 3 code 4 packet. This ICMP message contains the correct MTU to be used by the Web Server, which should receive this message and adjust the packet size accordingly.

We can view the ICMP messages sent by R2 by enabling the debug ip icmp command:

ICMP: dst (10.10.10.10) frag. needed and DF set unreachable sent to 10.1.3.4

Blocked ICMP Messages

A common problem occurs when ICMP messages are blocked along the path to the Web Server. When this happens, the ICMP packet never reaches the Web Server, thereby preventing data from passing from Client to Server.

Solutions

You can use any of the following four solutions:

Find out where along the path the ICMP message is blocked, and see if you can get it allowed.
Set the MTU on the Client's network interface to 1476 bytes, forcing the SMSS to be smaller, so packets won't have to be fragmented when they reach R2. However, if you change the MTU for the Client, you should also change the MTU for all devices that share the network with this Client. On an Ethernet segment, this could be a large number of devices.
Use a proxy-server (or, even better, a Web cache engine) between R2 and the Gateway router, and let the proxy-server request all the Internet pages.
If the GRE tunnel runs over links that can have an MTU greater than 1500 bytes plus the tunnel header, then another solution is to increase the MTU to 1524 (1500 plus 24 for the GRE overhead) on all interfaces and links between the GRE endpoint routers.
Now the best solution, make the MTU of the GRE tunnel interface as big as possibel, perhabs 2048. And the the Ethernet interface who goes to the Internet will fragment the encapsulated packet of 1524 Bytes. Tis is becose the GRE packet has not the df bit set!
You can add allso the following line directly in the tunnel interface: ip mtu 2048
Then you got the message: %Warning: MTU set 2048 is greater than default 1476, fragments will happen
Then every thing is file (If you do this on bouth ends of the tunnel)
If you got an error message the you have to update the ios version to 12.1 or higher.

Further Solutions

If the above options are not feasible then the following options may be useful:

Use policy routing to clear and set the DF bit in the data IP packet (available in Cisco IOS^® release 12.1(6) and higher).

·         interface ethernet0

·          ...

·          ip policy route-map clear-df

·

·         route-map clear-df permit 10

·          match ip address 101

·          set ip df 0

·

·         access-list 101 permit tcp 10.1.3.0 0.0.0.255 any

This will allow the data IP packet to be fragmented before it is GRE encapsulated. The receiving endhost must then reassemble the data IP packets. This is usually not a problem.

Change the TCP MSS option value on SYN packets that traverse through the router (available in IOS 12.2(4)T and higher). This reduces the MSS option value in the TCP SYN packet so that it's smaller than the value in the ip tcp adjust-mss <value> command, in this case 1476. The endhosts now send TCP/IP packets no larger than this value.

·         interface tunnel0

·          ...

·          ip tcp adjust-mss 1476

A final option is to increase the IP MTU on the tunnel interface to 1500 (available in IOS 12.0 and higher). However, increasing the tunnel IP MTU causes the tunnel packets to be fragmented because the DF bit of the original packet is not copied to the tunnel packet header. In this scenario, the router on the other end of the GRE tunnel must reassemble the GRE tunnel packet before it can remove the GRE header and forward the inner packet. IP packet reassembly is done in process-switch mode and uses memory. Therefore, this option can significantly reduce the packet throughput through the GRE tunnel.

In conclusion, the most common cause of not being able to browse the Internet over a GRE tunnel is due to the above mentioned fragmentation issue. The solution is to allow the ICMP packets or work around the ICMP problem with any of the above solutions.