WG can now be fragmented -- great!

[zrm]

On 5/24/19 04:48, Roman Mamedov wrote:
> Hello,
> 
> Just wanted to share my excitement about
> https://git.zx2c4.com/WireGuard/diff/?id=57a8ca7f49b5e70aae18b8b5a70cde8f9e4a9346&id2=7cf2dae97635c8c20a8943522bab2b56c6885c8d
> 
> This means WG packets can now be fragmented, and as such we can use arbitrary
> large MTU inside WG. This in turn means we can now use WG to transport full
> 9000 MTU VXLAN frames over the Internet:
> 
> # ifconfig wg10
> wg10      Link encap:UNSPEC  HWaddr 00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00
>            inet6 addr: fd39:aa:6089:5d42:7900:fcd:12a3:6181/64 Scope:Global
>            UP POINTOPOINT RUNNING NOARP  MTU:9070  Metric:1
>            RX packets:12405 errors:0 dropped:0 overruns:0 frame:0
>            TX packets:11130 errors:17 dropped:2 overruns:0 carrier:8
>            collisions:0 txqueuelen:1000
>            RX bytes:81966214 (78.1 MiB)  TX bytes:45563644 (43.4 MiB)
> 
> # ifconfig xwg10
> xwg10     Link encap:Ethernet  HWaddr 02:79:00:0f:cd:12
>            inet addr:10.123.0.250  Bcast:10.123.0.255  Mask:255.255.255.0
>            inet6 addr: fe80::79:ff:fe0f:cd12/64 Scope:Link
>            UP BROADCAST RUNNING MULTICAST  MTU:9000  Metric:1
>            RX packets:12369 errors:0 dropped:0 overruns:0 frame:0
>            TX packets:9577 errors:9 dropped:0 overruns:0 carrier:9
>            collisions:0 txqueuelen:1000
>            RX bytes:80678848 (76.9 MiB)  TX bytes:44408417 (42.3 MiB)
> 
> # ping 10.123.0.1 -s 8972 -M do
> PING 10.123.0.1 (10.123.0.1) 8972(9000) bytes of data.
> 8980 bytes from 10.123.0.1: icmp_seq=1 ttl=64 time=78.7 ms
> 8980 bytes from 10.123.0.1: icmp_seq=2 ttl=64 time=77.2 ms
> 8980 bytes from 10.123.0.1: icmp_seq=3 ttl=64 time=82.0 ms
> 8980 bytes from 10.123.0.1: icmp_seq=4 ttl=64 time=77.5 ms
> ^C
> --- 10.123.0.1 ping statistics ---
> 4 packets transmitted, 4 received, 0% packet loss, time 3003ms
> rtt min/avg/max/mdev = 77.214/78.881/82.054/1.940 ms
> 
> 08:39:47.573368 IP6 rin.romanrm.net > dynamic-2a02-2698-8024-0.tmn.ertelecom.ru: frag (0|1440) 710 > 710: UDP, bad length 9102 > 1432
> 08:39:47.573371 IP6 rin.romanrm.net > dynamic-2a02-2698-8024-0.tmn.ertelecom.ru: frag (1440|1440)
> 08:39:47.573374 IP6 rin.romanrm.net > dynamic-2a02-2698-8024-0.tmn.ertelecom.ru: frag (2880|1440)
> 08:39:47.573376 IP6 rin.romanrm.net > dynamic-2a02-2698-8024-0.tmn.ertelecom.ru: frag (4320|1440)
> 08:39:47.573378 IP6 rin.romanrm.net > dynamic-2a02-2698-8024-0.tmn.ertelecom.ru: frag (5760|1440)
> 08:39:47.573380 IP6 rin.romanrm.net > dynamic-2a02-2698-8024-0.tmn.ertelecom.ru: frag (7200|1440)
> 08:39:47.573383 IP6 rin.romanrm.net > dynamic-2a02-2698-8024-0.tmn.ertelecom.ru: frag (8640|470)
> 08:39:48.575079 IP6 dynamic-2a02-2698-8024-0.tmn.ertelecom.ru > rin.romanrm.net: frag (0|1440) 710 > 710: UDP, bad length 9102 > 1432
> 08:39:48.575189 IP6 dynamic-2a02-2698-8024-0.tmn.ertelecom.ru > rin.romanrm.net: frag (1440|1440)
> 08:39:48.575339 IP6 dynamic-2a02-2698-8024-0.tmn.ertelecom.ru > rin.romanrm.net: frag (2880|1440)
> 08:39:48.575448 IP6 dynamic-2a02-2698-8024-0.tmn.ertelecom.ru > rin.romanrm.net: frag (4320|1440)
> 08:39:48.575565 IP6 dynamic-2a02-2698-8024-0.tmn.ertelecom.ru > rin.romanrm.net: frag (5760|1440)
> 08:39:48.575691 IP6 dynamic-2a02-2698-8024-0.tmn.ertelecom.ru > rin.romanrm.net: frag (7200|1440)
> 08:39:48.575693 IP6 dynamic-2a02-2698-8024-0.tmn.ertelecom.ru > rin.romanrm.net: frag (8640|470)
> 08:39:48.575828 IP6 rin.romanrm.net > dynamic-2a02-2698-8024-0.tmn.ertelecom.ru: frag (0|1440) 710 > 710: UDP, bad length 9102 > 1432
> 08:39:48.575831 IP6 rin.romanrm.net > dynamic-2a02-2698-8024-0.tmn.ertelecom.ru: frag (1440|1440)
> 08:39:48.575833 IP6 rin.romanrm.net > dynamic-2a02-2698-8024-0.tmn.ertelecom.ru: frag (2880|1440)
> 08:39:48.575834 IP6 rin.romanrm.net > dynamic-2a02-2698-8024-0.tmn.ertelecom.ru: frag (4320|1440)
> 08:39:48.575837 IP6 rin.romanrm.net > dynamic-2a02-2698-8024-0.tmn.ertelecom.ru: frag (5760|1440)
> 08:39:48.575838 IP6 rin.romanrm.net > dynamic-2a02-2698-8024-0.tmn.ertelecom.ru: frag (7200|1440)
> 08:39:48.575840 IP6 rin.romanrm.net > dynamic-2a02-2698-8024-0.tmn.ertelecom.ru: frag (8640|470)
> 
> I also briefly tested performance and despite fragmentation having a bad
> reputation for some, I don't see much difference in iperf speeds to
> the same host vs going directly.
> 
> This is now usable to join multiple locations via VXLAN interfaces as members
> of L2 bridges to physical 1G/10G networks without hobbling MTU of the latter.
> 
> Thanks!
> 

I'm not saying this is a bad idea to support, but it may be good to 
document a couple of things about this.

The first is that this makes it apparent to an observer what the MTU on 
your other interfaces are, and which interface a connection routes 
through if they're different. This is only a small information leak, but 
it is one.

I would also suggest that if you're going to do this with the underlying 
transport, check the DF bit on the inner packet and send the ICMP[v6] 
too big message back to the sender as appropriate. Then you can set a 
high MTU on the wg interface but things that support PMTU discovery can 
still use it.

Moreover, you may want to performance test the common degenerate case of 
setting the MTU on the wg interface to not account for transport 
overhead which then fragments every packet into one full packet and one 
tiny packet. That is likely to be somewhat worse, and would occur in any 
case that the MTU on the originating and wg underlying transport 
interfaces are equal and the MTU on the wg interface is set to that or 
higher.