Table=off behavior (not adding any route *at all*)

[Jason A. Donenfeld]

Hi Alex,

On Thu, Apr 25, 2019 at 2:01 PM Alex Davies <alex at davz.net> wrote:
>
> Hi,
>
> We have been trying to use WireGuard on FreeBSD (we are using the WG plugin inside the open source opnsense.org software). These run FreeBSD 11.2-RELEASE-p9-HBSD (OPNsense 19.7.a_288-amd64).
>
> We noticed that by default (i.e. no Table=) wireguard-go wg0 adds default routes (as two /31's) as expected. However, if table=off, we get no route at all - not even to the VPN peer.
>
> The announcement for the Table= option[1] stated:
>
>   In collaboration with Luis Ressel, wg-quick(8) grew an option! We generally
>   do not like to add things to wg-quick or allow feature-creep, but this was
>   basic enough and mostly involves disabling functionality. Specifically,
>   wg-quick now accepts a Table= parameter with these semantics:
>
>     ~ Table=auto (default) selects the current behaviour
>     ~ Table=off disables creation of routes from allowed ips altogether
>     ~ All other values are passed through to "ip route add"'s table option
>
>   This should enable people to do basic policy routing. It also matches the
>   functionality provided by LEDE/OpenWRT's uci config as well as NixOS's
>   networking configuration.
>
> Ignoring the "creation of routes from allowed ips", it does not even add the subnet defined in [Interface]. netstat -r | grep wg returns nothing.
>
> As a concrete example, if I take the trivial config at https://wiki.archlinux.org/index.php/WireGuard:
>
> [Interface]
> Address = 10.200.200.2/24
> PrivateKey = [FOO's PRIVATE KEY]
> DNS = 10.200.200.1
>
> [Peer]
> PublicKey = [SERVER PUBLICKEY]
> PresharedKey = [PRE-SHARED KEY]
> AllowedIPs = 0.0.0.0/0, ::/0
> Endpoint = my.ddns.address.com:51820
>
> I would (naively) expect this:
> Table=auto: inject route for 10.200.200.2/24 *and* 0.0.0.0/0 via wg0
> Table=off: inject route for 10.200.200.2/24 *only* via wg0
>
> What actually happens is:
> Table=auto: as above/expected
> Table=off: no route out wg0
>
> This mean with Table=off, you are in the extremely confusing situation that you cant even ping the other peer.
>
> Testing on Linux (Kernel 4.15.0-1032-aws inside a 18.04 AMI (public AMI - ami-07dc734dc14746eab)) shows that the behavior is different - its as I expect for both Table values. With this wg0.conf:
>
> root at ip-172-31-39-185:~# cat /etc/wireguard/wg0.conf
> [Interface]
> Address = 192.168.2.1/24
> PrivateKey = eEIwdXp8jKV9/2MEwxYBqQLu4TZqBv9YWvG9fbMuaG4=
> Table = off
>
> [Peer]
> PublicKey = pHQfWzLAUM85vDO6+MZAneBYhapOHUkPAuxr0lJdZlY=
> AllowedIPs = 0.0.0.0/0
> Endpoint = 18.130.138.71:51820
>
> I get this route:
>
> root at ip-172-31-39-185:~# ip route show | grep wg0
> 192.168.2.0/24 dev wg0 proto kernel scope link src 192.168.2.1
>
> Note the /24 route (as expected). With Table undefined or set to auto, I get the 0.0.0.0 route (also as expected).
>
> I dont know much about FreeBSD, but I launched a test EC2 instance (FreeBSD 12.0-RELEASE based on public ami-0d244633039d93966 with kernel reported as 12.0-RELEASE-p3) and I think I see the same thing (i.e. no /24 route):
>
> root at freebsd:/etc/wireguard # netstat -rn | grep wg0
> 192.168.2.5        link#3             UH          wg0
> fe80::%wg0/64                     link#3                        U           wg0
> fe80::1427:e888:767c:dce1%wg0     link#3                        UHS         lo0
> root at freebsd:/etc/wireguard # ping 192.168.2.5
>
> Somebody more expert than me can comment on whether this is expected or not. At the very least, hopefully this post is useful for somebody else.  For our specific problem, we have fixed this by putting a static route in for the "Address" subnet across wg0.
>
> -Alex
>
> [1] https://lists.zx2c4.com/pipermail/wireguard/2017-December/002231.html

Sounds like a FreeBSD bug. I've CC'd the maintainer there. We're
adding the IP address with the subnet, via:

ifconfig "$INTERFACE" inet "$1" "${1%%/*}" alias

In your case, this expands to something like,

ifconfig wg0 inet 192.168.2.5/24 192.168.2.5 alias

Bernard - is there a FreeBSD reason why this wouldn't add the
automatic subnet route? Also, rather than using ${1%%/*}, is there a
way for us to just specify the interface directly?

Jason