wg-quick with default route fails on nfs root filesystem

[Tom Yan]

Hi Chris,

So I did a test and took a look at the log:
...
wg-quick[2003]: [#] ip -4 route add 0.0.0.0/0 dev aliyun table 51820
wg-quick[2003]: [#] ip -4 rule add not fwmark 51820 table 51820
wg-quick[2003]: [#] ip -4 rule add table main suppress_prefixlength 0
...

What if wg-quick adds the route after adding both rules? AFAIK, a rule
will be a no-op if the table it looks up is empty.

Regards,
Tom

On Wed, 14 Jul 2021 at 18:00, Chris <wireguard at spam-free.eu> wrote:
>
> When wg-quick detects a default route through the tunnel it does this through a
> new routing table with a default route.
> However not to destroy the existing non-default routes these will looked up and
> used first. This results in the follwing policy rule entries:
> (The priority numers may be different from system to system)
>
> 32764:    from all lookup main suppress_prefixlength 0
> 32765:    not from all fwmark 0xca6c lookup 51820
>
> It is very important of course, that the suppress_prefix rule comes first in the
> list, before
> the second rule introduces the new default route (preventig the wireguard
> traffic through it's own tunnel).
>
> The way to archive this is done by the following command sequence:
>
> ip -4 rule add not fwmark 51820 table 51820
> ip -4 rule add table main suppress_prefixlength 0
>
> The sequence of the commands is important as the latter command gets the higher
> priority (lower numer).
>
> BUT:
> In case your root filesystem needs the local network, the second command will
> not be reached as the
> first command (setting the new default route) kills the root filesystem and the
> system stalls!!!!!!
>
> One possible solution:
> Instead of adding the suppress_prefixlength 0 command secondly it must be first.
> The you must find the priority of that rule and the add the default route with
> the same priority.
> A rule with same priority will be added AFTER the other rules.
>
> Example:
> ip -4 rule add table main suppress_prefixlength 0
> PRIO=$(ip rule list from all|grep suppress_prefixlength|sed -e
> '{s/^\(.*\)\:.*/\1/;q}')
> ip -4 rule add not fwmark 51820 table 51820 priority $PRIO
>
> This will lead to the correct sequence:
> 32765:    from all lookup main suppress_prefixlength 0
> 32765:    not from all fwmark 0xca6c lookup 51820
> (Note the same priority number)
>
> There are probably better ways to cirumvent cutting off the root filesystem.
>
> Chris
>