Option to enable policy routing

[Kyle Rose]

I really like the straightforward configurability of Wireguard
out-of-the-box. It was astonishingly easy to configure a mesh to
replace my previous hub-and-spoke OpenVPN setup. Thank you for making
this easy.

That said, I'd like the ability to use Linux's policy routing engine
to allow for more complex packet flows across the VPN that are
currently incompatible with Wireguard's internal packet handling. For
example, let's say I have 4 nodes and want to be able to use each of
the nodes as the default gateway for different types of flows.
Modifying the sender side to respect a route's gateway is
straightforward:

--- a/drivers/net/wireguard/allowe
dips.c
+++ b/drivers/net/wireguard/allowe
dips.c
@@ -6,6 +6,8 @@
 #include "allowedips.h"
 #include "peer.h"

+#include <net/route.h>
+
 enum { MAX_ALLOWEDIPS_BITS = 128 };

 static struct kmem_cache *node_cache;
@@ -356,10 +358,18 @@ int wg_allowedips_read_node(struct
allowedips_node *node, u8 ip[16], u8 *cidr)
 struct wg_peer *wg_allowedips_lookup_dst(struct allowedips *table,
                                         struct sk_buff *skb)
 {
-       if (skb->protocol == htons(ETH_P_IP))
-               return lookup(table->root4, 32, &ip_hdr(skb)->daddr);
-       else if (skb->protocol == htons(ETH_P_IPV6))
-               return lookup(table->root6, 128, &ipv6_hdr(skb)->daddr);
+       struct rtable *rt = skb_rtable(skb);
+       if (rt->rt_uses_gateway) {
+               if (rt->rt_gw_family == AF_INET)
+                       return lookup(table->root4, 32, &rt->rt_gw4);
+               else if (rt->rt_gw_family == AF_INET6)
+                       return lookup(table->root6, 128, &rt->rt_gw6);
+       } else {
+               if (skb->protocol == htons(ETH_P_IP))
+                       return lookup(table->root4, 32, &ip_hdr(skb)->daddr);
+               else if (skb->protocol == htons(ETH_P_IPV6))
+                       return lookup(table->root6, 128, &ipv6_hdr(skb)->daddr);
+       }
        return NULL;
 }

The problem is that reply packets will be dropped via the source
address check for all but the peer with the default route listed in
AllowedIPs. The way the trie code works means a highly-invasive change
would be needed to allow for multiple peers to be associated with a
given prefix: I suspect any further complication (not to mention
possible additional data structure bloat) is undesirable, and anyway I
am looking to bypass most of the complexity created by Wireguard's
parallel packet routing infrastructure and instead leverage the far
more flexible Linux policy routing engine.

At this point, what I'd like to do is be able to skip the source
address check by configuration, instead relying on rp_filter and
firewall rules to reject bogus or unwanted packets. With such a config
knob, AllowedIPs would be used only for selecting the destination peer
based on the packet daddr or the route's gateway. For a mesh like I
described above, I would configure only a gateway IP for each peer in
AllowedIPs and use the policy routing engine for all other packet
routing behavior.

I appreciate that Wireguard works the way it does most likely because
routing is configured differently across the wide variety of devices
it supports (and in some cases may be unavailable to users), so I
don't think the AllowedIPs source address check should be removed by
default; but it would be nice if I could turn it off and rely on other
mechanisms in the kernel that would allow for more flexibility.

Thoughts?

Thanks,
Kyle