Re: [Babel-users] [RFC] Replace WireGuard AllowedIPs with IP route attribute

[Steffen Vogel]

Hi Daniel,

Interesting ideas! I am wondering if this complexity is really necessary?
How many routes do you have per peer? In my personal setup I have maximum of 1-100 routes per peer which I can handle with the current API quite comfortably.

My biggest concern about the introduction of a route attribute is that this adds complexity for users.
WireGuard's simplicity (and portability) have been important factors for its success.

A route attribute would introduce another source for the crypto-routing peer selection process.
What happens if the two mechanisms select different peers? Which one would have precedence?
Similarly also for incoming packets. WireGuard's current principle is really easy to understand. If the source address in in the peers AllowedIP list, we will accept the packet. If not its discarded. This is a central part of WireGuard's crypto-key routing feature which would become more complex.

Also implementation wise I would have doubts: Should WireGuard itself perform route lookups to determine which packets will be accepted? Or does WireGuard needs to synchronize the kernel routing table with its internal data structures itself?

A second concern I have with the use of route attributes is limited portability. Not all platforms support them.
How do we handle WireGuard userspace implementations?

I've tackled this problem in a userspace daemon. The synchronisation of a kernel routing table with a WireGuard AllowedIPs settings can be done by cunicu's route synchronization feature: https://cunicu.li/docs/features/rtsync

The route synchronization feature keeps the kernel routing table in sync with WireGuard's AllowedIPs setting.
This synchronization is bi-directional:

- Networks which are found in a Peers AllowedIP list will be installed as a kernel route
- Kernel routes with the peers unique IP address as next-hop will be added to the Peers AllowedIPs list.

This rather simple feature allows user to pair cunicu with a software routing daemon like Bird2 while using a single WireGuard interface with multiple peer-to-peer links.

I am assigning each WireGuard interface a link-local address which is derived from the peers public key.
I am using the peers link-local address as the next-hop in my routing daemon to differentiate to which Peer the AllowedIP entry must be added.

I am keeping track of the kernel's routing table and AllowedIPs by regularly polling the kernel.
As the route synchronisation is just one of cunicu's features [1], I have a central "watcher" routine in cunicu which observes any modification the the WireGuard interfaces and dispatches events which the individual features then can hook into. These observations are not limited to the AllowedIPs but basically any state of the WireGuard interface. E.g. last handshake time or per/peer traffic counters.

In my setup a periodic synchronization worked fine. But I agree that it would be nice if we could have a Netlink multicast group for subscribing to changes like we also have for other parts of the Linux network stack like routing tables, or link states. This feature was already discussed on the WireGuard mailing list [7]. But unfortunately the patch was never accepted. Maybe we can revisit this patch?

I would also be a big supported of extending the netlink API for supporting incremental updates the AllowedIP lists. The netlink APIs are already different for each platform. So extending it for one platform wouldn't hurt here.

Unfortunately, I have far too many ideas for cunicu and limited time to realize them all. So I've recently moved the whole cunicu project into its own organization at GitHub/Codeberg [6] in attempt to find more contributors.

Best regards,
Steffen (stv0g)

[1] Others planned features are:
- Endpoint discovery via ICE/STUN/TURN
- Peer discovery
- IP-autoconfiguration by deriving link-local addresses from peers public keys

I have a lot more ideas here like integrating my
- Go babel routing implementation [2]
- or Rosenpass PQC key-exchange [3]
- or performing proper path-MTU discovery using DPLPMTUD [4]
- or using hardware tokens, TPMs, secure enclaves to rotate pre-shared keys backed by a hardware source-of-trust [5]

[2] https://github.com/cunicu/go-babel
[3] https://github.com/cunicu/go-rosenpass
[4] https://github.com/cunicu/go-pmtud
[5] https://github.com/cunicu/go-skes
[6] https://codeberg.org/cunicu
[7] https://lists.zx2c4.com/pipermail/wireguard/2021-January/006318.html

On Saturday, August 19, 2023 16:02 CEST, Daniel Gröber <dxld at darkboxed.org> wrote:

> Hi wireguard, birds, and babelers,
> 
> tl;dr I want to add a new Linux route attribute (think "via $wgpeer") to
> supplement wireguard's internal AllowedIPs logic for both routing and
> source address filtering.
> 
> I've been pondering how to better integrate wireguard into dynamic routing
> daemons, particularly BIRD and babeld. Essentially we want to be able to
> dynamically add/remove AllowedIPs depending on current reachability and/or
> link quality stats.
> 
> Looking at the wg netlink API I see two major efficiency/scalability
> problems: 1) there is no way to be notified of changes in AllowedIPs made
> by other processes meaning we have to do periodic scans and 2) a peer's
> AllowedIPs set can only be replaced wholesale, not modified
> incrementally. This is problematic as "someone" might, in the worst case,
> want to install an entire internet routing table's worth of AllowedIPs and
> the set will likely change frequently. FYI: The IPv4 table has ~1M entries
> at present, yikes.
> 
> Assuming external AllowedIPs changes are infrequent occationally dumping
> them all to keep a consistent view of the state shouldn't be too much of an
> issue as long as the netlink interface is performant enoug, so I'm going to
> concentrate on the add/remove API for now.
> 
> Instead of doing the obvious thing and adding a more efficient incremental
> AllowedIPs netlink interface I figure why not just add a route attribute to
> select a target wg peer on a device. That way we could not only save memory
> (no separate AllowedIPs trie) but also simplify routing daemon
> implementation considerably.
> 
> This would mirror how on ethernet you can have `dev eth0 via $router_ip`.
> I'm still reviewing the net/ code to find the best way to do this, but I'm
> thinking either a new RTA_WGPEER, like: `default dev wg0 via-wgpeer
> $peer_pubkey` or perhaps re-using RTA_VIA and keying off a statically
> configured AllowedIP addresses.
> 
> To start I'd make this an opt-in replacement for our usual AllowedIPs
> logic, making sure to only activate it if any via* RTAs are active on a
> particular device, but if it proves to work well I don't see why we
> couldn't adapt the netlink code to maintain AllowedIPs using this RTA (but
> invisible to userspace) to re-use the same code and get rid of allowedips.c
> altogether. That's assuming this ends up being less code overall or perhaps
> more performant.
> 
> Happy to hear your thoughts,
> --Daniel
> 
> _______________________________________________
> Babel-users mailing list
> Babel-users at alioth-lists.debian.net
> https://alioth-lists.debian.net/cgi-bin/mailman/listinfo/babel-users