multiple endpoints for a single peer -- implementation details

[Toke Høiland-Jørgensen]

Motiejus Jakštys <motiejus.jakstys at gmail.com> writes:

> Hi all,
>
> I thought I'd implement a prototype to use multiple endpoints for a
> single peer, but after some analysis on "## Select new endpoint during
> each handshake"[1], I'd like to share the concerns with future readers
> who might try the same endeavor. TLDR: I think the kernel is not in
> the best position to do this, "decision making in user space" may be
> more appropriate.
>
> To make it happen, handshake process would change. New suggested flow:
> - Initiator sends a handshake packet to all endpoints quasi-simultaneously.
>   - Each handshake is a new message with a different ephemeral key et al.
> - Responder receives the first one and responds.
> - Responder receives more handshakes within 1/INITIATIONS_PER_SECOND
> and discards them.
> - Responder may receive more after 1/INITIATIONS_PER_SECOND and responds.
>
> Responder needs to maintain more than one handshake state for
> MAX_TIMER_HANDSHAKES, specifically, the whole `noise_handshake`
> struct. Following a later suggestion in the thread, this can have an
> upper bound of MAX_ENDPOINTS_PER_PEER (TBD constant).

Before you go and re-invent the happy eyeballs algorithm, may I suggest
you read the RFC? :)

https://tools.ietf.org/html/rfc8305

Specifically, the considerations for how to do multiple connection
attempts safely without overloading the network is relevant for this. As
is the bit about sorting DNS responses.

[...]

> 2. more a concern: neither kernel, nor wireguard-go implementations
> are willing to accept more than one endpoint, and it would be messy to
> extend:
>
> include/uapi/linux/wireguard.h
>   WGPEER_A_ENDPOINT: struct sockaddr_in or struct sockaddr_in6
>
> device/uapi.go:
>   case "endpoint":
>   ...
>     endpoint, err := CreateEndpoint(value)
>
> Endpoint is fixed to be a single UDP address, and both kernel and
> wireguard-go refuse unknown keys. To have tooling
> backwards-compatibility (i.e. use newer wireguard-tools with older
> kernel implementations), wireguard-tools would need to know the
> supported "features" of the underlying implementation. And there is no
> version negotiation between the user/kernel space. Which makes it
> tricky to add features like this.

Eh? The kernel API is netlink - you could just add multiple
WGPEER_A_ENDPOINT attributes. Or add a new one
(WGPEER_A_ENDPOINTS_MULTI?). Same thing for wireguard-go (I assume).

> I am suggesting that "## Decision-making in userspace" would work
> better here. Userspace would regularly* issue handshake initiations
> and measure how long it takes for each endpoint, and hand over the
> *single* endpoint to the kernel to connect.

Why not just let userspace add more endpoints to an already established
peer, and have the kernel figure out the selection between them?

-Toke