[RFC] Handling multiple endpoints for a single peer

[Jason A. Donenfeld]

Thanks for the proposal. Obviously the easiest solution is a userspace
decoupled one, but this might not necessarily be the most desirable.
However, it's possible the upcoming userspace event notification fd
support (epoll on an fd to learn when handshakes happen) and userspace
peer-message support (send an encrypted out of band non-IP packet
directly to a peer, for things like autoconfig) could play a nice role
in this.

But if it is in the kernel, the decision logic boils down essentially
to: there's a list of endpoints in a given order. Based on differing
metrics of success at differing points in time, the list gets
reordered, and packets are always sent to the top of the list. For
example, the list could be rotated or permutated on every handshake
retry. Or the various other RTT or routing metrics you mentioned
earlier.

However, this doesn't shine any light on the hardest problem: how to
update the list of addresses in a memory-bounded fashion. Right now,
if you receive an encrypted packet, the endpoint of that peer is
updated to the src address of that packet. With multi-endpoint, which
endpoint is updated? Is it simply appended to an ever-growing list of
recent endpoints? How to keep this clean and manageable?