Issues using multiple interfaces between two servers

[Ivan Labáth]

Hello,

I can't say for sure, but I would guess your issue is the result
of transient network states/outages coupled with wireguard
automatic roaming and wildcard listening.

Wireguard listens on all addresses and performs automatic roaming,
neither of which can be disabled without external help (e.g. firewall).
If a valid packet happens to reach the other address it will
(probably) take over.

If you wish to prevent tunnel flapping and don't care about anything
else, it should be sufficient to set an INPUT firewall rule on
both sides, permitting communication
A1 <-> B1
A2 <-> B2
while dropping cross-communication (mis-paired IPs).

To be clear, the remote enpoint setting is treated as bootstrapping hint.
If you want to use wireguard and set a fixed remote enpoint (ip+port),
you can do so with a 1:1 tunnel, if you e.g. sacrifice a port number
and set a strict firewall. With 1:N tunnels, the only option I can
see is limiting to a set of endpoints, or a code changes in wireguard
sources.

Regards,
Ivan


On Tue, Dec 22, 2020 at 12:57:35PM -0300, wireguard at meta-cti.com.br wrote:
> Hello guys, I'm having problems with my wireguard setup and I don't know
> how to solve it. I have two computers running linux in remote locations.
> One, which I will call computer A, is in a data center where we advertise
> a block of IPs using BGP. The other computer is in a different location
> and has two links connecting to the internet and with different providers.
> I configured on computer A two wireguard tunnels with different keys and
> ports. On computer B I did the same and added two routing tables, one for
> each WAN interface and using the ip rule I created rules with destination
> on two different IPs of computer A so that they leave through different
> links.
> 
> As soon as I start the wireguard interfaces of both computers everything
> works normally and I can ping both addresses from both tunnels. Then I use
> the bird with OSPF and ECMP to take a subnet from the block that is
> advertised on computer A to computer B. Everything works normally.
> 
> When I execute the wg command on computers A and B, I can see both IPs of
> computer B's WAN interfaces in the tunnel's "peer" fields, one from each
> remote WAN.
> 
> After some time working, it can vary from minutes to a few hours, suddenly
> I see that both tunnels started to work on a single WAN interface of
> computers A and B. If at this moment I execute the wg command on computer
> A, I see that now the "peers" have the same address as only one of the WAN
> interfaces of computers A and B, even with the routing rule forcing
> packets to go out through different interfaces. Has anyone experienced a
> similar problem and knows how it can be solved?
> 
> When I run the traceroute command on both computers A and B with the
> destination address in the remote computer's WAN IPs, they actually come
> out through the correct interface.