Src addr code review (Was: Source IP incorrect on multi homed systems)

[Nico Schottelius]

Hey Daniel,

thanks a lot for diving in ...

Daniel Gröber <dxld at darkboxed.org> writes:
> Let's look at the code (heavily culled):
>
> 	struct flowi4 fl = {
> 		.saddr = endpoint->src4.s_addr,
> 	};
> 	if (cache)
> 		rt = dst_cache_get_ip4(cache, &fl.saddr);

What I am wondering is, how did it get into the cache in the first place?

> [...]
>
> @Nico could it perhaps simply be that you're hitting one of these zero'ing
> cases and that's why it's using regular kernel src addr selection instead
> of the cached endpoint src4 address?

That could absolutely be the case. What is funky is that I see the
problem on two very different systems, but maybe it's a good time to
elaborate on this:

- System A:
  - Wireguard module loaded on the host
  - Wireguard wg-quick used within a kubernetes pods that has
    permissions for managing wireguard
  - The same pod also runs bird for BGP peering

- System B:
  - Wireguard running as wireguard-go on OpnSense / FreeBSD
  - BGP running with frr

Both systems exhibit the behaviour, but maybe it's better to focus on
System A first, as this seems to be more the "upstream" source.

Best regards,

Nico

--
Sustainable and modern Infrastructures by ungleich.ch