Why is the src_valid_mark needed and not allowed (in kubernetes)?

[Nico Schottelius]

Hello again,

while debugging wireguard in Kubernetes, we noticed that adding a
default route for IPv4 is not possible/allowed, wg-quick fails with:

        sysctl: error setting key 'net.ipv4.conf.all.src_valid_mark': Read-only file system

Which can be traced back to:

        [[ $proto == -4 ]] && cmd sysctl -q net.ipv4.conf.all.src_valid_mark=1

inside the

       add_default() {

function of wg-quick. If one tries to escape this using a pod
specification to set the sysctl using

    spec:
       securityContext:
         sysctls:
           - name: net.ipv4.conf.all.src_valid_mark
             value: "1"

Kubernetes by default disallows it and returns a "SysctlForbidden"
error.

Now, one can easily "fool" wg-quick using

    AllowedIPs = 0.0.0.0/1, 128.0.0.0/1

instead of

    AllowedIPs = 0.0.0.0/0

which makes the container work w/o the sysctl, as the add_default()
function is not called. Running it like this in kubernetes results in:

+ wg show
interface: ipv4
  public key: HV9IgFLPSHJgG4jbDzaJg97Uch3eebrV5we4wUkAcUg=
  private key: (hidden)
  listening port: 51828

peer: 6BRnQ+dmeFzVCH9RbM1pbJ7u3y3qrl+zUzzYCmC88kE=
  endpoint: [2a0a:e5c0:2:2:0:84ff:fe41:f265]:51820
  allowed ips: 0.0.0.0/1, 128.0.0.0/1

And allows outside IPv4 communication within an IPv6 only kubernetes
cluster, which was the objective in the first place.

My questions:

   - Why is net.ipv4.conf.all.src_valid_mark needed?
   - And why does working around using 0.0.0.0/1, 128.0.0.0/1 works /
     what is the drawback of this?

Best regards and a good evening from container land,

Nico

--
Sustainable and modern Infrastructures by ungleich.ch