From Jason at zx2c4.com  Fri Dec  3 22:20:11 2021
From: Jason at zx2c4.com (Jason A. Donenfeld)
Date: Fri, 3 Dec 2021 23:20:11 +0100
Subject: [PATCH 0/2] wireguard-linux-compat: grsecurity compat patches
In-Reply-To: <c772d6c2-dac7-8c11-7c41-aaa6435a0bd6@grsecurity.net>
References: <20210706132714.8220-1-minipli@grsecurity.net>
 <CAHmME9o38fA4_nDVyS9FvO1EL-Z4fS+a2eCo6Xq7VzMSrPYNKQ@mail.gmail.com>
 <c772d6c2-dac7-8c11-7c41-aaa6435a0bd6@grsecurity.net>
Message-ID: <CAHmME9oyopU8euSuVy7su91fia6xG5V+9i+QodCBdTGdR_rD5Q@mail.gmail.com>

Hey Mathias,

This resulted in kind of an interesting regression with old compilers
on old kernel versions when I backported this to
wireguard-linux-compat:
https://git.zx2c4.com/wireguard-linux-compat/commit/?id=8118c247a75ae95169f0a9a539dfc661ffda8bc5

The 25519 tests fail for 4.8.17, 4.7.10, 4.6.7, 4.5.7 with gcc 6:

https://build.wireguard.com/wireguard-linux-compat/e8db181d62467da6c476cf4ac21e13dd477612c8/4.8.17-x86_64.log
https://build.wireguard.com/wireguard-linux-compat/e8db181d62467da6c476cf4ac21e13dd477612c8/4.7.10-x86_64.log
https://build.wireguard.com/wireguard-linux-compat/e8db181d62467da6c476cf4ac21e13dd477612c8/4.6.7-x86_64.log
https://build.wireguard.com/wireguard-linux-compat/e8db181d62467da6c476cf4ac21e13dd477612c8/4.5.7-x86_64.log

But then they crash for 4.0.9, 3.19.8, 3.17.8 with gcc 5:

https://build.wireguard.com/wireguard-linux-compat/e8db181d62467da6c476cf4ac21e13dd477612c8/4.0.9-x86_64.log
https://build.wireguard.com/wireguard-linux-compat/e8db181d62467da6c476cf4ac21e13dd477612c8/3.19.8-x86_64.log
https://build.wireguard.com/wireguard-linux-compat/e8db181d62467da6c476cf4ac21e13dd477612c8/3.17.8-x86_64.log

And also crash with 3.16.85, 3.15.10, 3.14.79, 3.12.74, 3.11.10 with gcc 4:

https://build.wireguard.com/wireguard-linux-compat/e8db181d62467da6c476cf4ac21e13dd477612c8/3.16.85-x86_64.log
https://build.wireguard.com/wireguard-linux-compat/e8db181d62467da6c476cf4ac21e13dd477612c8/3.15.10-x86_64.log
https://build.wireguard.com/wireguard-linux-compat/e8db181d62467da6c476cf4ac21e13dd477612c8/3.14.79-x86_64.log
https://build.wireguard.com/wireguard-linux-compat/e8db181d62467da6c476cf4ac21e13dd477612c8/3.13.11-x86_64.log
https://build.wireguard.com/wireguard-linux-compat/e8db181d62467da6c476cf4ac21e13dd477612c8/3.12.74-x86_64.log
https://build.wireguard.com/wireguard-linux-compat/e8db181d62467da6c476cf4ac21e13dd477612c8/3.11.10-x86_64.log

Any intuition about what might have happened?

Jason

From Jason at zx2c4.com  Fri Dec  3 22:25:13 2021
From: Jason at zx2c4.com (Jason A. Donenfeld)
Date: Fri, 3 Dec 2021 23:25:13 +0100
Subject: [PATCH 0/2] wireguard-linux-compat: grsecurity compat patches
In-Reply-To: <CAHmME9oyopU8euSuVy7su91fia6xG5V+9i+QodCBdTGdR_rD5Q@mail.gmail.com>
References: <20210706132714.8220-1-minipli@grsecurity.net>
 <CAHmME9o38fA4_nDVyS9FvO1EL-Z4fS+a2eCo6Xq7VzMSrPYNKQ@mail.gmail.com>
 <c772d6c2-dac7-8c11-7c41-aaa6435a0bd6@grsecurity.net>
 <CAHmME9oyopU8euSuVy7su91fia6xG5V+9i+QodCBdTGdR_rD5Q@mail.gmail.com>
Message-ID: <CAHmME9o=xuWJ610OXZi7_gE0EWxWdMxjcp7F3b8R=8h89kLUHg@mail.gmail.com>

On Fri, Dec 3, 2021 at 11:20 PM Jason A. Donenfeld <Jason at zx2c4.com> wrote:
> on old kernel versions when I backported this to

Er, nix the "when I backported this" part: you sent it originally for
compat and never for mainline.

From minipli at grsecurity.net  Mon Dec  6 14:04:34 2021
From: minipli at grsecurity.net (Mathias Krause)
Date: Mon, 6 Dec 2021 15:04:34 +0100
Subject: [PATCH 0/2] wireguard-linux-compat: grsecurity compat patches
In-Reply-To: <CAHmME9oyopU8euSuVy7su91fia6xG5V+9i+QodCBdTGdR_rD5Q@mail.gmail.com>
References: <20210706132714.8220-1-minipli@grsecurity.net>
 <CAHmME9o38fA4_nDVyS9FvO1EL-Z4fS+a2eCo6Xq7VzMSrPYNKQ@mail.gmail.com>
 <c772d6c2-dac7-8c11-7c41-aaa6435a0bd6@grsecurity.net>
 <CAHmME9oyopU8euSuVy7su91fia6xG5V+9i+QodCBdTGdR_rD5Q@mail.gmail.com>
Message-ID: <3a2d41dc-effb-158c-4a52-d7eb282ecb7e@grsecurity.net>

Hi Jason,

Am 03.12.21 um 23:20 schrieb Jason A. Donenfeld:
> This resulted in kind of an interesting regression with old compilers
> on old kernel versions when I backported this to
> wireguard-linux-compat:
> https://git.zx2c4.com/wireguard-linux-compat/commit/?id=8118c247a75ae95169f0a9a539dfc661ffda8bc5
> 
> The 25519 tests fail for 4.8.17, 4.7.10, 4.6.7, 4.5.7 with gcc 6:
> 
> https://build.wireguard.com/wireguard-linux-compat/e8db181d62467da6c476cf4ac21e13dd477612c8/4.8.17-x86_64.log
> https://build.wireguard.com/wireguard-linux-compat/e8db181d62467da6c476cf4ac21e13dd477612c8/4.7.10-x86_64.log
> https://build.wireguard.com/wireguard-linux-compat/e8db181d62467da6c476cf4ac21e13dd477612c8/4.6.7-x86_64.log
> https://build.wireguard.com/wireguard-linux-compat/e8db181d62467da6c476cf4ac21e13dd477612c8/4.5.7-x86_64.log
> 
> But then they crash for 4.0.9, 3.19.8, 3.17.8 with gcc 5:
> 
> https://build.wireguard.com/wireguard-linux-compat/e8db181d62467da6c476cf4ac21e13dd477612c8/4.0.9-x86_64.log
> https://build.wireguard.com/wireguard-linux-compat/e8db181d62467da6c476cf4ac21e13dd477612c8/3.19.8-x86_64.log
> https://build.wireguard.com/wireguard-linux-compat/e8db181d62467da6c476cf4ac21e13dd477612c8/3.17.8-x86_64.log
> 
> And also crash with 3.16.85, 3.15.10, 3.14.79, 3.12.74, 3.11.10 with gcc 4:
> 
> https://build.wireguard.com/wireguard-linux-compat/e8db181d62467da6c476cf4ac21e13dd477612c8/3.16.85-x86_64.log
> https://build.wireguard.com/wireguard-linux-compat/e8db181d62467da6c476cf4ac21e13dd477612c8/3.15.10-x86_64.log
> https://build.wireguard.com/wireguard-linux-compat/e8db181d62467da6c476cf4ac21e13dd477612c8/3.14.79-x86_64.log
> https://build.wireguard.com/wireguard-linux-compat/e8db181d62467da6c476cf4ac21e13dd477612c8/3.13.11-x86_64.log
> https://build.wireguard.com/wireguard-linux-compat/e8db181d62467da6c476cf4ac21e13dd477612c8/3.12.74-x86_64.log
> https://build.wireguard.com/wireguard-linux-compat/e8db181d62467da6c476cf4ac21e13dd477612c8/3.11.10-x86_64.log
> 
> Any intuition about what might have happened?

Sorry to hear that. I didn't ran into such issues when doing the
backport and, in fact, trying to reproduce the selftest errors / crashes
failed so far on v4.8.17 with gcc 6.3 and 4.6.3:

[    0.137871] wireguard: chacha20 self-tests: pass
[    0.141106] wireguard: poly1305 self-tests: pass
[    0.141604] wireguard: chacha20poly1305 self-tests: pass
[    0.142309] wireguard: blake2s self-tests: pass
[    0.157012] wireguard: curve25519 self-tests: pass
[    0.157430] wireguard: allowedips self-tests: pass
[    0.158354] wireguard: nonce counter self-tests: pass
[    0.388426] wireguard: ratelimiter self-tests: pass
[    0.389045] wireguard: WireGuard 1.0.20210606 loaded. See
www.wireguard.com for information.
[    0.389874] wireguard: Copyright (C) 2015-2019 Jason A. Donenfeld
<Jason at zx2c4.com>. All Rights Reserved.

I'll try older kernels and see if they trigger. In case not, can you
send me the object files of a failing kernel?

Thanks,
Mathias

From Jason at zx2c4.com  Mon Dec  6 14:48:21 2021
From: Jason at zx2c4.com (Jason A. Donenfeld)
Date: Mon, 6 Dec 2021 15:48:21 +0100
Subject: [PATCH 0/2] wireguard-linux-compat: grsecurity compat patches
In-Reply-To: <3a2d41dc-effb-158c-4a52-d7eb282ecb7e@grsecurity.net>
References: <20210706132714.8220-1-minipli@grsecurity.net>
 <CAHmME9o38fA4_nDVyS9FvO1EL-Z4fS+a2eCo6Xq7VzMSrPYNKQ@mail.gmail.com>
 <c772d6c2-dac7-8c11-7c41-aaa6435a0bd6@grsecurity.net>
 <CAHmME9oyopU8euSuVy7su91fia6xG5V+9i+QodCBdTGdR_rD5Q@mail.gmail.com>
 <3a2d41dc-effb-158c-4a52-d7eb282ecb7e@grsecurity.net>
Message-ID: <CAHmME9r4gspVnZGhgHhB1nR6FvOaryLoZvAJwpRU=-=rrbdoqQ@mail.gmail.com>

Hey Mathias,

I couldn't repro with a new kernel either. Only these old ones.

Here are some object files for the logs in the earlier message:
https://data.zx2c4.com/curve25519-miscompile-a87440f8-7d1e-4179-be83-c82b4636a448.tar.zst

Jason

From minipli at grsecurity.net  Mon Dec  6 16:24:35 2021
From: minipli at grsecurity.net (Mathias Krause)
Date: Mon, 6 Dec 2021 17:24:35 +0100
Subject: [PATCH 0/2] wireguard-linux-compat: grsecurity compat patches
In-Reply-To: <CAHmME9r4gspVnZGhgHhB1nR6FvOaryLoZvAJwpRU=-=rrbdoqQ@mail.gmail.com>
References: <20210706132714.8220-1-minipli@grsecurity.net>
 <CAHmME9o38fA4_nDVyS9FvO1EL-Z4fS+a2eCo6Xq7VzMSrPYNKQ@mail.gmail.com>
 <c772d6c2-dac7-8c11-7c41-aaa6435a0bd6@grsecurity.net>
 <CAHmME9oyopU8euSuVy7su91fia6xG5V+9i+QodCBdTGdR_rD5Q@mail.gmail.com>
 <3a2d41dc-effb-158c-4a52-d7eb282ecb7e@grsecurity.net>
 <CAHmME9r4gspVnZGhgHhB1nR6FvOaryLoZvAJwpRU=-=rrbdoqQ@mail.gmail.com>
Message-ID: <5bcdecdb-fbc0-4714-895d-1b1e6a87287a@grsecurity.net>

Am 06.12.21 um 15:48 schrieb Jason A. Donenfeld:
> I couldn't repro with a new kernel either. Only these old ones.
> 
> Here are some object files for the logs in the earlier message:
> https://data.zx2c4.com/curve25519-miscompile-a87440f8-7d1e-4179-be83-c82b4636a448.tar.zst

Just a heads-up, some of these seem to have been compiled with a recent
gcc which might not be what was intended?:

$ strings -fa curve25519-*.o | grep -i gcc
curve25519-3.11.10.o: GCC: (GNU) 4.9.4
curve25519-3.12.74.o: GCC: (Gentoo 11.2.0 p1) 11.2.0
curve25519-3.14.79.o: GCC: (Gentoo 11.2.0 p1) 11.2.0
curve25519-3.15.10.o: GCC: (GNU) 4.9.4
curve25519-3.16.85.o: GCC: (Gentoo 11.2.0 p1) 11.2.0
curve25519-3.17.8.o: GCC: (GNU) 5.5.0
curve25519-3.19.8.o: GCC: (GNU) 5.5.0
curve25519-4.0.9.o: GCC: (GNU) 5.5.0
curve25519-4.5.7.o: GCC: (GNU) 6.5.0
curve25519-4.6.7.o: GCC: (GNU) 6.5.0
curve25519-4.7.10.o: GCC: (GNU) 6.5.0
curve25519-4.8.17.o: GCC: (GNU) 6.5.0

Anyhow, now I'm able to reproduce it myself. Was a PEBCAK config failure.

I'll look into it and work on a fix!

Thanks,
Mathias


From Jason at zx2c4.com  Mon Dec  6 16:27:07 2021
From: Jason at zx2c4.com (Jason A. Donenfeld)
Date: Mon, 6 Dec 2021 17:27:07 +0100
Subject: [PATCH 0/2] wireguard-linux-compat: grsecurity compat patches
In-Reply-To: <5bcdecdb-fbc0-4714-895d-1b1e6a87287a@grsecurity.net>
References: <20210706132714.8220-1-minipli@grsecurity.net>
 <CAHmME9o38fA4_nDVyS9FvO1EL-Z4fS+a2eCo6Xq7VzMSrPYNKQ@mail.gmail.com>
 <c772d6c2-dac7-8c11-7c41-aaa6435a0bd6@grsecurity.net>
 <CAHmME9oyopU8euSuVy7su91fia6xG5V+9i+QodCBdTGdR_rD5Q@mail.gmail.com>
 <3a2d41dc-effb-158c-4a52-d7eb282ecb7e@grsecurity.net>
 <CAHmME9r4gspVnZGhgHhB1nR6FvOaryLoZvAJwpRU=-=rrbdoqQ@mail.gmail.com>
 <5bcdecdb-fbc0-4714-895d-1b1e6a87287a@grsecurity.net>
Message-ID: <CAHmME9opYoaoSTHud9sF4BOFBPS=NVYPJSGQr37bxNai26MGFg@mail.gmail.com>

Hi Mathias,

Oh, you're right about recent gcc. That actually _is_ intended, yet
they still fail. It would seem, then, that the problem is not so much
gcc version as it is some kernel patch that never made it to these
ancient kernels.

Jason

From minipli at grsecurity.net  Mon Dec  6 18:18:34 2021
From: minipli at grsecurity.net (Mathias Krause)
Date: Mon, 6 Dec 2021 19:18:34 +0100
Subject: [PATCH 0/2] wireguard-linux-compat: grsecurity compat patches
In-Reply-To: <CAHmME9opYoaoSTHud9sF4BOFBPS=NVYPJSGQr37bxNai26MGFg@mail.gmail.com>
References: <20210706132714.8220-1-minipli@grsecurity.net>
 <CAHmME9o38fA4_nDVyS9FvO1EL-Z4fS+a2eCo6Xq7VzMSrPYNKQ@mail.gmail.com>
 <c772d6c2-dac7-8c11-7c41-aaa6435a0bd6@grsecurity.net>
 <CAHmME9oyopU8euSuVy7su91fia6xG5V+9i+QodCBdTGdR_rD5Q@mail.gmail.com>
 <3a2d41dc-effb-158c-4a52-d7eb282ecb7e@grsecurity.net>
 <CAHmME9r4gspVnZGhgHhB1nR6FvOaryLoZvAJwpRU=-=rrbdoqQ@mail.gmail.com>
 <5bcdecdb-fbc0-4714-895d-1b1e6a87287a@grsecurity.net>
 <CAHmME9opYoaoSTHud9sF4BOFBPS=NVYPJSGQr37bxNai26MGFg@mail.gmail.com>
Message-ID: <1ef9bbf4-b8f8-18df-3c80-3181c2b5b9c5@grsecurity.net>

Hi Jason,

Am 06.12.21 um 17:27 schrieb Jason A. Donenfeld:
> Oh, you're right about recent gcc. That actually _is_ intended, yet
> they still fail. It would seem, then, that the problem is not so much
> gcc version as it is some kernel patch that never made it to these
> ancient kernels.

actually, it's the i/o constraints, they're wrong. 'out' is an input
operand but we specify it as an output one. Now this works when gcc
respects the "+" constraint, as in marking this operand as being read
and written, thereby implicitly requiring it to be initialized. But
looks like older gcc ignore that (at least when using alternatives) and
make the asm work on a stale 'out' operand, resulting in the selftest
failures and crashes you've seen.

The following change fixes it by putting 'out' to the input operand
list, where it really belongs to:

diff --git a/src/crypto/zinc/curve25519/curve25519-x86_64.c
b/src/crypto/zinc/curve25519/curve25519-x86_64.c
index 67f55affcf88..f26ed5d897ac 100644
--- a/src/crypto/zinc/curve25519/curve25519-x86_64.c
+++ b/src/crypto/zinc/curve25519/curve25519-x86_64.c
@@ -581,8 +581,8 @@ static inline void fsqr(u64 *out, const u64 *f, u64
*tmp)
                "  cmovc %%rdx, %%rax;"
                "  add %%rax, %%r8;"
                "  movq %%r8, 0(%0);"
-       : "+&r,&r" (tmp), "+&r,&r" (f), "+&r,m" (out)
-       :
+       : "+&r,&r" (tmp), "+&r,&r" (f)
+       : "r,m" (out)
        : "%rax", "%rcx", "%rdx", "%r8", "%r9", "%r10", "%r11", "%rbx",
"%r13", "%r14", "%r15", "memory", "cc"
        );
 }
@@ -743,8 +743,8 @@ static inline void fsqr2(u64 *out, const u64 *f, u64
*tmp)
                "  cmovc %%rdx, %%rax;"
                "  add %%rax, %%r8;"
                "  movq %%r8, 32(%0);"
-       : "+&r,&r" (tmp), "+&r,&r" (f), "+&r,m" (out)
-       :
+       : "+&r,&r" (tmp), "+&r,&r" (f)
+       : "r,m" (out)
        : "%rax", "%rcx", "%rdx", "%r8", "%r9", "%r10", "%r11", "%rbx",
"%r13", "%r14", "%r15", "memory", "cc"
        );
 }

We still need the early clobber constraint ("&") for 'tmp' and 'f' as
they are, in fact, written to early. But 'out' is only ever read, so can
be a normal input operand.

I'll create a proper patch and send it out tomorrow, if you don't beat
me to.


Thanks,
Mathias

From Jason at zx2c4.com  Mon Dec  6 18:55:46 2021
From: Jason at zx2c4.com (Jason A. Donenfeld)
Date: Mon, 6 Dec 2021 19:55:46 +0100
Subject: [PATCH 0/2] wireguard-linux-compat: grsecurity compat patches
In-Reply-To: <1ef9bbf4-b8f8-18df-3c80-3181c2b5b9c5@grsecurity.net>
References: <20210706132714.8220-1-minipli@grsecurity.net>
 <CAHmME9o38fA4_nDVyS9FvO1EL-Z4fS+a2eCo6Xq7VzMSrPYNKQ@mail.gmail.com>
 <c772d6c2-dac7-8c11-7c41-aaa6435a0bd6@grsecurity.net>
 <CAHmME9oyopU8euSuVy7su91fia6xG5V+9i+QodCBdTGdR_rD5Q@mail.gmail.com>
 <3a2d41dc-effb-158c-4a52-d7eb282ecb7e@grsecurity.net>
 <CAHmME9r4gspVnZGhgHhB1nR6FvOaryLoZvAJwpRU=-=rrbdoqQ@mail.gmail.com>
 <5bcdecdb-fbc0-4714-895d-1b1e6a87287a@grsecurity.net>
 <CAHmME9opYoaoSTHud9sF4BOFBPS=NVYPJSGQr37bxNai26MGFg@mail.gmail.com>
 <1ef9bbf4-b8f8-18df-3c80-3181c2b5b9c5@grsecurity.net>
Message-ID: <CAHmME9odv1V9wUrLsX16J68wVD3rq+op0VvaNxtsLGG8Rrz4tQ@mail.gmail.com>

Hi Mathias,

Nice detective work! I just loaded this up on the CI, so we'll see if
this does work across the board.

It sounds like the original code also had this bug -- the "r"(out)
should be in the output constraints, not in the input constraints,
right? Sounds like something I should report to the EverCrypt authors
and also fix upstream too then.

Jason

From Jason at zx2c4.com  Mon Dec  6 19:28:02 2021
From: Jason at zx2c4.com (Jason A. Donenfeld)
Date: Mon, 6 Dec 2021 20:28:02 +0100
Subject: [PATCH 0/2] wireguard-linux-compat: grsecurity compat patches
In-Reply-To: <CAHmME9odv1V9wUrLsX16J68wVD3rq+op0VvaNxtsLGG8Rrz4tQ@mail.gmail.com>
References: <20210706132714.8220-1-minipli@grsecurity.net>
 <CAHmME9o38fA4_nDVyS9FvO1EL-Z4fS+a2eCo6Xq7VzMSrPYNKQ@mail.gmail.com>
 <c772d6c2-dac7-8c11-7c41-aaa6435a0bd6@grsecurity.net>
 <CAHmME9oyopU8euSuVy7su91fia6xG5V+9i+QodCBdTGdR_rD5Q@mail.gmail.com>
 <3a2d41dc-effb-158c-4a52-d7eb282ecb7e@grsecurity.net>
 <CAHmME9r4gspVnZGhgHhB1nR6FvOaryLoZvAJwpRU=-=rrbdoqQ@mail.gmail.com>
 <5bcdecdb-fbc0-4714-895d-1b1e6a87287a@grsecurity.net>
 <CAHmME9opYoaoSTHud9sF4BOFBPS=NVYPJSGQr37bxNai26MGFg@mail.gmail.com>
 <1ef9bbf4-b8f8-18df-3c80-3181c2b5b9c5@grsecurity.net>
 <CAHmME9odv1V9wUrLsX16J68wVD3rq+op0VvaNxtsLGG8Rrz4tQ@mail.gmail.com>
Message-ID: <CAHmME9qhZZWf2SNFGWJepzuLXZx1o+AEa9-QunWupxBE8sk2mA@mail.gmail.com>

On Mon, Dec 6, 2021 at 7:55 PM Jason A. Donenfeld <Jason at zx2c4.com> wrote:
> Nice detective work! I just loaded this up on the CI, so we'll see if
> this does work across the board.

Looks like https://git.zx2c4.com/wireguard-linux-compat/commit/?id=42c931dbccf9570f10a84e282daf79f385d51623
is all green on https://www.wireguard.com/build-status/ in the
wireguard-linux-compat category. Let me know if that commit looks okay
to you or if you want to adjust something about it. After that, I'll
cut a new compat snapshot release.

Jason

From minipli at grsecurity.net  Mon Dec  6 20:54:12 2021
From: minipli at grsecurity.net (Mathias Krause)
Date: Mon, 6 Dec 2021 21:54:12 +0100
Subject: [PATCH 0/2] wireguard-linux-compat: grsecurity compat patches
In-Reply-To: <CAHmME9qhZZWf2SNFGWJepzuLXZx1o+AEa9-QunWupxBE8sk2mA@mail.gmail.com>
References: <20210706132714.8220-1-minipli@grsecurity.net>
 <CAHmME9o38fA4_nDVyS9FvO1EL-Z4fS+a2eCo6Xq7VzMSrPYNKQ@mail.gmail.com>
 <c772d6c2-dac7-8c11-7c41-aaa6435a0bd6@grsecurity.net>
 <CAHmME9oyopU8euSuVy7su91fia6xG5V+9i+QodCBdTGdR_rD5Q@mail.gmail.com>
 <3a2d41dc-effb-158c-4a52-d7eb282ecb7e@grsecurity.net>
 <CAHmME9r4gspVnZGhgHhB1nR6FvOaryLoZvAJwpRU=-=rrbdoqQ@mail.gmail.com>
 <5bcdecdb-fbc0-4714-895d-1b1e6a87287a@grsecurity.net>
 <CAHmME9opYoaoSTHud9sF4BOFBPS=NVYPJSGQr37bxNai26MGFg@mail.gmail.com>
 <1ef9bbf4-b8f8-18df-3c80-3181c2b5b9c5@grsecurity.net>
 <CAHmME9odv1V9wUrLsX16J68wVD3rq+op0VvaNxtsLGG8Rrz4tQ@mail.gmail.com>
 <CAHmME9qhZZWf2SNFGWJepzuLXZx1o+AEa9-QunWupxBE8sk2mA@mail.gmail.com>
Message-ID: <490048f6-0e1b-f058-753c-fdc1892b9167@grsecurity.net>

Hi Jason,

Am 06.12.21 um 20:28 schrieb Jason A. Donenfeld:
> On Mon, Dec 6, 2021 at 7:55 PM Jason A. Donenfeld <Jason at zx2c4.com> wrote:
>> Nice detective work! I just loaded this up on the CI, so we'll see if
>> this does work across the board.
> 
> Looks like https://git.zx2c4.com/wireguard-linux-compat/commit/?id=42c931dbccf9570f10a84e282daf79f385d51623
> is all green on https://www.wireguard.com/build-status/ in the
> wireguard-linux-compat category. Let me know if that commit looks okay
> to you or if you want to adjust something about it. After that, I'll
> cut a new compat snapshot release.

ah, you modified the original commit of mine. Yeah, that works too.
However, I'd add the following to the commit log to account for the
output to input operand move:

"""
Also make 'out' an input-only operand as it is only used as such. This
not only allows gcc to optimize its usage further, but also works around
older gcc versions, apparently failing to handle multiple alternatives
correctly, as in failing to initialize the 'out' operand with its input
value.
"""

Thanks,
Mathias

From minipli at grsecurity.net  Mon Dec  6 21:00:16 2021
From: minipli at grsecurity.net (Mathias Krause)
Date: Mon, 6 Dec 2021 22:00:16 +0100
Subject: [PATCH 0/2] wireguard-linux-compat: grsecurity compat patches
In-Reply-To: <CAHmME9odv1V9wUrLsX16J68wVD3rq+op0VvaNxtsLGG8Rrz4tQ@mail.gmail.com>
References: <20210706132714.8220-1-minipli@grsecurity.net>
 <CAHmME9o38fA4_nDVyS9FvO1EL-Z4fS+a2eCo6Xq7VzMSrPYNKQ@mail.gmail.com>
 <c772d6c2-dac7-8c11-7c41-aaa6435a0bd6@grsecurity.net>
 <CAHmME9oyopU8euSuVy7su91fia6xG5V+9i+QodCBdTGdR_rD5Q@mail.gmail.com>
 <3a2d41dc-effb-158c-4a52-d7eb282ecb7e@grsecurity.net>
 <CAHmME9r4gspVnZGhgHhB1nR6FvOaryLoZvAJwpRU=-=rrbdoqQ@mail.gmail.com>
 <5bcdecdb-fbc0-4714-895d-1b1e6a87287a@grsecurity.net>
 <CAHmME9opYoaoSTHud9sF4BOFBPS=NVYPJSGQr37bxNai26MGFg@mail.gmail.com>
 <1ef9bbf4-b8f8-18df-3c80-3181c2b5b9c5@grsecurity.net>
 <CAHmME9odv1V9wUrLsX16J68wVD3rq+op0VvaNxtsLGG8Rrz4tQ@mail.gmail.com>
Message-ID: <3ed9270f-00da-5c88-cadd-59b4419b9a98@grsecurity.net>

Am 06.12.21 um 19:55 schrieb Jason A. Donenfeld:
> Nice detective work! I just loaded this up on the CI, so we'll see if
> this does work across the board.

Thanks.

> It sounds like the original code also had this bug -- the "r"(out)
> should be in the output constraints, not in the input constraints,
> right? Sounds like something I should report to the EverCrypt authors
> and also fix upstream too then.

Yes, probably, but you're mixing up the two. "r"(out) should be an input
operand as it's only read in the inline asm. The other two ('tmp' and
'f') are read and written, so need to be output operands (as they
already are!) but also marked as earlyclobber ("&") as they're modified
before all input operands are read ('out', in this case). It's just a
hint to the compiler to not make the register allocation overlap with
any (other) input operand register.

Thanks,
Mathias

From liuhangbin at gmail.com  Tue Dec  7 02:54:21 2021
From: liuhangbin at gmail.com (Hangbin Liu)
Date: Tue,  7 Dec 2021 10:54:21 +0800
Subject: [PATCH wireguard] wireguard: remove redundancy include files
Message-ID: <20211207025421.1903782-1-liuhangbin@gmail.com>

Remove redundancy include files based on tool include-what-you-use
result. The tool will suggest using direct include headers and removing
some big headers. e.g. it will suggests to use linux/spinlock_types.h
for spinlock_t. But actually, we only need to include linux/spinlock.h
directly. So I just take the result as a reference and remove the include
files I do not find related structure/functions.

Tested with gcc-8.5.0 and the compilation passed.

Signed-off-by: Hangbin Liu <liuhangbin at gmail.com>
---
 drivers/net/wireguard/device.c   | 1 -
 drivers/net/wireguard/device.h   | 1 -
 drivers/net/wireguard/noise.c    | 4 ----
 drivers/net/wireguard/noise.h    | 1 -
 drivers/net/wireguard/peer.h     | 1 -
 drivers/net/wireguard/queueing.h | 2 --
 drivers/net/wireguard/send.c     | 5 -----
 drivers/net/wireguard/socket.c   | 2 --
 drivers/net/wireguard/socket.h   | 5 -----
 9 files changed, 22 deletions(-)

diff --git a/drivers/net/wireguard/device.c b/drivers/net/wireguard/device.c
index a46067c38bf5..06b74da2133f 100644
--- a/drivers/net/wireguard/device.c
+++ b/drivers/net/wireguard/device.c
@@ -11,7 +11,6 @@
 #include "peer.h"
 #include "messages.h"
 
-#include <linux/module.h>
 #include <linux/rtnetlink.h>
 #include <linux/inet.h>
 #include <linux/netdevice.h>
diff --git a/drivers/net/wireguard/device.h b/drivers/net/wireguard/device.h
index 43c7cebbf50b..0782dcef9fc1 100644
--- a/drivers/net/wireguard/device.h
+++ b/drivers/net/wireguard/device.h
@@ -8,7 +8,6 @@
 
 #include "noise.h"
 #include "allowedips.h"
-#include "peerlookup.h"
 #include "cookie.h"
 
 #include <linux/types.h>
diff --git a/drivers/net/wireguard/noise.c b/drivers/net/wireguard/noise.c
index c0cfd9b36c0b..59be94077767 100644
--- a/drivers/net/wireguard/noise.c
+++ b/drivers/net/wireguard/noise.c
@@ -7,14 +7,10 @@
 #include "device.h"
 #include "peer.h"
 #include "messages.h"
-#include "queueing.h"
 #include "peerlookup.h"
 
 #include <linux/rcupdate.h>
 #include <linux/slab.h>
-#include <linux/bitmap.h>
-#include <linux/scatterlist.h>
-#include <linux/highmem.h>
 #include <crypto/algapi.h>
 
 /* This implements Noise_IKpsk2:
diff --git a/drivers/net/wireguard/noise.h b/drivers/net/wireguard/noise.h
index c527253dba80..75b8f0d50084 100644
--- a/drivers/net/wireguard/noise.h
+++ b/drivers/net/wireguard/noise.h
@@ -12,7 +12,6 @@
 #include <linux/spinlock.h>
 #include <linux/atomic.h>
 #include <linux/rwsem.h>
-#include <linux/mutex.h>
 #include <linux/kref.h>
 
 struct noise_replay_counter {
diff --git a/drivers/net/wireguard/peer.h b/drivers/net/wireguard/peer.h
index 76e4d3128ad4..05af76b62583 100644
--- a/drivers/net/wireguard/peer.h
+++ b/drivers/net/wireguard/peer.h
@@ -11,7 +11,6 @@
 #include "cookie.h"
 
 #include <linux/types.h>
-#include <linux/netfilter.h>
 #include <linux/spinlock.h>
 #include <linux/kref.h>
 #include <net/dst_cache.h>
diff --git a/drivers/net/wireguard/queueing.h b/drivers/net/wireguard/queueing.h
index e2388107f7fd..bd54373b4485 100644
--- a/drivers/net/wireguard/queueing.h
+++ b/drivers/net/wireguard/queueing.h
@@ -9,8 +9,6 @@
 #include "peer.h"
 #include <linux/types.h>
 #include <linux/skbuff.h>
-#include <linux/ip.h>
-#include <linux/ipv6.h>
 #include <net/ip_tunnels.h>
 
 struct wg_device;
diff --git a/drivers/net/wireguard/send.c b/drivers/net/wireguard/send.c
index 5368f7c35b4b..f5b25f6fc55b 100644
--- a/drivers/net/wireguard/send.c
+++ b/drivers/net/wireguard/send.c
@@ -11,12 +11,7 @@
 #include "messages.h"
 #include "cookie.h"
 
-#include <linux/uio.h>
-#include <linux/inetdevice.h>
-#include <linux/socket.h>
 #include <net/ip_tunnels.h>
-#include <net/udp.h>
-#include <net/sock.h>
 
 static void wg_packet_send_handshake_initiation(struct wg_peer *peer)
 {
diff --git a/drivers/net/wireguard/socket.c b/drivers/net/wireguard/socket.c
index 6f07b949cb81..0d645613349a 100644
--- a/drivers/net/wireguard/socket.c
+++ b/drivers/net/wireguard/socket.c
@@ -9,9 +9,7 @@
 #include "queueing.h"
 #include "messages.h"
 
-#include <linux/ctype.h>
 #include <linux/net.h>
-#include <linux/if_vlan.h>
 #include <linux/if_ether.h>
 #include <linux/inetdevice.h>
 #include <net/udp_tunnel.h>
diff --git a/drivers/net/wireguard/socket.h b/drivers/net/wireguard/socket.h
index bab5848efbcd..d69c84f12e66 100644
--- a/drivers/net/wireguard/socket.h
+++ b/drivers/net/wireguard/socket.h
@@ -6,11 +6,6 @@
 #ifndef _WG_SOCKET_H
 #define _WG_SOCKET_H
 
-#include <linux/netdevice.h>
-#include <linux/udp.h>
-#include <linux/if_vlan.h>
-#include <linux/if_ether.h>
-
 int wg_socket_init(struct wg_device *wg, u16 port);
 void wg_socket_reinit(struct wg_device *wg, struct sock *new4,
 		      struct sock *new6);
-- 
2.31.1


From ich at malte.de  Mon Dec  6 18:19:24 2021
From: ich at malte.de (Malte Wedel)
Date: Mon, 6 Dec 2021 19:19:24 +0100
Subject: macOS client issue with line breaks in the configuration
Message-ID: <7838e77c-33a2-f829-81d2-c7d43a91a4e5@malte.de>

Hello,

I had an interesting issue with the macOS client, where I used copy and 
paste from an e-mail to fill in the configuration. Although everything 
looked syntactically correct, it was not syntax highlighted correctly 
and could not be saved. It took me a while to realize, this was caused 
by DOS line breaks in the source mail, which were copied with the text 
and could not be parsed properly.

 From my perspective, either the parser should be able to deal with 
unexpected line breaks, or line breaks should be normalized, when text 
is pasted to the text area, to avoid such an issue, which is hard to 
understand for the average user.

Regards,
Malte

-------------- next part --------------
A non-text attachment was scrubbed...
Name: wireguard.png
Type: image/png
Size: 58064 bytes
Desc: not available
URL: <http://lists.zx2c4.com/pipermail/wireguard/attachments/20211206/a78fd820/attachment.png>

From tobias at tangemann.org  Wed Dec  8 14:26:27 2021
From: tobias at tangemann.org (Tobias Tangemann)
Date: Wed, 8 Dec 2021 15:26:27 +0100
Subject: [PATCH] Periodically update endpoint IP in windows app (dynamic DNS)
Message-ID: <CAGqN2XZNsSgRAXpk8BaYE-ApOnBF0xynAjaQwLRi=CcmyJg5Ng@mail.gmail.com>

Hi,

I know of the reresolvedns script for Linux but there is no nice
solution for the other platforms. I think this is a feature many users
would benefit from if directly integrated into the wireguard apps
(including me).

I implemented a new config option which periodically updates the
endpoint IP in the same way the reresolvedns script does it, but
inside the windows app.
A go routine gets created inside the tunnel manager for each peer
which has the option set. if the option is not set for any peer there
is no change in behavior.
Kudos to the well-crafted build system and awesome build.bat for
windows, this made it very easy to get started.
I think there are some points left open:
- The name of the option cannot easily be changed later as it would
require additional migration code (I?m not tied to the name ?Update
endpoint IP?).
- Is the amount of logging this feature currently creates ok?
- I haven?t updated the localization to not blow this patch up to
much, this can be done after settling on a name for the option.
- Most of the code is ui related and pretty straight forward. I
implemented the PeerUpdateEndpointConfiguration based on my analysis
of the wireguard-tools set.c source and it seems to work but I think
someone with a deeper understanding of the driver interface should
check that. During my testing the feature seems to work fine with 1
peer, maybe someone can test this in a more complicated setup.
- I think this will not work if all traffic is routed through the VPN
including DNS therefor once the IP of the peer changes, the link
breaks and updating the endpoint IP will not work. I?m not sure if
there is a way to fix this.
- I?m not sure on the position of config options only being supported
on specific clients. I think this is already the case but I don?t have
a good overview.
It would be nice to also have this feature in the other clients like
iOS/macOS and android. I think it is not necessary to update the Linux
tool. If desired I think it could be included in wg-quick.
I?m unsure how to tackle this feature for macOS/iOS, my assumption is
that it would need to be included in the PacketTunnelProvider and be
part of the NetworkExtension or maybe in the userland driver
wireguard-go?
Sadly, due to all the apple signing and provisioning hoops you have to
jump through I was not able to get even the macOS app to build using
my free developer account.

I hope this patch can be reviewed and upstreamed.

The patch can also be found here:
https://github.com/WireGuard/wireguard-windows/pull/18

Best regards - Tobias Tangemann
-------------- next part --------------
A non-text attachment was scrubbed...
Name: 0001-tunnel-conf-ui-periodic-update-of-endpoint-ip.patch
Type: application/octet-stream
Size: 12617 bytes
Desc: not available
URL: <http://lists.zx2c4.com/pipermail/wireguard/attachments/20211208/9d57e6e8/attachment.obj>

From Jason at zx2c4.com  Wed Dec  8 14:56:15 2021
From: Jason at zx2c4.com (Jason A. Donenfeld)
Date: Wed, 8 Dec 2021 15:56:15 +0100
Subject: [PATCH 0/2] wireguard-linux-compat: grsecurity compat patches
In-Reply-To: <490048f6-0e1b-f058-753c-fdc1892b9167@grsecurity.net>
References: <20210706132714.8220-1-minipli@grsecurity.net>
 <CAHmME9o38fA4_nDVyS9FvO1EL-Z4fS+a2eCo6Xq7VzMSrPYNKQ@mail.gmail.com>
 <c772d6c2-dac7-8c11-7c41-aaa6435a0bd6@grsecurity.net>
 <CAHmME9oyopU8euSuVy7su91fia6xG5V+9i+QodCBdTGdR_rD5Q@mail.gmail.com>
 <3a2d41dc-effb-158c-4a52-d7eb282ecb7e@grsecurity.net>
 <CAHmME9r4gspVnZGhgHhB1nR6FvOaryLoZvAJwpRU=-=rrbdoqQ@mail.gmail.com>
 <5bcdecdb-fbc0-4714-895d-1b1e6a87287a@grsecurity.net>
 <CAHmME9opYoaoSTHud9sF4BOFBPS=NVYPJSGQr37bxNai26MGFg@mail.gmail.com>
 <1ef9bbf4-b8f8-18df-3c80-3181c2b5b9c5@grsecurity.net>
 <CAHmME9odv1V9wUrLsX16J68wVD3rq+op0VvaNxtsLGG8Rrz4tQ@mail.gmail.com>
 <CAHmME9qhZZWf2SNFGWJepzuLXZx1o+AEa9-QunWupxBE8sk2mA@mail.gmail.com>
 <490048f6-0e1b-f058-753c-fdc1892b9167@grsecurity.net>
Message-ID: <CAHmME9rrpQRFKuF37h6PJBU736kNfKUjAd7cLQ4WouYFWBnjwA@mail.gmail.com>

On Mon, Dec 6, 2021 at 9:54 PM Mathias Krause <minipli at grsecurity.net> wrote:
>
> Hi Jason,
>
> Am 06.12.21 um 20:28 schrieb Jason A. Donenfeld:
> > On Mon, Dec 6, 2021 at 7:55 PM Jason A. Donenfeld <Jason at zx2c4.com> wrote:
> >> Nice detective work! I just loaded this up on the CI, so we'll see if
> >> this does work across the board.
> >
> > Looks like https://git.zx2c4.com/wireguard-linux-compat/commit/?id=42c931dbccf9570f10a84e282daf79f385d51623
> > is all green on https://www.wireguard.com/build-status/ in the
> > wireguard-linux-compat category. Let me know if that commit looks okay
> > to you or if you want to adjust something about it. After that, I'll
> > cut a new compat snapshot release.
>
> ah, you modified the original commit of mine. Yeah, that works too.
> However, I'd add the following to the commit log to account for the
> output to input operand move:
>
> """
> Also make 'out' an input-only operand as it is only used as such. This
> not only allows gcc to optimize its usage further, but also works around
> older gcc versions, apparently failing to handle multiple alternatives
> correctly, as in failing to initialize the 'out' operand with its input
> value.
> """
>

Sure, will do.

Jason

From Jason at zx2c4.com  Wed Dec  8 14:56:50 2021
From: Jason at zx2c4.com (Jason A. Donenfeld)
Date: Wed, 8 Dec 2021 15:56:50 +0100
Subject: [PATCH 0/2] wireguard-linux-compat: grsecurity compat patches
In-Reply-To: <3ed9270f-00da-5c88-cadd-59b4419b9a98@grsecurity.net>
References: <20210706132714.8220-1-minipli@grsecurity.net>
 <CAHmME9o38fA4_nDVyS9FvO1EL-Z4fS+a2eCo6Xq7VzMSrPYNKQ@mail.gmail.com>
 <c772d6c2-dac7-8c11-7c41-aaa6435a0bd6@grsecurity.net>
 <CAHmME9oyopU8euSuVy7su91fia6xG5V+9i+QodCBdTGdR_rD5Q@mail.gmail.com>
 <3a2d41dc-effb-158c-4a52-d7eb282ecb7e@grsecurity.net>
 <CAHmME9r4gspVnZGhgHhB1nR6FvOaryLoZvAJwpRU=-=rrbdoqQ@mail.gmail.com>
 <5bcdecdb-fbc0-4714-895d-1b1e6a87287a@grsecurity.net>
 <CAHmME9opYoaoSTHud9sF4BOFBPS=NVYPJSGQr37bxNai26MGFg@mail.gmail.com>
 <1ef9bbf4-b8f8-18df-3c80-3181c2b5b9c5@grsecurity.net>
 <CAHmME9odv1V9wUrLsX16J68wVD3rq+op0VvaNxtsLGG8Rrz4tQ@mail.gmail.com>
 <3ed9270f-00da-5c88-cadd-59b4419b9a98@grsecurity.net>
Message-ID: <CAHmME9rFHkydmrYcS83PEf=MJP8brwsdis+7TQ=oLkO4RNTX9g@mail.gmail.com>

On Mon, Dec 6, 2021 at 10:00 PM Mathias Krause <minipli at grsecurity.net> wrote:
> Yes, probably, but you're mixing up the two.

Oh, thanks, right.

I'll talk to EverCrypt upstream and see.

Jason

From Jason at zx2c4.com  Wed Dec  8 15:20:37 2021
From: Jason at zx2c4.com (Jason A. Donenfeld)
Date: Wed, 08 Dec 2021 16:20:37 +0100
Subject: [ANNOUNCE] wireguard-linux-compat v1.0.20211208 released
Message-ID: <20211208152043.C5CF5C00446@smtp.kernel.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

Hello,

A new version, v1.0.20211208, of the backported WireGuard kernel module for
3.10 <= Linux <= 5.5.y has been tagged in the git repository.

== Changes ==

  * compat: account for latest c8s backports
  * compat: update for RHEL 8.5
  
  We now support RHEL 8.5 and C8S.
  
  * compat: account for grsecurity backports and changes
  * crypto: curve25519-x86_64: solve register constraints with reserved registers
  
  Better compatibility with recent grsecurity kernels.
  
  * netns: actually test for routing loops
  * main: rename 'mod_init' & 'mod_exit' functions to be module-specific
  * device: reset peer src endpoint when netns exits
  * receive: use ring buffer for incoming handshakes
  * receive: drop handshakes if queue lock is contended
  * ratelimiter: use kvcalloc() instead of kvzalloc()
  * compat: siphash: use _unaligned version by default
  
  Upstream commits from this merge:
  https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=cbd92e7d74314e10a90d12a8f18f6fc66be4fb16
  
  * compat: udp_tunnel: don't take reference to non-init namespace
  
  This is an old bug unearthed by the routing loop test case added in this
  cycle. It turns out that for kernels ?3.16, interfaces weren't being deleted
  when namespaces exited, due to improper namespace handling for kernel sockets
  in the udp_tunnel backport.

This release contains commits from: Jason A. Donenfeld, Mathias Krause, Randy 
Dunlap, Peter Georg, Gustavo A. R. Silva, and Arnd Bergmann.

As always, the source is available at https://git.zx2c4.com/wireguard-linux-compat/
and information about the project is available at https://www.wireguard.com/ .

This version is available in compressed tarball form here:
  https://git.zx2c4.com/wireguard-linux-compat/snapshot/wireguard-linux-compat-1.0.20211208.tar.xz
  SHA2-256: c0e607138a17daac656f508d8e63ea3737b5221fa5d9288191ddeb099f5a3b92

A PGP signature of that file decompressed is available here:
  https://git.zx2c4.com/wireguard-linux-compat/snapshot/wireguard-linux-compat-1.0.20211208.tar.asc
  Signing key: AB9942E6D4A4CFC3412620A749FC7012A5DE03AE
  Remember to unxz the tarball before verifying the signature.

If you're a package maintainer, please bump your package version. If you're a
user, the WireGuard team welcomes any and all feedback on this latest version.

Finally, WireGuard development thrives on donations. By popular demand, we
have a webpage for this: https://www.wireguard.com/donations/

Thank you,
Jason Donenfeld


-----BEGIN PGP SIGNATURE-----

iQJEBAEBCAAuFiEEq5lC5tSkz8NBJiCnSfxwEqXeA64FAmGwzR0QHGphc29uQHp4
MmM0LmNvbQAKCRBJ/HASpd4DrswXEACxL+6G+4gsFiUUlc/Z60QDYWXFVGJdlTIN
MTgypGU08JNyb1QlaKsc94R1aoV99rKyxS7luHvXr9d6EMF7O0j3SIFqUMD3c4fV
s2IZETCxe85qBdT6xSNhlf5QcuC+aBRXyruZ6WBUk5PvGUWn2/54ERV8UQxydRRu
PZNIvZpRpiHDqHBrCo+PaFsrxbhh3Kw2uDE/CntYILo2pJHhAwvSFpjxX+IzCcKO
/FFOQl99gWQT4ijEDscNz6KjQMLIM0mZ5vFjpjhfGIYLc1iXgVYdzCHKxqKmfUCP
ttYPbnrlnQrwwT63qCqJA7fncYGxVxFiZICdOnA4myr1w+0oCvR2h/FwRdtsEhYP
pwUJJeKugfZkqtNTy8lm5FfA/tfk06wn6WwUuyfXQHMJ9EMR/Uf44v9+qlOzg1cq
RcbXYpwTZ92zUMDCRX4sNvbaBwW01V/BiLtjijYh/UyNI0vvephk4e/+45bB71jd
6dfEREIiHzxNO6HZ16ZTAAj3pRn2YWpp5xYC+mqKIhkwML1g+3u7lP+f66Gql0rG
iX/ln3qqUNijJ3cUwpjgq3stejL6mCd87SvIo8G5j80wRbT1vDXQwpYQO+KxIRZP
NEi1HjWKVdg7zwBjAR0REwfIeqNFalQlfdxDPE2kmjSLFxHf0pvQUIpXgn+x+6TE
WzWDF0eboA==
=vx+c
-----END PGP SIGNATURE-----

From minipli at grsecurity.net  Thu Dec  9 07:59:41 2021
From: minipli at grsecurity.net (Mathias Krause)
Date: Thu, 9 Dec 2021 08:59:41 +0100
Subject: [PATCH 0/2] wireguard-linux-compat: grsecurity compat patches
In-Reply-To: <CAHmME9rFHkydmrYcS83PEf=MJP8brwsdis+7TQ=oLkO4RNTX9g@mail.gmail.com>
References: <20210706132714.8220-1-minipli@grsecurity.net>
 <CAHmME9o38fA4_nDVyS9FvO1EL-Z4fS+a2eCo6Xq7VzMSrPYNKQ@mail.gmail.com>
 <c772d6c2-dac7-8c11-7c41-aaa6435a0bd6@grsecurity.net>
 <CAHmME9oyopU8euSuVy7su91fia6xG5V+9i+QodCBdTGdR_rD5Q@mail.gmail.com>
 <3a2d41dc-effb-158c-4a52-d7eb282ecb7e@grsecurity.net>
 <CAHmME9r4gspVnZGhgHhB1nR6FvOaryLoZvAJwpRU=-=rrbdoqQ@mail.gmail.com>
 <5bcdecdb-fbc0-4714-895d-1b1e6a87287a@grsecurity.net>
 <CAHmME9opYoaoSTHud9sF4BOFBPS=NVYPJSGQr37bxNai26MGFg@mail.gmail.com>
 <1ef9bbf4-b8f8-18df-3c80-3181c2b5b9c5@grsecurity.net>
 <CAHmME9odv1V9wUrLsX16J68wVD3rq+op0VvaNxtsLGG8Rrz4tQ@mail.gmail.com>
 <3ed9270f-00da-5c88-cadd-59b4419b9a98@grsecurity.net>
 <CAHmME9rFHkydmrYcS83PEf=MJP8brwsdis+7TQ=oLkO4RNTX9g@mail.gmail.com>
Message-ID: <30231bb2-50e0-3880-b705-942b263eafe8@grsecurity.net>

Am 08.12.21 um 15:56 schrieb Jason A. Donenfeld:
> On Mon, Dec 6, 2021 at 10:00 PM Mathias Krause <minipli at grsecurity.net> wrote:
>> Yes, probably, but you're mixing up the two.
> 
> Oh, thanks, right.
> 
> I'll talk to EverCrypt upstream and see.

FWIW, 'out' is also wrongly flagged as output operand in fmul() and
fmul2(). But making it an input operand needs more surgery, as the
operand order changes and this requires some code churn.

Mathias

From Jason at zx2c4.com  Fri Dec 10 22:36:22 2021
From: Jason at zx2c4.com (Jason A. Donenfeld)
Date: Fri, 10 Dec 2021 23:36:22 +0100
Subject: [PATCH 0/2] wireguard-linux-compat: grsecurity compat patches
In-Reply-To: <30231bb2-50e0-3880-b705-942b263eafe8@grsecurity.net>
References: <20210706132714.8220-1-minipli@grsecurity.net>
 <CAHmME9o38fA4_nDVyS9FvO1EL-Z4fS+a2eCo6Xq7VzMSrPYNKQ@mail.gmail.com>
 <c772d6c2-dac7-8c11-7c41-aaa6435a0bd6@grsecurity.net>
 <CAHmME9oyopU8euSuVy7su91fia6xG5V+9i+QodCBdTGdR_rD5Q@mail.gmail.com>
 <3a2d41dc-effb-158c-4a52-d7eb282ecb7e@grsecurity.net>
 <CAHmME9r4gspVnZGhgHhB1nR6FvOaryLoZvAJwpRU=-=rrbdoqQ@mail.gmail.com>
 <5bcdecdb-fbc0-4714-895d-1b1e6a87287a@grsecurity.net>
 <CAHmME9opYoaoSTHud9sF4BOFBPS=NVYPJSGQr37bxNai26MGFg@mail.gmail.com>
 <1ef9bbf4-b8f8-18df-3c80-3181c2b5b9c5@grsecurity.net>
 <CAHmME9odv1V9wUrLsX16J68wVD3rq+op0VvaNxtsLGG8Rrz4tQ@mail.gmail.com>
 <3ed9270f-00da-5c88-cadd-59b4419b9a98@grsecurity.net>
 <CAHmME9rFHkydmrYcS83PEf=MJP8brwsdis+7TQ=oLkO4RNTX9g@mail.gmail.com>
 <30231bb2-50e0-3880-b705-942b263eafe8@grsecurity.net>
Message-ID: <CAHmME9pYPXm8xTZkWYS8_ir1j0Dq7QcbtNKM2tn4uO51HTbr4Q@mail.gmail.com>

CC'ing in Aymeric, who's working on Vale's codegen.

On Thu, Dec 9, 2021 at 8:59 AM Mathias Krause <minipli at grsecurity.net> wrote:
>
> Am 08.12.21 um 15:56 schrieb Jason A. Donenfeld:
> > On Mon, Dec 6, 2021 at 10:00 PM Mathias Krause <minipli at grsecurity.net> wrote:
> >> Yes, probably, but you're mixing up the two.
> >
> > Oh, thanks, right.
> >
> > I'll talk to EverCrypt upstream and see.
>
> FWIW, 'out' is also wrongly flagged as output operand in fmul() and
> fmul2(). But making it an input operand needs more surgery, as the
> operand order changes and this requires some code churn.
>
> Mathias

From Jason at zx2c4.com  Fri Dec 10 22:58:01 2021
From: Jason at zx2c4.com (Jason A. Donenfeld)
Date: Fri, 10 Dec 2021 23:58:01 +0100
Subject: [PATCH 0/2] wireguard-linux-compat: grsecurity compat patches
In-Reply-To: <30231bb2-50e0-3880-b705-942b263eafe8@grsecurity.net>
References: <20210706132714.8220-1-minipli@grsecurity.net>
 <CAHmME9o38fA4_nDVyS9FvO1EL-Z4fS+a2eCo6Xq7VzMSrPYNKQ@mail.gmail.com>
 <c772d6c2-dac7-8c11-7c41-aaa6435a0bd6@grsecurity.net>
 <CAHmME9oyopU8euSuVy7su91fia6xG5V+9i+QodCBdTGdR_rD5Q@mail.gmail.com>
 <3a2d41dc-effb-158c-4a52-d7eb282ecb7e@grsecurity.net>
 <CAHmME9r4gspVnZGhgHhB1nR6FvOaryLoZvAJwpRU=-=rrbdoqQ@mail.gmail.com>
 <5bcdecdb-fbc0-4714-895d-1b1e6a87287a@grsecurity.net>
 <CAHmME9opYoaoSTHud9sF4BOFBPS=NVYPJSGQr37bxNai26MGFg@mail.gmail.com>
 <1ef9bbf4-b8f8-18df-3c80-3181c2b5b9c5@grsecurity.net>
 <CAHmME9odv1V9wUrLsX16J68wVD3rq+op0VvaNxtsLGG8Rrz4tQ@mail.gmail.com>
 <3ed9270f-00da-5c88-cadd-59b4419b9a98@grsecurity.net>
 <CAHmME9rFHkydmrYcS83PEf=MJP8brwsdis+7TQ=oLkO4RNTX9g@mail.gmail.com>
 <30231bb2-50e0-3880-b705-942b263eafe8@grsecurity.net>
Message-ID: <CAHmME9rT+aR-FJKFxVaW+tsACKX0kH-2T6cw7xbjjaeVsJbybw@mail.gmail.com>

CC'ing in Aymeric, who's working on Vale's codegen.

On Thu, Dec 9, 2021 at 8:59 AM Mathias Krause <minipli at grsecurity.net> wrote:
>
> Am 08.12.21 um 15:56 schrieb Jason A. Donenfeld:
> > On Mon, Dec 6, 2021 at 10:00 PM Mathias Krause <minipli at grsecurity.net> wrote:
> >> Yes, probably, but you're mixing up the two.
> >
> > Oh, thanks, right.
> >
> > I'll talk to EverCrypt upstream and see.
>
> FWIW, 'out' is also wrongly flagged as output operand in fmul() and
> fmul2(). But making it an input operand needs more surgery, as the
> operand order changes and this requires some code churn.
>
> Mathias

From bigeasy at linutronix.de  Wed Dec  8 17:32:05 2021
From: bigeasy at linutronix.de (Sebastian Andrzej Siewior)
Date: Wed, 8 Dec 2021 18:32:05 +0100
Subject: [RFC] wiregard RX packet processing.
Message-ID: <20211208173205.zajfvg6zvi4g5kln@linutronix.de>

I didn't understand everything, I just stumbled upon this while looking
for something else and don't have the time to figure everything out.
Also I might haven taken a wrong turn somewhere?

need_resched() is something you want avoid unless you write core code.
On a PREEMPT kernel you never observe true here and cond_resched() is a
nop. On non-PREEMPT kernels need_resched() can return true/ false _and_
should_resched() (which is part of cond_resched()) returns only true if
the same bit is true. This means invoking only cond_resched() saves one
read access. Bonus points: On x86 that bit is folded into the preemption
counter so you avoid reading that bit entirely plus the whole thing is
optimized away on a PREEMPT kernel.

wg_queue_enqueue_per_peer_rx() enqueues somehow skb for NAPI processing
(this bit I haven't figured out yet but it has to) and then invokes
napi_schedule(). This napi_schedule() wasn't meant to be invoked from
preemptible context, only from an actual IRQ handler:
- if NAPI is already active (which can only happen if it is running on a
  remote CPU) then nothing happens. Good.

- if NAPI is idle then __napi_schedule() will "schedule" it. Here is
  the thing: You are in process context (kworker) so nothing happens
  right away: NET_RX_SOFTIRQ is set for the local CPU and NAPI struct is
  added to the list. Now you need to wait until a random interrupt
  appears which will notice that a softirq bit is set and will process
  it. So it will happen eventually?

I would suggest to either:
- add a comment that this is know and it doesn't not matter because
  $REASON. I would imagine you might want to batch multiple skbs but?

- add a BH disable section around wg_queue_enqueue_per_peer_rx() (see
  below). That bh-enable() will invoke pending softirqs which in your
  case should invoke wg_packet_rx_poll() where you see only one skb.

diff --git a/drivers/net/wireguard/receive.c b/drivers/net/wireguard/receive.c
index 7b8df406c7737..64e4ca1ded108 100644
--- a/drivers/net/wireguard/receive.c
+++ b/drivers/net/wireguard/receive.c
@@ -507,9 +507,11 @@ void wg_packet_decrypt_worker(struct work_struct *work)
 		enum packet_state state =
 			likely(decrypt_packet(skb, PACKET_CB(skb)->keypair)) ?
 				PACKET_STATE_CRYPTED : PACKET_STATE_DEAD;
+		local_bh_disable();
 		wg_queue_enqueue_per_peer_rx(skb, state);
-		if (need_resched())
-			cond_resched();
+		local_bh_enable();
+
+		cond_resched();
 	}
 }
 

Sebastian

From powergame_coder2 at yahoo.com  Thu Dec  9 10:47:26 2021
From: powergame_coder2 at yahoo.com (Mehdi Haghgoo)
Date: Thu, 9 Dec 2021 10:47:26 +0000 (UTC)
Subject: Cannot add wg0 on CentOS 8
References: <1877736804.292900.1639046846065.ref@mail.yahoo.com>
Message-ID: <1877736804.292900.1639046846065@mail.yahoo.com>

Hi,

I'm trying to add? a wireguard device on CentOS Stream (was Linux just converted to Stream). 
When I enter "ip link add wg0 type wireguard" or try it using wg-quick wg0 up, I get the following error: 

RTNETLINK answers: Operation not supported

I have wireguard-tools and kmod-wireguard installed. 
Additional information: 

uname -r: 4.18.0-147.8.1.el8_1.x86_64

wg --version: wireguard-tools v1.0.20210914 - https://git.zx2c4.com/wireguard-tools/

kmod-wireguard version: 1.0.20211208


Best regards,
Mehdi

From aep at exys.org  Thu Dec  9 15:36:17 2021
From: aep at exys.org (Arvid Picciani)
Date: Thu, 9 Dec 2021 16:36:17 +0100
Subject: separate allowedips from routing for multipath
Message-ID: <CACD9L=CAinAO0CHRR25-sEoxCqNp_8BCqU9kJLK-Wzt++oy4bg@mail.gmail.com>

Hey,

i'm considering using wireguard for a system where there can be
multiple routes to an ip.

The nearest solution i can think of is constantly changing the peer
AllowedIps from userspace and load a large list of routes into it. But
the challenge is that this is used for both routing decisions AND
policy decisions.

let's say we have 3 machines A1, A2, B and a machine that moves around V

V is currently connected via layer2 at A1,
B has V in AllowedIps for peer A1, so that any packet for V is routed to A1

now we connect V to A2 instead
we need to move the AllowedIps entry from A1 to A2 so that a packet
for V is routed to A2
old packets still arriving FROM V through A1 are now dropped

This wouldnt be an issue if AllowedIps could be separated from the
routing decision somehow.

I could have a wg interface per peer so i can do the routing using
regular linux tools, but wg doesnt like reusing the same port for
multiple wg endpoints. We'll eventually run out of ports.

I suppose port reuse is intentionally not allowed?

thanks
Arvid

From reidrankin at gmail.com  Fri Dec 10 23:29:59 2021
From: reidrankin at gmail.com (Reid Rankin)
Date: Fri, 10 Dec 2021 18:29:59 -0500
Subject: separate allowedips from routing for multipath
In-Reply-To: <CACD9L=CAinAO0CHRR25-sEoxCqNp_8BCqU9kJLK-Wzt++oy4bg@mail.gmail.com>
References: <CACD9L=CAinAO0CHRR25-sEoxCqNp_8BCqU9kJLK-Wzt++oy4bg@mail.gmail.com>
Message-ID: <CAMaqUZ1yWooNbHdDhTAzrUeHtQCFn7QunSWdAdhdG+Ru6=K9Mg@mail.gmail.com>

WireGuard doesn't care what address V has, because peers are defined
by their public key. If V switches from A1 to A2, upon receiving a
handshake packet from V's key from the address A2, B will start
sending packets for V out to A2. Incoming packets can arrive from any
IP; if they're from a connection with V, V's AllowedIP rules will be
applied.

From pperry at elrepo.org  Fri Dec 10 23:33:47 2021
From: pperry at elrepo.org (Phil Perry)
Date: Fri, 10 Dec 2021 23:33:47 +0000
Subject: Cannot add wg0 on CentOS 8
In-Reply-To: <1877736804.292900.1639046846065@mail.yahoo.com>
References: <1877736804.292900.1639046846065.ref@mail.yahoo.com>
 <1877736804.292900.1639046846065@mail.yahoo.com>
Message-ID: <46de3fe6-b355-41c3-d92e-dfdffb163dd9@elrepo.org>

On 09/12/2021 10:47, Mehdi Haghgoo wrote:
> Hi,
> 
> I'm trying to add? a wireguard device on CentOS Stream (was Linux just converted to Stream).
> When I enter "ip link add wg0 type wireguard" or try it using wg-quick wg0 up, I get the following error:
> 
> RTNETLINK answers: Operation not supported
> 
> I have wireguard-tools and kmod-wireguard installed.
> Additional information:
> 
> uname -r: 4.18.0-147.8.1.el8_1.x86_64
> 
> wg --version: wireguard-tools v1.0.20210914 - https://git.zx2c4.com/wireguard-tools/
> 
> kmod-wireguard version: 1.0.20211208
> 
> 
> Best regards,
> Mehdi

Is the wireguard module loaded? I doubt it.

You are running a *very* old kernel, and I suspect the version of 
kmod-wireguard you are running does not support that kernel. Why are you 
running a kernel that is two years out of date and full of unpatched 
security holes? Updating your system is probably the first step to 
fixing your issue.


From aep at exys.org  Sat Dec 11 07:28:37 2021
From: aep at exys.org (Arvid Picciani)
Date: Sat, 11 Dec 2021 08:28:37 +0100
Subject: separate allowedips from routing for multipath
In-Reply-To: <CAMaqUZ1yWooNbHdDhTAzrUeHtQCFn7QunSWdAdhdG+Ru6=K9Mg@mail.gmail.com>
References: <CACD9L=CAinAO0CHRR25-sEoxCqNp_8BCqU9kJLK-Wzt++oy4bg@mail.gmail.com>
 <CAMaqUZ1yWooNbHdDhTAzrUeHtQCFn7QunSWdAdhdG+Ru6=K9Mg@mail.gmail.com>
Message-ID: <CACD9L=BWM_OPmBjTh7JUsBkL3A6V_hiyHDos=1MrEkJ5P3QhWg@mail.gmail.com>

Thanks Reid,

i'm aware of how that works, but that's not the question.
V is an IP routed on A1 or A2, not a "road warrior" case.

I can look into if port reuse is possible if a patchset doing that
would be acceptable.

On Sat, Dec 11, 2021 at 12:30 AM Reid Rankin <reidrankin at gmail.com> wrote:
>
> WireGuard doesn't care what address V has, because peers are defined
> by their public key. If V switches from A1 to A2, upon receiving a
> handshake packet from V's key from the address A2, B will start
> sending packets for V out to A2. Incoming packets can arrive from any
> IP; if they're from a connection with V, V's AllowedIP rules will be
> applied.



-- 
+4916093821054

From aymeric.fromherz at inria.fr  Sat Dec 11 16:35:04 2021
From: aymeric.fromherz at inria.fr (Aymeric Fromherz)
Date: Sat, 11 Dec 2021 17:35:04 +0100 (CET)
Subject: [PATCH 0/2] wireguard-linux-compat: grsecurity compat patches
In-Reply-To: <CAHmME9rT+aR-FJKFxVaW+tsACKX0kH-2T6cw7xbjjaeVsJbybw@mail.gmail.com>
References: <20210706132714.8220-1-minipli@grsecurity.net>
 <CAHmME9opYoaoSTHud9sF4BOFBPS=NVYPJSGQr37bxNai26MGFg@mail.gmail.com>
 <1ef9bbf4-b8f8-18df-3c80-3181c2b5b9c5@grsecurity.net>
 <CAHmME9odv1V9wUrLsX16J68wVD3rq+op0VvaNxtsLGG8Rrz4tQ@mail.gmail.com>
 <3ed9270f-00da-5c88-cadd-59b4419b9a98@grsecurity.net>
 <CAHmME9rFHkydmrYcS83PEf=MJP8brwsdis+7TQ=oLkO4RNTX9g@mail.gmail.com>
 <30231bb2-50e0-3880-b705-942b263eafe8@grsecurity.net>
 <CAHmME9rT+aR-FJKFxVaW+tsACKX0kH-2T6cw7xbjjaeVsJbybw@mail.gmail.com>
Message-ID: <1554725710.1290070.1639240504281.JavaMail.zimbra@inria.fr>

Thanks for the heads-up. We were being overly conservative during verification of inline assembly code in Vale, and marked several registers as possibly modified while they were only read.

This is now fixed for fmul, fmul2, fsqr and fsqr2, and will be merged into the master branch of EverCrypt shortly.
In the meantime, the diff for the resulting inline assembly after Vale codegen is available here: https://github.com/project-everest/hacl-star/pull/501/commits/1a71adb40c3f78da16e16975dbb1d4de5adeab8c#diff-5aabe9f6aa87508c9d81d4c9e89eff0b06b1e2aeaf5b04eba51da71c5bea6940

Cheers,
Aymeric

----- Mail original -----
> De: "Jason A. Donenfeld" <Jason at zx2c4.com>
> ?: "Mathias Krause" <minipli at grsecurity.net>, "aymeric fromherz" <aymeric.fromherz at inria.fr>
> Cc: "WireGuard mailing list" <wireguard at lists.zx2c4.com>
> Envoy?: Vendredi 10 D?cembre 2021 23:58:01
> Objet: Re: [PATCH 0/2] wireguard-linux-compat: grsecurity compat patches

> CC'ing in Aymeric, who's working on Vale's codegen.
> 
> On Thu, Dec 9, 2021 at 8:59 AM Mathias Krause <minipli at grsecurity.net> wrote:
>>
>> Am 08.12.21 um 15:56 schrieb Jason A. Donenfeld:
>> > On Mon, Dec 6, 2021 at 10:00 PM Mathias Krause <minipli at grsecurity.net> wrote:
>> >> Yes, probably, but you're mixing up the two.
>> >
>> > Oh, thanks, right.
>> >
>> > I'll talk to EverCrypt upstream and see.
>>
>> FWIW, 'out' is also wrongly flagged as output operand in fmul() and
>> fmul2(). But making it an input operand needs more surgery, as the
>> operand order changes and this requires some code churn.
>>
> > Mathias

From toke at toke.dk  Sat Dec 11 17:25:24 2021
From: toke at toke.dk (Toke =?utf-8?Q?H=C3=B8iland-J=C3=B8rgensen?=)
Date: Sat, 11 Dec 2021 18:25:24 +0100
Subject: Cannot add wg0 on CentOS 8
In-Reply-To: <46de3fe6-b355-41c3-d92e-dfdffb163dd9@elrepo.org>
References: <1877736804.292900.1639046846065.ref@mail.yahoo.com>
 <1877736804.292900.1639046846065@mail.yahoo.com>
 <46de3fe6-b355-41c3-d92e-dfdffb163dd9@elrepo.org>
Message-ID: <871r2jvyt7.fsf@toke.dk>

Phil Perry <pperry at elrepo.org> writes:

> On 09/12/2021 10:47, Mehdi Haghgoo wrote:
>> Hi,
>> 
>> I'm trying to add? a wireguard device on CentOS Stream (was Linux just converted to Stream).
>> When I enter "ip link add wg0 type wireguard" or try it using wg-quick wg0 up, I get the following error:
>> 
>> RTNETLINK answers: Operation not supported
>> 
>> I have wireguard-tools and kmod-wireguard installed.
>> Additional information:
>> 
>> uname -r: 4.18.0-147.8.1.el8_1.x86_64
>> 
>> wg --version: wireguard-tools v1.0.20210914 - https://git.zx2c4.com/wireguard-tools/
>> 
>> kmod-wireguard version: 1.0.20211208
>> 
>> 
>> Best regards,
>> Mehdi
>
> Is the wireguard module loaded? I doubt it.
>
> You are running a *very* old kernel, and I suspect the version of 
> kmod-wireguard you are running does not support that kernel. Why are you 
> running a kernel that is two years out of date and full of unpatched 
> security holes?

RHEL (and thus CentOS) kernel version numbers are a complete fiction, so
don't mind the '4.18'. The important bit is the '-147' at the end, which
is still way out of date, though...

> Updating your system is probably the first step to fixing your issue.

My guess would be that he actually did update the system, it's just
running the old kernel still? I.e., if you install kmod-wireguard from
elrepo, that depends on a specific version of the kernel (-348 in the
current version), but if you haven't rebooted to actually run that
kernel, the module is obviously not going to work...

So, erm, Mehdi, have you tried turning it off and back on again? ;)

-Toke

From Jason at zx2c4.com  Sun Dec 12 21:43:45 2021
From: Jason at zx2c4.com (Jason A. Donenfeld)
Date: Sun, 12 Dec 2021 22:43:45 +0100
Subject: [PATCH 0/2] wireguard-linux-compat: grsecurity compat patches
In-Reply-To: <1554725710.1290070.1639240504281.JavaMail.zimbra@inria.fr>
References: <20210706132714.8220-1-minipli@grsecurity.net>
 <CAHmME9opYoaoSTHud9sF4BOFBPS=NVYPJSGQr37bxNai26MGFg@mail.gmail.com>
 <1ef9bbf4-b8f8-18df-3c80-3181c2b5b9c5@grsecurity.net>
 <CAHmME9odv1V9wUrLsX16J68wVD3rq+op0VvaNxtsLGG8Rrz4tQ@mail.gmail.com>
 <3ed9270f-00da-5c88-cadd-59b4419b9a98@grsecurity.net>
 <CAHmME9rFHkydmrYcS83PEf=MJP8brwsdis+7TQ=oLkO4RNTX9g@mail.gmail.com>
 <30231bb2-50e0-3880-b705-942b263eafe8@grsecurity.net>
 <CAHmME9rT+aR-FJKFxVaW+tsACKX0kH-2T6cw7xbjjaeVsJbybw@mail.gmail.com>
 <1554725710.1290070.1639240504281.JavaMail.zimbra@inria.fr>
Message-ID: <CAHmME9ppTs-jmmV9uV+C0jbtQC5a5RWO5zJB5_-dRD3BMjDZZA@mail.gmail.com>

Hi Aymeric, Mathias,

On Sat, Dec 11, 2021 at 5:35 PM Aymeric Fromherz
<aymeric.fromherz at inria.fr> wrote:
>
> Thanks for the heads-up. We were being overly conservative during verification of inline assembly code in Vale, and marked several registers as possibly modified while they were only read.
>
> This is now fixed for fmul, fmul2, fsqr and fsqr2, and will be merged into the master branch of EverCrypt shortly.
> In the meantime, the diff for the resulting inline assembly after Vale codegen is available here: https://github.com/project-everest/hacl-star/pull/501/commits/1a71adb40c3f78da16e16975dbb1d4de5adeab8c#diff-5aabe9f6aa87508c9d81d4c9e89eff0b06b1e2aeaf5b04eba51da71c5bea6940

Thanks for that! I've imported this into my staging tree here:
https://w-g.pw/l/rmQK

I'll wait a few days -- and until the Vale PR has been merged -- to
submit that to the linux-crypto mailing list, but there's a preview.
Mathias - I wonder if you still need alternatives with the new codegen
there.

Jason

From minipli at grsecurity.net  Mon Dec 13 07:44:51 2021
From: minipli at grsecurity.net (Mathias Krause)
Date: Mon, 13 Dec 2021 08:44:51 +0100
Subject: [PATCH 0/2] wireguard-linux-compat: grsecurity compat patches
In-Reply-To: <1554725710.1290070.1639240504281.JavaMail.zimbra@inria.fr>
References: <20210706132714.8220-1-minipli@grsecurity.net>
 <CAHmME9opYoaoSTHud9sF4BOFBPS=NVYPJSGQr37bxNai26MGFg@mail.gmail.com>
 <1ef9bbf4-b8f8-18df-3c80-3181c2b5b9c5@grsecurity.net>
 <CAHmME9odv1V9wUrLsX16J68wVD3rq+op0VvaNxtsLGG8Rrz4tQ@mail.gmail.com>
 <3ed9270f-00da-5c88-cadd-59b4419b9a98@grsecurity.net>
 <CAHmME9rFHkydmrYcS83PEf=MJP8brwsdis+7TQ=oLkO4RNTX9g@mail.gmail.com>
 <30231bb2-50e0-3880-b705-942b263eafe8@grsecurity.net>
 <CAHmME9rT+aR-FJKFxVaW+tsACKX0kH-2T6cw7xbjjaeVsJbybw@mail.gmail.com>
 <1554725710.1290070.1639240504281.JavaMail.zimbra@inria.fr>
Message-ID: <3a1be638-4ddd-44c0-87aa-56cd26f4f396@grsecurity.net>

Hi Aymeric,

the changes look good to me -- quite what we already had in grsec. Just
one more nit. The constraints for the 'out' parameter in fmul(),
fmul2(), fsqr() and fsqr2() can be further lifted to "rm" as 'out' is
only referenced once. This allows gcc to choose either a register or a
memory operand, as it sees fit. In our experiments the latter lead to
much better code gen.

Thanks,
Mathias

Am 11.12.21 um 17:35 schrieb Aymeric Fromherz:
> Thanks for the heads-up. We were being overly conservative during verification of inline assembly code in Vale, and marked several registers as possibly modified while they were only read.
> 
> This is now fixed for fmul, fmul2, fsqr and fsqr2, and will be merged into the master branch of EverCrypt shortly.
> In the meantime, the diff for the resulting inline assembly after Vale codegen is available here: https://github.com/project-everest/hacl-star/pull/501/commits/1a71adb40c3f78da16e16975dbb1d4de5adeab8c#diff-5aabe9f6aa87508c9d81d4c9e89eff0b06b1e2aeaf5b04eba51da71c5bea6940
> 
> Cheers,
> Aymeric
> 
> ----- Mail original -----
>> De: "Jason A. Donenfeld" <Jason at zx2c4.com>
>> ?: "Mathias Krause" <minipli at grsecurity.net>, "aymeric fromherz" <aymeric.fromherz at inria.fr>
>> Cc: "WireGuard mailing list" <wireguard at lists.zx2c4.com>
>> Envoy?: Vendredi 10 D?cembre 2021 23:58:01
>> Objet: Re: [PATCH 0/2] wireguard-linux-compat: grsecurity compat patches
> 
>> CC'ing in Aymeric, who's working on Vale's codegen.
>>
>> On Thu, Dec 9, 2021 at 8:59 AM Mathias Krause <minipli at grsecurity.net> wrote:
>>>
>>> Am 08.12.21 um 15:56 schrieb Jason A. Donenfeld:
>>>> On Mon, Dec 6, 2021 at 10:00 PM Mathias Krause <minipli at grsecurity.net> wrote:
>>>>> Yes, probably, but you're mixing up the two.
>>>>
>>>> Oh, thanks, right.
>>>>
>>>> I'll talk to EverCrypt upstream and see.
>>>
>>> FWIW, 'out' is also wrongly flagged as output operand in fmul() and
>>> fmul2(). But making it an input operand needs more surgery, as the
>>> operand order changes and this requires some code churn.
>>>
>>> Mathias

From minipli at grsecurity.net  Mon Dec 13 07:54:17 2021
From: minipli at grsecurity.net (Mathias Krause)
Date: Mon, 13 Dec 2021 08:54:17 +0100
Subject: [PATCH 0/2] wireguard-linux-compat: grsecurity compat patches
In-Reply-To: <CAHmME9ppTs-jmmV9uV+C0jbtQC5a5RWO5zJB5_-dRD3BMjDZZA@mail.gmail.com>
References: <20210706132714.8220-1-minipli@grsecurity.net>
 <CAHmME9opYoaoSTHud9sF4BOFBPS=NVYPJSGQr37bxNai26MGFg@mail.gmail.com>
 <1ef9bbf4-b8f8-18df-3c80-3181c2b5b9c5@grsecurity.net>
 <CAHmME9odv1V9wUrLsX16J68wVD3rq+op0VvaNxtsLGG8Rrz4tQ@mail.gmail.com>
 <3ed9270f-00da-5c88-cadd-59b4419b9a98@grsecurity.net>
 <CAHmME9rFHkydmrYcS83PEf=MJP8brwsdis+7TQ=oLkO4RNTX9g@mail.gmail.com>
 <30231bb2-50e0-3880-b705-942b263eafe8@grsecurity.net>
 <CAHmME9rT+aR-FJKFxVaW+tsACKX0kH-2T6cw7xbjjaeVsJbybw@mail.gmail.com>
 <1554725710.1290070.1639240504281.JavaMail.zimbra@inria.fr>
 <CAHmME9ppTs-jmmV9uV+C0jbtQC5a5RWO5zJB5_-dRD3BMjDZZA@mail.gmail.com>
Message-ID: <d1921390-7fcb-a627-11ea-ce580214db63@grsecurity.net>

Hi Jason,

Am 12.12.21 um 22:43 schrieb Jason A. Donenfeld:
> [snip]
>  
> Thanks for that! I've imported this into my staging tree here:
> https://w-g.pw/l/rmQK
> 
> I'll wait a few days -- and until the Vale PR has been merged -- to
> submit that to the linux-crypto mailing list, but there's a preview.
> Mathias - I wonder if you still need alternatives with the new codegen
> there.

yes, the alternative is still needed, as Aymeric's change doesn't lower
the number of required registers -- neither would a "rm" operand
constraint do, as gcc still insists in all constraints to be possible at
least, even if it would choose "m" instead of "r" in the end.

So, after importing Aymeric's changes into wireguard-linux-compat, the
alternatives change is still needed. I can help you create one, if you
want me to (we have that change in grsec anyway, just used named asm
operands here, as the %0/%1/%2/... was rather confusing and hard to follow).

Thanks,
Mathias



From Jason at zx2c4.com  Mon Dec 13 11:36:25 2021
From: Jason at zx2c4.com (Jason A. Donenfeld)
Date: Mon, 13 Dec 2021 12:36:25 +0100
Subject: [PATCH 0/2] wireguard-linux-compat: grsecurity compat patches
In-Reply-To: <d1921390-7fcb-a627-11ea-ce580214db63@grsecurity.net>
References: <20210706132714.8220-1-minipli@grsecurity.net>
 <CAHmME9opYoaoSTHud9sF4BOFBPS=NVYPJSGQr37bxNai26MGFg@mail.gmail.com>
 <1ef9bbf4-b8f8-18df-3c80-3181c2b5b9c5@grsecurity.net>
 <CAHmME9odv1V9wUrLsX16J68wVD3rq+op0VvaNxtsLGG8Rrz4tQ@mail.gmail.com>
 <3ed9270f-00da-5c88-cadd-59b4419b9a98@grsecurity.net>
 <CAHmME9rFHkydmrYcS83PEf=MJP8brwsdis+7TQ=oLkO4RNTX9g@mail.gmail.com>
 <30231bb2-50e0-3880-b705-942b263eafe8@grsecurity.net>
 <CAHmME9rT+aR-FJKFxVaW+tsACKX0kH-2T6cw7xbjjaeVsJbybw@mail.gmail.com>
 <1554725710.1290070.1639240504281.JavaMail.zimbra@inria.fr>
 <CAHmME9ppTs-jmmV9uV+C0jbtQC5a5RWO5zJB5_-dRD3BMjDZZA@mail.gmail.com>
 <d1921390-7fcb-a627-11ea-ce580214db63@grsecurity.net>
Message-ID: <CAHmME9r8udYyD+R9PPyN15G4QKsm45ZH_UbYXAjXXDwuvvob=g@mail.gmail.com>

Hi Mathias,

On Mon, Dec 13, 2021 at 8:54 AM Mathias Krause <minipli at grsecurity.net> wrote:
> So, after importing Aymeric's changes into wireguard-linux-compat, the
> alternatives change is still needed. I can help you create one, if you
> want me to (we have that change in grsec anyway, just used named asm
> operands here, as the %0/%1/%2/... was rather confusing and hard to follow).

Sure, I'll take a patch for that. Let's wait until Aymeric
incorporates your suggestion on the out param, merges the PR, and then
I'll reimport that to wireguard-linux-compat, and you can send a
patch.

Jason

From Jason at zx2c4.com  Mon Dec 13 14:10:30 2021
From: Jason at zx2c4.com (Jason A. Donenfeld)
Date: Mon, 13 Dec 2021 15:10:30 +0100
Subject: final call for Ubuntu 14.04 WireGuard support
In-Reply-To: <CAHmME9qVyk-7xou1K=97xdrg4UAA3W8YO6q6jevWc6YY6aTxSg@mail.gmail.com>
References: <CAHmME9rckipsdZYW+LA=x6wCMybdFFA+VqoogFXnR=kHYiCteg@mail.gmail.com>
 <CAHmME9qVyk-7xou1K=97xdrg4UAA3W8YO6q6jevWc6YY6aTxSg@mail.gmail.com>
Message-ID: <CAHmME9qVBWPxHEwBOJ7iJ90ONxGF7B7geMq3GT2YWEEf8Gq+cA@mail.gmail.com>

On Wed, Oct 7, 2020 at 1:23 PM Jason A. Donenfeld <Jason at zx2c4.com> wrote:
>
> An update on this thread: with dwindling usage and the poor quality of
> Canonical's ESM offering, 14.04 support has been sunset and the PPA
> removed.
>
> All users should update to a newer distro.

Over a year later, the vestiges of support have now been removed from
the repo: https://w-g.pw/l/kup9

Regards,
Jason

From aymeric.fromherz at inria.fr  Mon Dec 13 14:20:20 2021
From: aymeric.fromherz at inria.fr (Aymeric Fromherz)
Date: Mon, 13 Dec 2021 15:20:20 +0100 (CET)
Subject: [PATCH 0/2] wireguard-linux-compat: grsecurity compat patches
In-Reply-To: <3a1be638-4ddd-44c0-87aa-56cd26f4f396@grsecurity.net>
References: <20210706132714.8220-1-minipli@grsecurity.net>
 <CAHmME9odv1V9wUrLsX16J68wVD3rq+op0VvaNxtsLGG8Rrz4tQ@mail.gmail.com>
 <3ed9270f-00da-5c88-cadd-59b4419b9a98@grsecurity.net>
 <CAHmME9rFHkydmrYcS83PEf=MJP8brwsdis+7TQ=oLkO4RNTX9g@mail.gmail.com>
 <30231bb2-50e0-3880-b705-942b263eafe8@grsecurity.net>
 <CAHmME9rT+aR-FJKFxVaW+tsACKX0kH-2T6cw7xbjjaeVsJbybw@mail.gmail.com>
 <1554725710.1290070.1639240504281.JavaMail.zimbra@inria.fr>
 <3a1be638-4ddd-44c0-87aa-56cd26f4f396@grsecurity.net>
Message-ID: <2026222873.1815432.1639405220767.JavaMail.zimbra@inria.fr>

Hi Mathias,

Thanks for the comments. Unfortunately, changing "r" to "rm" for the out parameter in a systematic way is quite trick.
Allowing arguments to be passed independently in a register or in memory is currently out of scope of the Vale model we use for verification.
We must decide early on in the verification pipeline if the argument is passed in a register, or if it stored in memory.
Doing this in a sound way would require a significant change to our (trusted) model.

Cheers,
Aymeric

----- Mail original -----
> De: "Mathias Krause" <minipli at grsecurity.net>
> ?: "Aymeric Fromherz" <aymeric.fromherz at inria.fr>, "Jason A. Donenfeld" <Jason at zx2c4.com>
> Cc: "WireGuard mailing list" <wireguard at lists.zx2c4.com>
> Envoy?: Lundi 13 D?cembre 2021 08:44:51
> Objet: Re: [PATCH 0/2] wireguard-linux-compat: grsecurity compat patches

> Hi Aymeric,
> 
> the changes look good to me -- quite what we already had in grsec. Just
> one more nit. The constraints for the 'out' parameter in fmul(),
> fmul2(), fsqr() and fsqr2() can be further lifted to "rm" as 'out' is
> only referenced once. This allows gcc to choose either a register or a
> memory operand, as it sees fit. In our experiments the latter lead to
> much better code gen.
> 
> Thanks,
> Mathias
> 
> Am 11.12.21 um 17:35 schrieb Aymeric Fromherz:
>> Thanks for the heads-up. We were being overly conservative during verification
>> of inline assembly code in Vale, and marked several registers as possibly
>> modified while they were only read.
>> 
>> This is now fixed for fmul, fmul2, fsqr and fsqr2, and will be merged into the
>> master branch of EverCrypt shortly.
>> In the meantime, the diff for the resulting inline assembly after Vale codegen
>> is available here:
>> https://github.com/project-everest/hacl-star/pull/501/commits/1a71adb40c3f78da16e16975dbb1d4de5adeab8c#diff-5aabe9f6aa87508c9d81d4c9e89eff0b06b1e2aeaf5b04eba51da71c5bea6940
>> 
>> Cheers,
>> Aymeric
>> 
>> ----- Mail original -----
>>> De: "Jason A. Donenfeld" <Jason at zx2c4.com>
>>> ?: "Mathias Krause" <minipli at grsecurity.net>, "aymeric fromherz"
>>> <aymeric.fromherz at inria.fr>
>>> Cc: "WireGuard mailing list" <wireguard at lists.zx2c4.com>
>>> Envoy?: Vendredi 10 D?cembre 2021 23:58:01
>>> Objet: Re: [PATCH 0/2] wireguard-linux-compat: grsecurity compat patches
>> 
>>> CC'ing in Aymeric, who's working on Vale's codegen.
>>>
>>> On Thu, Dec 9, 2021 at 8:59 AM Mathias Krause <minipli at grsecurity.net> wrote:
>>>>
>>>> Am 08.12.21 um 15:56 schrieb Jason A. Donenfeld:
>>>>> On Mon, Dec 6, 2021 at 10:00 PM Mathias Krause <minipli at grsecurity.net> wrote:
>>>>>> Yes, probably, but you're mixing up the two.
>>>>>
>>>>> Oh, thanks, right.
>>>>>
>>>>> I'll talk to EverCrypt upstream and see.
>>>>
>>>> FWIW, 'out' is also wrongly flagged as output operand in fmul() and
>>>> fmul2(). But making it an input operand needs more surgery, as the
>>>> operand order changes and this requires some code churn.
>>>>
> >>> Mathias

From minipli at grsecurity.net  Mon Dec 13 14:33:26 2021
From: minipli at grsecurity.net (Mathias Krause)
Date: Mon, 13 Dec 2021 15:33:26 +0100
Subject: [PATCH 0/2] wireguard-linux-compat: grsecurity compat patches
In-Reply-To: <2026222873.1815432.1639405220767.JavaMail.zimbra@inria.fr>
References: <20210706132714.8220-1-minipli@grsecurity.net>
 <CAHmME9odv1V9wUrLsX16J68wVD3rq+op0VvaNxtsLGG8Rrz4tQ@mail.gmail.com>
 <3ed9270f-00da-5c88-cadd-59b4419b9a98@grsecurity.net>
 <CAHmME9rFHkydmrYcS83PEf=MJP8brwsdis+7TQ=oLkO4RNTX9g@mail.gmail.com>
 <30231bb2-50e0-3880-b705-942b263eafe8@grsecurity.net>
 <CAHmME9rT+aR-FJKFxVaW+tsACKX0kH-2T6cw7xbjjaeVsJbybw@mail.gmail.com>
 <1554725710.1290070.1639240504281.JavaMail.zimbra@inria.fr>
 <3a1be638-4ddd-44c0-87aa-56cd26f4f396@grsecurity.net>
 <2026222873.1815432.1639405220767.JavaMail.zimbra@inria.fr>
Message-ID: <fa249625-2698-3890-7e39-9273072810b9@grsecurity.net>

Hi Aymeric,

yeah, don't worry. We can keep this change downstream in grsec. Dunno if
Jason wants to take it as well, his call. It was just a side observation
which came out of our need to read and understand the code to provide a
workaround for the gcc bug we were hitting.

Thanks,
Mathias

Am 13.12.21 um 15:20 schrieb Aymeric Fromherz:
> Hi Mathias,
> 
> Thanks for the comments. Unfortunately, changing "r" to "rm" for the out parameter in a systematic way is quite trick.
> Allowing arguments to be passed independently in a register or in memory is currently out of scope of the Vale model we use for verification.
> We must decide early on in the verification pipeline if the argument is passed in a register, or if it stored in memory.
> Doing this in a sound way would require a significant change to our (trusted) model.
> 
> Cheers,
> Aymeric
> 
> ----- Mail original -----
>> De: "Mathias Krause" <minipli at grsecurity.net>
>> ?: "Aymeric Fromherz" <aymeric.fromherz at inria.fr>, "Jason A. Donenfeld" <Jason at zx2c4.com>
>> Cc: "WireGuard mailing list" <wireguard at lists.zx2c4.com>
>> Envoy?: Lundi 13 D?cembre 2021 08:44:51
>> Objet: Re: [PATCH 0/2] wireguard-linux-compat: grsecurity compat patches
> 
>> Hi Aymeric,
>>
>> the changes look good to me -- quite what we already had in grsec. Just
>> one more nit. The constraints for the 'out' parameter in fmul(),
>> fmul2(), fsqr() and fsqr2() can be further lifted to "rm" as 'out' is
>> only referenced once. This allows gcc to choose either a register or a
>> memory operand, as it sees fit. In our experiments the latter lead to
>> much better code gen.
>>
>> Thanks,
>> Mathias
>>
>> Am 11.12.21 um 17:35 schrieb Aymeric Fromherz:
>>> Thanks for the heads-up. We were being overly conservative during verification
>>> of inline assembly code in Vale, and marked several registers as possibly
>>> modified while they were only read.
>>>
>>> This is now fixed for fmul, fmul2, fsqr and fsqr2, and will be merged into the
>>> master branch of EverCrypt shortly.
>>> In the meantime, the diff for the resulting inline assembly after Vale codegen
>>> is available here:
>>> https://github.com/project-everest/hacl-star/pull/501/commits/1a71adb40c3f78da16e16975dbb1d4de5adeab8c#diff-5aabe9f6aa87508c9d81d4c9e89eff0b06b1e2aeaf5b04eba51da71c5bea6940
>>>
>>> Cheers,
>>> Aymeric
>>>
>>> ----- Mail original -----
>>>> De: "Jason A. Donenfeld" <Jason at zx2c4.com>
>>>> ?: "Mathias Krause" <minipli at grsecurity.net>, "aymeric fromherz"
>>>> <aymeric.fromherz at inria.fr>
>>>> Cc: "WireGuard mailing list" <wireguard at lists.zx2c4.com>
>>>> Envoy?: Vendredi 10 D?cembre 2021 23:58:01
>>>> Objet: Re: [PATCH 0/2] wireguard-linux-compat: grsecurity compat patches
>>>
>>>> CC'ing in Aymeric, who's working on Vale's codegen.
>>>>
>>>> On Thu, Dec 9, 2021 at 8:59 AM Mathias Krause <minipli at grsecurity.net> wrote:
>>>>>
>>>>> Am 08.12.21 um 15:56 schrieb Jason A. Donenfeld:
>>>>>> On Mon, Dec 6, 2021 at 10:00 PM Mathias Krause <minipli at grsecurity.net> wrote:
>>>>>>> Yes, probably, but you're mixing up the two.
>>>>>>
>>>>>> Oh, thanks, right.
>>>>>>
>>>>>> I'll talk to EverCrypt upstream and see.
>>>>>
>>>>> FWIW, 'out' is also wrongly flagged as output operand in fmul() and
>>>>> fmul2(). But making it an input operand needs more surgery, as the
>>>>> operand order changes and this requires some code churn.
>>>>>
>>>>> Mathias

From Jason at zx2c4.com  Mon Dec 13 14:37:50 2021
From: Jason at zx2c4.com (Jason A. Donenfeld)
Date: Mon, 13 Dec 2021 15:37:50 +0100
Subject: [PATCH 0/2] wireguard-linux-compat: grsecurity compat patches
In-Reply-To: <fa249625-2698-3890-7e39-9273072810b9@grsecurity.net>
References: <20210706132714.8220-1-minipli@grsecurity.net>
 <CAHmME9odv1V9wUrLsX16J68wVD3rq+op0VvaNxtsLGG8Rrz4tQ@mail.gmail.com>
 <3ed9270f-00da-5c88-cadd-59b4419b9a98@grsecurity.net>
 <CAHmME9rFHkydmrYcS83PEf=MJP8brwsdis+7TQ=oLkO4RNTX9g@mail.gmail.com>
 <30231bb2-50e0-3880-b705-942b263eafe8@grsecurity.net>
 <CAHmME9rT+aR-FJKFxVaW+tsACKX0kH-2T6cw7xbjjaeVsJbybw@mail.gmail.com>
 <1554725710.1290070.1639240504281.JavaMail.zimbra@inria.fr>
 <3a1be638-4ddd-44c0-87aa-56cd26f4f396@grsecurity.net>
 <2026222873.1815432.1639405220767.JavaMail.zimbra@inria.fr>
 <fa249625-2698-3890-7e39-9273072810b9@grsecurity.net>
Message-ID: <CAHmME9pC=eVFq-4eiU6iFVGOqZfr7RgniAP7H5YGBXDqc7W9jg@mail.gmail.com>

On Mon, Dec 13, 2021 at 3:33 PM Mathias Krause <minipli at grsecurity.net> wrote:
> yeah, don't worry. We can keep this change downstream in grsec. Dunno if
> Jason wants to take it as well, his call. It was just a side observation
> which came out of our need to read and understand the code to provide a
> workaround for the gcc bug we were hitting.

I suppose I can load it up in kbench9000 to see if it makes a
difference. If it doesn't matter much, I'd prefer sticking with the
formally verified stuff. But if there is a nice speedup, then I guess
we can revisit more handwavy "obviously this is the same thing"
arguments.

Jason

From Jason at zx2c4.com  Mon Dec 13 16:29:19 2021
From: Jason at zx2c4.com (Jason A. Donenfeld)
Date: Mon, 13 Dec 2021 17:29:19 +0100
Subject: [PATCH 0/2] wireguard-linux-compat: grsecurity compat patches
In-Reply-To: <CAHmME9r8udYyD+R9PPyN15G4QKsm45ZH_UbYXAjXXDwuvvob=g@mail.gmail.com>
References: <20210706132714.8220-1-minipli@grsecurity.net>
 <CAHmME9opYoaoSTHud9sF4BOFBPS=NVYPJSGQr37bxNai26MGFg@mail.gmail.com>
 <1ef9bbf4-b8f8-18df-3c80-3181c2b5b9c5@grsecurity.net>
 <CAHmME9odv1V9wUrLsX16J68wVD3rq+op0VvaNxtsLGG8Rrz4tQ@mail.gmail.com>
 <3ed9270f-00da-5c88-cadd-59b4419b9a98@grsecurity.net>
 <CAHmME9rFHkydmrYcS83PEf=MJP8brwsdis+7TQ=oLkO4RNTX9g@mail.gmail.com>
 <30231bb2-50e0-3880-b705-942b263eafe8@grsecurity.net>
 <CAHmME9rT+aR-FJKFxVaW+tsACKX0kH-2T6cw7xbjjaeVsJbybw@mail.gmail.com>
 <1554725710.1290070.1639240504281.JavaMail.zimbra@inria.fr>
 <CAHmME9ppTs-jmmV9uV+C0jbtQC5a5RWO5zJB5_-dRD3BMjDZZA@mail.gmail.com>
 <d1921390-7fcb-a627-11ea-ce580214db63@grsecurity.net>
 <CAHmME9r8udYyD+R9PPyN15G4QKsm45ZH_UbYXAjXXDwuvvob=g@mail.gmail.com>
Message-ID: <CAHmME9qJottk_hHxjPKF=5Ls2Ux5=9YF6pdy1pwbS3L4UKZbcw@mail.gmail.com>

On Mon, Dec 13, 2021 at 12:36 PM Jason A. Donenfeld <Jason at zx2c4.com> wrote:
>
> Hi Mathias,
>
> On Mon, Dec 13, 2021 at 8:54 AM Mathias Krause <minipli at grsecurity.net> wrote:
> > So, after importing Aymeric's changes into wireguard-linux-compat, the
> > alternatives change is still needed. I can help you create one, if you
> > want me to (we have that change in grsec anyway, just used named asm
> > operands here, as the %0/%1/%2/... was rather confusing and hard to follow).
>
> Sure, I'll take a patch for that. Let's wait until Aymeric
> incorporates your suggestion on the out param, merges the PR, and then
> I'll reimport that to wireguard-linux-compat, and you can send a
> patch.

Actually, wasn't so bad, did it myself: https://w-g.pw/l/LjOY
Hopefully nothing to butchered in the process.

From minipli at grsecurity.net  Mon Dec 13 16:32:07 2021
From: minipli at grsecurity.net (Mathias Krause)
Date: Mon, 13 Dec 2021 17:32:07 +0100
Subject: [PATCH 0/2] wireguard-linux-compat: grsecurity compat patches
In-Reply-To: <CAHmME9pC=eVFq-4eiU6iFVGOqZfr7RgniAP7H5YGBXDqc7W9jg@mail.gmail.com>
References: <20210706132714.8220-1-minipli@grsecurity.net>
 <CAHmME9odv1V9wUrLsX16J68wVD3rq+op0VvaNxtsLGG8Rrz4tQ@mail.gmail.com>
 <3ed9270f-00da-5c88-cadd-59b4419b9a98@grsecurity.net>
 <CAHmME9rFHkydmrYcS83PEf=MJP8brwsdis+7TQ=oLkO4RNTX9g@mail.gmail.com>
 <30231bb2-50e0-3880-b705-942b263eafe8@grsecurity.net>
 <CAHmME9rT+aR-FJKFxVaW+tsACKX0kH-2T6cw7xbjjaeVsJbybw@mail.gmail.com>
 <1554725710.1290070.1639240504281.JavaMail.zimbra@inria.fr>
 <3a1be638-4ddd-44c0-87aa-56cd26f4f396@grsecurity.net>
 <2026222873.1815432.1639405220767.JavaMail.zimbra@inria.fr>
 <fa249625-2698-3890-7e39-9273072810b9@grsecurity.net>
 <CAHmME9pC=eVFq-4eiU6iFVGOqZfr7RgniAP7H5YGBXDqc7W9jg@mail.gmail.com>
Message-ID: <ffb81284-53e4-dc94-75ec-d31b6a79f269@grsecurity.net>

Am 13.12.21 um 15:37 schrieb Jason A. Donenfeld:
> On Mon, Dec 13, 2021 at 3:33 PM Mathias Krause <minipli at grsecurity.net> wrote:
>> yeah, don't worry. We can keep this change downstream in grsec. Dunno if
>> Jason wants to take it as well, his call. It was just a side observation
>> which came out of our need to read and understand the code to provide a
>> workaround for the gcc bug we were hitting.
> 
> I suppose I can load it up in kbench9000 to see if it makes a
> difference. If it doesn't matter much, I'd prefer sticking with the
> formally verified stuff. But if there is a nice speedup, then I guess
> we can revisit more handwavy "obviously this is the same thing"
> arguments.

Had to hack main.c and run.sh a little, but here are some numbers:

root at box:~# ./run.sh
[+] Setting no-turbo to status 1
[+] Setting non-boot CPUs to status 0
[+] Inserting module to run tests
insmod: ERROR: could not insert module kbench9000.ko: Unknown symbol in
module
[+] Gathering results
                 ever64: 115100 cycles per call
           ever64_out_r: 115080 cycles per call
          ever64_out_rm: 113957 cycles per call
[+] Setting non-boot CPUs to status 1
[+] Setting no-turbo to status 0

Slightly faster.

Mathias

From Jason at zx2c4.com  Mon Dec 13 16:33:57 2021
From: Jason at zx2c4.com (Jason A. Donenfeld)
Date: Mon, 13 Dec 2021 17:33:57 +0100
Subject: [PATCH 0/2] wireguard-linux-compat: grsecurity compat patches
In-Reply-To: <ffb81284-53e4-dc94-75ec-d31b6a79f269@grsecurity.net>
References: <20210706132714.8220-1-minipli@grsecurity.net>
 <CAHmME9odv1V9wUrLsX16J68wVD3rq+op0VvaNxtsLGG8Rrz4tQ@mail.gmail.com>
 <3ed9270f-00da-5c88-cadd-59b4419b9a98@grsecurity.net>
 <CAHmME9rFHkydmrYcS83PEf=MJP8brwsdis+7TQ=oLkO4RNTX9g@mail.gmail.com>
 <30231bb2-50e0-3880-b705-942b263eafe8@grsecurity.net>
 <CAHmME9rT+aR-FJKFxVaW+tsACKX0kH-2T6cw7xbjjaeVsJbybw@mail.gmail.com>
 <1554725710.1290070.1639240504281.JavaMail.zimbra@inria.fr>
 <3a1be638-4ddd-44c0-87aa-56cd26f4f396@grsecurity.net>
 <2026222873.1815432.1639405220767.JavaMail.zimbra@inria.fr>
 <fa249625-2698-3890-7e39-9273072810b9@grsecurity.net>
 <CAHmME9pC=eVFq-4eiU6iFVGOqZfr7RgniAP7H5YGBXDqc7W9jg@mail.gmail.com>
 <ffb81284-53e4-dc94-75ec-d31b6a79f269@grsecurity.net>
Message-ID: <CAHmME9oh4Zx01E-+5S3Ds6mj6tVhJj81sTnSreDAujaFSX1u_Q@mail.gmail.com>

Hi Mathias,

On Mon, Dec 13, 2021 at 5:32 PM Mathias Krause <minipli at grsecurity.net> wrote:
>
> Am 13.12.21 um 15:37 schrieb Jason A. Donenfeld:
> > On Mon, Dec 13, 2021 at 3:33 PM Mathias Krause <minipli at grsecurity.net> wrote:
> >> yeah, don't worry. We can keep this change downstream in grsec. Dunno if
> >> Jason wants to take it as well, his call. It was just a side observation
> >> which came out of our need to read and understand the code to provide a
> >> workaround for the gcc bug we were hitting.
> >
> > I suppose I can load it up in kbench9000 to see if it makes a
> > difference. If it doesn't matter much, I'd prefer sticking with the
> > formally verified stuff. But if there is a nice speedup, then I guess
> > we can revisit more handwavy "obviously this is the same thing"
> > arguments.
>
> Had to hack main.c and run.sh a little, but here are some numbers:
>
> root at box:~# ./run.sh
> [+] Setting no-turbo to status 1
> [+] Setting non-boot CPUs to status 0
> [+] Inserting module to run tests
> insmod: ERROR: could not insert module kbench9000.ko: Unknown symbol in
> module
> [+] Gathering results
>                  ever64: 115100 cycles per call
>            ever64_out_r: 115080 cycles per call
>           ever64_out_rm: 113957 cycles per call
> [+] Setting non-boot CPUs to status 1
> [+] Setting no-turbo to status 0
>
> Slightly faster.

Huh, that's actually a pretty nice speedup for just changing register
allocation... What CPU is this?

Jason

From minipli at grsecurity.net  Mon Dec 13 16:39:26 2021
From: minipli at grsecurity.net (Mathias Krause)
Date: Mon, 13 Dec 2021 17:39:26 +0100
Subject: [PATCH 0/2] wireguard-linux-compat: grsecurity compat patches
In-Reply-To: <CAHmME9oh4Zx01E-+5S3Ds6mj6tVhJj81sTnSreDAujaFSX1u_Q@mail.gmail.com>
References: <20210706132714.8220-1-minipli@grsecurity.net>
 <CAHmME9odv1V9wUrLsX16J68wVD3rq+op0VvaNxtsLGG8Rrz4tQ@mail.gmail.com>
 <3ed9270f-00da-5c88-cadd-59b4419b9a98@grsecurity.net>
 <CAHmME9rFHkydmrYcS83PEf=MJP8brwsdis+7TQ=oLkO4RNTX9g@mail.gmail.com>
 <30231bb2-50e0-3880-b705-942b263eafe8@grsecurity.net>
 <CAHmME9rT+aR-FJKFxVaW+tsACKX0kH-2T6cw7xbjjaeVsJbybw@mail.gmail.com>
 <1554725710.1290070.1639240504281.JavaMail.zimbra@inria.fr>
 <3a1be638-4ddd-44c0-87aa-56cd26f4f396@grsecurity.net>
 <2026222873.1815432.1639405220767.JavaMail.zimbra@inria.fr>
 <fa249625-2698-3890-7e39-9273072810b9@grsecurity.net>
 <CAHmME9pC=eVFq-4eiU6iFVGOqZfr7RgniAP7H5YGBXDqc7W9jg@mail.gmail.com>
 <ffb81284-53e4-dc94-75ec-d31b6a79f269@grsecurity.net>
 <CAHmME9oh4Zx01E-+5S3Ds6mj6tVhJj81sTnSreDAujaFSX1u_Q@mail.gmail.com>
Message-ID: <d52c29d2-3530-e2e9-7096-77da1eb21078@grsecurity.net>

Am 13.12.21 um 17:33 schrieb Jason A. Donenfeld:
>> root at box:~# ./run.sh
>> [+] Setting no-turbo to status 1
>> [+] Setting non-boot CPUs to status 0
>> [+] Inserting module to run tests
>> insmod: ERROR: could not insert module kbench9000.ko: Unknown symbol in
>> module
>> [+] Gathering results
>>                  ever64: 115100 cycles per call
>>            ever64_out_r: 115080 cycles per call
>>           ever64_out_rm: 113957 cycles per call
>> [+] Setting non-boot CPUs to status 1
>> [+] Setting no-turbo to status 0
>>
>> Slightly faster.
> 
> Huh, that's actually a pretty nice speedup for just changing register
> allocation... What CPU is this?

It's an "Intel(R) Core(TM) i9-9900 CPU @ 3.10GHz" But shouldn't matter
all that much, as the inline ASM is integer ops only.

Mathias

From minipli at grsecurity.net  Mon Dec 13 16:46:55 2021
From: minipli at grsecurity.net (Mathias Krause)
Date: Mon, 13 Dec 2021 17:46:55 +0100
Subject: [PATCH 0/2] wireguard-linux-compat: grsecurity compat patches
In-Reply-To: <CAHmME9qJottk_hHxjPKF=5Ls2Ux5=9YF6pdy1pwbS3L4UKZbcw@mail.gmail.com>
References: <20210706132714.8220-1-minipli@grsecurity.net>
 <CAHmME9opYoaoSTHud9sF4BOFBPS=NVYPJSGQr37bxNai26MGFg@mail.gmail.com>
 <1ef9bbf4-b8f8-18df-3c80-3181c2b5b9c5@grsecurity.net>
 <CAHmME9odv1V9wUrLsX16J68wVD3rq+op0VvaNxtsLGG8Rrz4tQ@mail.gmail.com>
 <3ed9270f-00da-5c88-cadd-59b4419b9a98@grsecurity.net>
 <CAHmME9rFHkydmrYcS83PEf=MJP8brwsdis+7TQ=oLkO4RNTX9g@mail.gmail.com>
 <30231bb2-50e0-3880-b705-942b263eafe8@grsecurity.net>
 <CAHmME9rT+aR-FJKFxVaW+tsACKX0kH-2T6cw7xbjjaeVsJbybw@mail.gmail.com>
 <1554725710.1290070.1639240504281.JavaMail.zimbra@inria.fr>
 <CAHmME9ppTs-jmmV9uV+C0jbtQC5a5RWO5zJB5_-dRD3BMjDZZA@mail.gmail.com>
 <d1921390-7fcb-a627-11ea-ce580214db63@grsecurity.net>
 <CAHmME9r8udYyD+R9PPyN15G4QKsm45ZH_UbYXAjXXDwuvvob=g@mail.gmail.com>
 <CAHmME9qJottk_hHxjPKF=5Ls2Ux5=9YF6pdy1pwbS3L4UKZbcw@mail.gmail.com>
Message-ID: <13edfd3f-7645-085e-df17-836732225613@grsecurity.net>



Am 13.12.21 um 17:29 schrieb Jason A. Donenfeld:
> On Mon, Dec 13, 2021 at 12:36 PM Jason A. Donenfeld <Jason at zx2c4.com> wrote:
>>
>> Hi Mathias,
>>
>> On Mon, Dec 13, 2021 at 8:54 AM Mathias Krause <minipli at grsecurity.net> wrote:
>>> So, after importing Aymeric's changes into wireguard-linux-compat, the
>>> alternatives change is still needed. I can help you create one, if you
>>> want me to (we have that change in grsec anyway, just used named asm
>>> operands here, as the %0/%1/%2/... was rather confusing and hard to follow).
>>
>> Sure, I'll take a patch for that. Let's wait until Aymeric
>> incorporates your suggestion on the out param, merges the PR, and then
>> I'll reimport that to wireguard-linux-compat, and you can send a
>> patch.
> 
> Actually, wasn't so bad, did it myself: https://w-g.pw/l/LjOY
> Hopefully nothing to butchered in the process.

Looks good to me. Actually had to fetch the repo and use `git show
--color-words -w` to get a sensible signal-to-noise ratio ;)

Mathias

From Jason at zx2c4.com  Mon Dec 13 16:53:26 2021
From: Jason at zx2c4.com (Jason A. Donenfeld)
Date: Mon, 13 Dec 2021 17:53:26 +0100
Subject: [PATCH 0/2] wireguard-linux-compat: grsecurity compat patches
In-Reply-To: <d52c29d2-3530-e2e9-7096-77da1eb21078@grsecurity.net>
References: <20210706132714.8220-1-minipli@grsecurity.net>
 <CAHmME9odv1V9wUrLsX16J68wVD3rq+op0VvaNxtsLGG8Rrz4tQ@mail.gmail.com>
 <3ed9270f-00da-5c88-cadd-59b4419b9a98@grsecurity.net>
 <CAHmME9rFHkydmrYcS83PEf=MJP8brwsdis+7TQ=oLkO4RNTX9g@mail.gmail.com>
 <30231bb2-50e0-3880-b705-942b263eafe8@grsecurity.net>
 <CAHmME9rT+aR-FJKFxVaW+tsACKX0kH-2T6cw7xbjjaeVsJbybw@mail.gmail.com>
 <1554725710.1290070.1639240504281.JavaMail.zimbra@inria.fr>
 <3a1be638-4ddd-44c0-87aa-56cd26f4f396@grsecurity.net>
 <2026222873.1815432.1639405220767.JavaMail.zimbra@inria.fr>
 <fa249625-2698-3890-7e39-9273072810b9@grsecurity.net>
 <CAHmME9pC=eVFq-4eiU6iFVGOqZfr7RgniAP7H5YGBXDqc7W9jg@mail.gmail.com>
 <ffb81284-53e4-dc94-75ec-d31b6a79f269@grsecurity.net>
 <CAHmME9oh4Zx01E-+5S3Ds6mj6tVhJj81sTnSreDAujaFSX1u_Q@mail.gmail.com>
 <d52c29d2-3530-e2e9-7096-77da1eb21078@grsecurity.net>
Message-ID: <CAHmME9qhcdAZSzcWQ3fSFix+_66jKkdvPvs-JeCQevJYde_HXA@mail.gmail.com>

On Mon, Dec 13, 2021 at 5:39 PM Mathias Krause <minipli at grsecurity.net> wrote:
>
> Am 13.12.21 um 17:33 schrieb Jason A. Donenfeld:
> >> root at box:~# ./run.sh
> >> [+] Setting no-turbo to status 1
> >> [+] Setting non-boot CPUs to status 0
> >> [+] Inserting module to run tests
> >> insmod: ERROR: could not insert module kbench9000.ko: Unknown symbol in
> >> module
> >> [+] Gathering results
> >>                  ever64: 115100 cycles per call
> >>            ever64_out_r: 115080 cycles per call
> >>           ever64_out_rm: 113957 cycles per call
> >> [+] Setting non-boot CPUs to status 1
> >> [+] Setting no-turbo to status 0
> >>
> >> Slightly faster.
> >
> > Huh, that's actually a pretty nice speedup for just changing register
> > allocation... What CPU is this?
>
> It's an "Intel(R) Core(TM) i9-9900 CPU @ 3.10GHz" But shouldn't matter
> all that much, as the inline ASM is integer ops only.

Yet it does. Here are results on a tigerlake, i7-11850H:

[+] Setting no-turbo to status 1
[+] Setting non-boot CPUs to status 0
[+] Inserting module to run tests
[+] Gathering results
    donna64: 141081 cycles per call
     hacl64: 135645 cycles per call
     fiat64: 132547 cycles per call
    sandy2x: 126933 cycles per call
precomp_bmi2: 120276 cycles per call
precomp_adx: 113578 cycles per call
     ever64: 106943 cycles per call
    ever64r: 105924 cycles per call
   ever64rm: 108139 cycles per call
     fiat32: 297854 cycles per call
    donna32: 467667 cycles per call
  tweetnacl: 1195385 cycles per call
[+] Setting non-boot CPUs to status 1
[+] Setting no-turbo to status 0

https://xn--4db.cc/c4B65RUy/diff

Looks like Tigerlake is able to make use of free stack accesses thanks
to the new "fast forwarding" feature.

From antonio.teixeira at bry.com.br  Tue Dec 14 17:15:17 2021
From: antonio.teixeira at bry.com.br (Antonio Teixeira)
Date: Tue, 14 Dec 2021 14:15:17 -0300
Subject: Running PTP (Precision Time Protocol) over Wireguard
Message-ID: <CACD1g0AfxsGNdO9e-fRLUvONfPJXJnLbzGjQb-s8t+iizXdB6A@mail.gmail.com>

Hey,

Has anyone ever got PTP (precision time protocol) working with
hardware timestamping over a wireguard VPN?

I need to start linuxptp (http://linuxptp.sourceforge.net/) listening
on a Wireguard virtual network interface, however, it seems that
linuxptp relies on the SO_TIMESTAMPING kernel socket option and the
wireguard virtual interface doesn't support that. Moreover, I need
this working with hardware timestamping, and right now I can't even
get it working with software timestamping.

Would it even be possible to get this working? Anyone got any tips?
Can I somehow make the Wireguard virtual interface mirror the physical
NIC's capabilities?

From social.nst at mailbox.org  Thu Dec 16 19:27:36 2021
From: social.nst at mailbox.org (Nils)
Date: Thu, 16 Dec 2021 20:27:36 +0100
Subject: macOS socks5 system proxy not working
In-Reply-To: <C88C9563-32AD-4777-80BC-32B4F9244644@mailbox.org>
References: <C88C9563-32AD-4777-80BC-32B4F9244644@mailbox.org>
Message-ID: <8891729E-6370-40F5-8EDF-A363A94E154B@mailbox.org>

Hello,
I resonantly figured out that my socks5 system proxy is working with the VPN Providers App (using wireguard), but not when using the WireGuard App (App Store). 

When using the WireGuard App the system socks5 proxy seems to be ignored by all applications.
My VPN Provider is aware of this Issue and recommends using his App, they think it has something to do with the way the WireGuard App adds the tunnels to the macOS network settings panel.

Is this a know issue?

I'm on running WireGuard on Apple M1 Pro macOS 12.1

Cheers,
Nils

From wireguard at centromere.net  Sat Dec 18 00:06:59 2021
From: wireguard at centromere.net (Alex)
Date: Fri, 17 Dec 2021 19:06:59 -0500
Subject: eBPF + IPv6 + WireGuard
Message-ID: <20211217190659.251f006a@poseidon.quill.lan>

Hi all,

I am championing WireGuard at work, and I have been granted permission
to use it for establishing remote access to a private IPv6 VLAN for all
employees. I have experimented with different approaches and ran in to
an issue.

With the help of a machine dedicated fully to the job of remote access
(Ubuntu 20.04 / Linux 5.4), I've managed to establish end-to-end
connectivity between my work laptop and the servers. The VPN gateway
has a wg0 interface for employees and a "private0" interface for
private VLAN connectivity. The goal is to link the two.

I noticed an odd quirk: It only works when
"net.ipv6.conf.all.forwarding" is 1. If this value is 0,
"net.ipv6.conf.[wg0 | private0].forwarding" has no effect. In other
words, I cannot seem to enable forwarding for *only* the interfaces
that need it. It only works when forwarding is enabled for *all*
interfaces. This is a problem because when the "all" value is set to 1,
the machine will start to behave as a router on VLANs for which it is
most definitely not a router.

For the servers, I am using the officially defined[0] subnet-router
anycast address as the default IPv6 gateway, and it works well.
However, when I flipped the switch on "net.ipv6.conf.all.forwarding",
the VPN gateway started using NDP to announce that it was a router
across *all* VLANs! Specifically, it was claiming ownership over our
global subnet anycast address 2001:aaaa:bbbb:cccc::. This is neither
true nor desired. The VPN gateway started to receive outbound non-VPN
Internet traffic which broke Internet connectivity for all servers. 

This leads me to my first question: Why does
"net.ipv6.conf.wg0.forwarding" have no effect?

In researching a solution, I decided to give eBPF a try. I attached an
XDP program to "private0" (a layer 2 device, naturally) and
successfully redirected packets to "wg0" (a layer 3 device) with
bpf_redirect[1]. If the packets are unmodified, WireGuard will happily
pass them to the remote hosts. This is an issue because the remote
hosts are expecting to receive IP(v6) packets, not Ethernet frames. If
I use bpf_xdp_adjust_head[1] to strip off the Ethernet frame, WireGuard
will drop the packet[2] before it can be sent to the remote host.

My theory is that bpf_xdp_adjust_head is modifying the data pointer
only and not any underlying structures that may be associated with the
packet (sk_buff perhaps).

This leads me to my second question: Why can't I redirect traffic
received on an L2 interface to an L3 interface, *even after stripping
off the Ethernet frame?*

Finally, during this whole process I was using WireShark to inspect
traffic received by the remote host (i.e. my work laptop). With the
help of the extract-handshakes.sh script, I was able to decrypt traffic.
I did discover a bug though. I believe "index_hashtable_insert" should
actually be "wg_index_hashtable_insert"[3].

Does anyone have any insights as to what a proper solution would look
like? Is there a way to achieve my goal without introducing eBPF? Is
XDP completely unsuited for this particular purpose? Do I actually need
to operate on the sk_buff level, as opposed to the xdp_buff level?

Thank you all for your time.

 - Alex

[0] https://datatracker.ietf.org/doc/rfc1884/ (Section 2.5.1)
[1] https://www.man7.org/linux/man-pages/man7/bpf-helpers.7.html
[2] https://git.zx2c4.com/wireguard-linux/tree/drivers/net/wireguard/device.c#n131
[3] https://git.zx2c4.com/wireguard-tools/tree/contrib/extract-handshakes/extract-handshakes.sh#n46

From Jason at zx2c4.com  Mon Dec 20 17:00:34 2021
From: Jason at zx2c4.com (Jason A. Donenfeld)
Date: Mon, 20 Dec 2021 18:00:34 +0100
Subject: Running PTP (Precision Time Protocol) over Wireguard
In-Reply-To: <CACD1g0AfxsGNdO9e-fRLUvONfPJXJnLbzGjQb-s8t+iizXdB6A@mail.gmail.com>
References: <CACD1g0AfxsGNdO9e-fRLUvONfPJXJnLbzGjQb-s8t+iizXdB6A@mail.gmail.com>
Message-ID: <CAHmME9pxjOO_0pQNprxdxRK9zPWk3dDKMrk6X4YWz-B7BvLncA@mail.gmail.com>

I've never used these APIs and I wasn't aware it was something that
needed explicit support inside of virtual drivers. What happens when
you try to use it? Are the timestamps bogus or does it return an
error? If the former, in what way are they bogus? If the latter,
what's the error?

Jason

From Jason at zx2c4.com  Mon Dec 20 17:29:49 2021
From: Jason at zx2c4.com (Jason A. Donenfeld)
Date: Mon, 20 Dec 2021 18:29:49 +0100
Subject: [RFC] wiregard RX packet processing.
In-Reply-To: <20211208173205.zajfvg6zvi4g5kln@linutronix.de>
References: <20211208173205.zajfvg6zvi4g5kln@linutronix.de>
Message-ID: <CAHmME9rzEjKg41eq5jBtsLXF+vZSEnvdomZJ-rTzx8Q=ac1ayg@mail.gmail.com>

Hi Sebastian,

Seems like you've identified two things, the use of need_resched, and
potentially surrounding napi_schedule in local_bh_{disable,enable}.

Regarding need_resched, I pulled that out of other code that seemed to
have the "same requirements", as vaguely conceived. It indeed might
not be right. The intent is to have that worker running at maximum
throughput for extended periods of time, but not preventing other
threads from running elsewhere, so that, e.g., a user's machine
doesn't have a jenky mouse when downloading a file.

What are the effects of unconditionally calling cond_resched() without
checking for if (need_resched())? Sounds like you're saying none at
all?

Regarding napi_schedule, I actually wasn't aware that it's requirement
to _only_ ever run from softirq was a strict one. When I switched to
using napi_schedule in this way, throughput really jumped up
significantly. Part of this indeed is from the batching, so that the
napi callback can then handle more packets in one go later. But I
assumed it was something inside of NAPI that was batching and
scheduling it, rather than a mistake on my part to call this from a wq
and not from a softirq.

What, then, are the effects of surrounding that in
local_bh_{disable,enable} as you've done in the patch? You mentioned
one aspect is that it will "invoke wg_packet_rx_poll() where you see
only one skb." It sounds like that'd be bad for performance, though,
given that the design of napi is really geared toward batching.

Jason

From toke at toke.dk  Thu Dec 23 15:25:20 2021
From: toke at toke.dk (Toke =?utf-8?Q?H=C3=B8iland-J=C3=B8rgensen?=)
Date: Thu, 23 Dec 2021 16:25:20 +0100
Subject: eBPF + IPv6 + WireGuard
In-Reply-To: <20211217190659.251f006a@poseidon.quill.lan>
References: <20211217190659.251f006a@poseidon.quill.lan>
Message-ID: <877dbvbaxr.fsf@toke.dk>

Alex <wireguard at centromere.net> writes:

> Hi all,
>
> I am championing WireGuard at work, and I have been granted permission
> to use it for establishing remote access to a private IPv6 VLAN for all
> employees. I have experimented with different approaches and ran in to
> an issue.
>
> With the help of a machine dedicated fully to the job of remote access
> (Ubuntu 20.04 / Linux 5.4), I've managed to establish end-to-end
> connectivity between my work laptop and the servers. The VPN gateway
> has a wg0 interface for employees and a "private0" interface for
> private VLAN connectivity. The goal is to link the two.
>
> I noticed an odd quirk: It only works when
> "net.ipv6.conf.all.forwarding" is 1. If this value is 0,
> "net.ipv6.conf.[wg0 | private0].forwarding" has no effect. In other
> words, I cannot seem to enable forwarding for *only* the interfaces
> that need it. It only works when forwarding is enabled for *all*
> interfaces. This is a problem because when the "all" value is set to 1,
> the machine will start to behave as a router on VLANs for which it is
> most definitely not a router.
>
> For the servers, I am using the officially defined[0] subnet-router
> anycast address as the default IPv6 gateway, and it works well.
> However, when I flipped the switch on "net.ipv6.conf.all.forwarding",
> the VPN gateway started using NDP to announce that it was a router
> across *all* VLANs! Specifically, it was claiming ownership over our
> global subnet anycast address 2001:aaaa:bbbb:cccc::. This is neither
> true nor desired. The VPN gateway started to receive outbound non-VPN
> Internet traffic which broke Internet connectivity for all servers. 
>
> This leads me to my first question: Why does
> "net.ipv6.conf.wg0.forwarding" have no effect?

That's how forwarding works in the kernel for IPv6:

https://elixir.bootlin.com/linux/latest/source/Documentation/networking/ip-sysctl.rst#L1981

I.e., there *is* no per-interface forwarding setting, you'll need to
setup netfilter rules or some other mechanism to control which
interfaces packets will be forwarded to/from. The per-interface
"forwarding" attribute only controls things like whether the kernel will
accept router advertisements on that interface. Yes, this is a bit
confusing (and also different from IPv4).

The isRouter flag in neighbour advertisements should be controllable via
the per-interface 'forwarding' setting; the default changes when you
enable the global setting, but I think it should be possible to re-set
it? See:

https://elixir.bootlin.com/linux/latest/source/Documentation/networking/ip-sysctl.rst#L2148

> In researching a solution, I decided to give eBPF a try. I attached an
> XDP program to "private0" (a layer 2 device, naturally) and
> successfully redirected packets to "wg0" (a layer 3 device) with
> bpf_redirect[1]. If the packets are unmodified, WireGuard will happily
> pass them to the remote hosts. This is an issue because the remote
> hosts are expecting to receive IP(v6) packets, not Ethernet frames. If
> I use bpf_xdp_adjust_head[1] to strip off the Ethernet frame, WireGuard
> will drop the packet[2] before it can be sent to the remote host.
>
> My theory is that bpf_xdp_adjust_head is modifying the data pointer
> only and not any underlying structures that may be associated with the
> packet (sk_buff perhaps).
>
> This leads me to my second question: Why can't I redirect traffic
> received on an L2 interface to an L3 interface, *even after stripping
> off the Ethernet frame?*

I assume you were using generic XDP here? Anyway, XDP is layer2-only by
definition, so there's not really any way to achieve what you're trying
to do with XDP.

> Finally, during this whole process I was using WireShark to inspect
> traffic received by the remote host (i.e. my work laptop). With the
> help of the extract-handshakes.sh script, I was able to decrypt traffic.
> I did discover a bug though. I believe "index_hashtable_insert" should
> actually be "wg_index_hashtable_insert"[3].
>
> Does anyone have any insights as to what a proper solution would look
> like? Is there a way to achieve my goal without introducing eBPF? Is
> XDP completely unsuited for this particular purpose? Do I actually need
> to operate on the sk_buff level, as opposed to the xdp_buff level?

You could maybe do something with TC-BPF, but why can't you just enable
net.ipv6.conf.all.forwarding and just use netfilter to control which
interfaces packets will be forwarded across? That seems like it would be
the simplest solution...

-Toke

From phillip.mcmahon at gmail.com  Wed Dec 29 08:48:20 2021
From: phillip.mcmahon at gmail.com (Phillip McMahon)
Date: Wed, 29 Dec 2021 09:48:20 +0100
Subject: Install Wireguard macos without having to use AppleID?
Message-ID: <CABtXGiBDnVhKg-EcwSnSmkQkf=1k8_vnXADLmrUbgjFefkqOLw@mail.gmail.com>

Background is that I have a new Macbook Pro for the M1 silicon and
video editing on the go. I have all my apps installed without having
to use an Apple ID (thanks to brew.sh) with the exception of
wireguard.

Is there any way outside of the app store to install?

-- 
Phillip McMahon

https://flowcrypt.com/attester/pub/phillip.mcmahon at gmail.com
OpenPGP: EA04 83D4 C864 AA7C 1099 4BE6 A11E 70AD FDA6 0CF9

From khaberz at gmail.com  Wed Dec 29 13:08:11 2021
From: khaberz at gmail.com (Kai Haberzettl)
Date: Wed, 29 Dec 2021 14:08:11 +0100
Subject: Install Wireguard macos without having to use AppleID?
In-Reply-To: <CABtXGiBDnVhKg-EcwSnSmkQkf=1k8_vnXADLmrUbgjFefkqOLw@mail.gmail.com>
References: <CABtXGiBDnVhKg-EcwSnSmkQkf=1k8_vnXADLmrUbgjFefkqOLw@mail.gmail.com>
Message-ID: <CA+CGqoK=a_Vheas4z+8Wf8-WYvUfmdd4rgg51v3iER7qQ0-acg@mail.gmail.com>

You can install the wireguard-go homebrew package:
https://formulae.brew.sh/formula/wireguard-go


On Wed, Dec 29, 2021 at 9:52 AM Phillip McMahon
<phillip.mcmahon at gmail.com> wrote:
>
> Background is that I have a new Macbook Pro for the M1 silicon and
> video editing on the go. I have all my apps installed without having
> to use an Apple ID (thanks to brew.sh) with the exception of
> wireguard.
>
> Is there any way outside of the app store to install?
>
> --
> Phillip McMahon
>
> https://flowcrypt.com/attester/pub/phillip.mcmahon at gmail.com
> OpenPGP: EA04 83D4 C864 AA7C 1099 4BE6 A11E 70AD FDA6 0CF9

From mike at jazmin.com.au  Mon Dec 20 21:49:25 2021
From: mike at jazmin.com.au (Mike O'Connor)
Date: Tue, 21 Dec 2021 08:19:25 +1030
Subject: eBPF + IPv6 + WireGuard
In-Reply-To: <20211217190659.251f006a@poseidon.quill.lan>
References: <20211217190659.251f006a@poseidon.quill.lan>
Message-ID: <35F53B88-F1A7-452D-BC86-04A49677B372@jazmin.com.au>

Hi

Wireguard is a layer 3 system. As a result you must route traffic not bridge, this also means that ip forwarding must be enabled.

You will need to firewall the traffic or setup a separate routing table. 

Other routers in your network will need to know about the IP addresses allocated to Wireguard and have a route for this range.

Mike

> On 21 Dec 2021, at 3:21 am, Alex <wireguard at centromere.net> wrote:
> 
> ?Hi all,
> 
> I am championing WireGuard at work, and I have been granted permission
> to use it for establishing remote access to a private IPv6 VLAN for all
> employees. I have experimented with different approaches and ran in to
> an issue.
> 
> With the help of a machine dedicated fully to the job of remote access
> (Ubuntu 20.04 / Linux 5.4), I've managed to establish end-to-end
> connectivity between my work laptop and the servers. The VPN gateway
> has a wg0 interface for employees and a "private0" interface for
> private VLAN connectivity. The goal is to link the two.
> 
> I noticed an odd quirk: It only works when
> "net.ipv6.conf.all.forwarding" is 1. If this value is 0,
> "net.ipv6.conf.[wg0 | private0].forwarding" has no effect. In other
> words, I cannot seem to enable forwarding for *only* the interfaces
> that need it. It only works when forwarding is enabled for *all*
> interfaces. This is a problem because when the "all" value is set to 1,
> the machine will start to behave as a router on VLANs for which it is
> most definitely not a router.
> 
> For the servers, I am using the officially defined[0] subnet-router
> anycast address as the default IPv6 gateway, and it works well.
> However, when I flipped the switch on "net.ipv6.conf.all.forwarding",
> the VPN gateway started using NDP to announce that it was a router
> across *all* VLANs! Specifically, it was claiming ownership over our
> global subnet anycast address 2001:aaaa:bbbb:cccc::. This is neither
> true nor desired. The VPN gateway started to receive outbound non-VPN
> Internet traffic which broke Internet connectivity for all servers. 
> 
> This leads me to my first question: Why does
> "net.ipv6.conf.wg0.forwarding" have no effect?
> 
> In researching a solution, I decided to give eBPF a try. I attached an
> XDP program to "private0" (a layer 2 device, naturally) and
> successfully redirected packets to "wg0" (a layer 3 device) with
> bpf_redirect[1]. If the packets are unmodified, WireGuard will happily
> pass them to the remote hosts. This is an issue because the remote
> hosts are expecting to receive IP(v6) packets, not Ethernet frames. If
> I use bpf_xdp_adjust_head[1] to strip off the Ethernet frame, WireGuard
> will drop the packet[2] before it can be sent to the remote host.
> 
> My theory is that bpf_xdp_adjust_head is modifying the data pointer
> only and not any underlying structures that may be associated with the
> packet (sk_buff perhaps).
> 
> This leads me to my second question: Why can't I redirect traffic
> received on an L2 interface to an L3 interface, *even after stripping
> off the Ethernet frame?*
> 
> Finally, during this whole process I was using WireShark to inspect
> traffic received by the remote host (i.e. my work laptop). With the
> help of the extract-handshakes.sh script, I was able to decrypt traffic.
> I did discover a bug though. I believe "index_hashtable_insert" should
> actually be "wg_index_hashtable_insert"[3].
> 
> Does anyone have any insights as to what a proper solution would look
> like? Is there a way to achieve my goal without introducing eBPF? Is
> XDP completely unsuited for this particular purpose? Do I actually need
> to operate on the sk_buff level, as opposed to the xdp_buff level?
> 
> Thank you all for your time.
> 
> - Alex
> 
> [0] https://datatracker.ietf.org/doc/rfc1884/ (Section 2.5.1)
> [1] https://www.man7.org/linux/man-pages/man7/bpf-helpers.7.html
> [2] https://git.zx2c4.com/wireguard-linux/tree/drivers/net/wireguard/device.c#n131
> [3] https://git.zx2c4.com/wireguard-tools/tree/contrib/extract-handshakes/extract-handshakes.sh#n46


From zoredache at gmail.com  Tue Dec 21 07:51:55 2021
From: zoredache at gmail.com (Chris Francy)
Date: Mon, 20 Dec 2021 23:51:55 -0800
Subject: Windows tunnel configured as only IPv6 gets an IPv4 APIPA address
Message-ID: <CAM8ZctXygkOUvzEpROjcfZeFC69epbc8ef=k5OF949XB6S1-BQ@mail.gmail.com>

I really wish that the IPv4 binding would be completely disabled when an
interface has no IPv4 addresses.

Consider a simple configuration like this

    # wg_zoredache.conf
    [Interface]
    PrivateKey = ...
    Address = 2001:db8:0:0:24ae:ebff:fef0:97e6/128

When that tunnel is started on Windows you will get an 169.254.0.0/16
address. But I didn't really expect or want that.

I am sure some will tell me this doesn't really hurt much since a person
configuring an IPv6 only tunnel, would not have AllowedIPs set that would
allow this address to actually communicate with anything, so the tunnel
will not accept or transmit these packets. But it is still kinda annoying,
since having that IPv4 address will result in IPv4 broadcast traffic
happening on the interface. I noticed this when I was trying to do a
wireshark capture for something else, and was surprised how much IPv4
broadcasts I was seeing on an interface that I intended to be IPv6 only.

So next I am sure people will suggest adding a `PreUp' or 'PostUp` command
that runs a powershell command to disable the interface. I know with
powershell you can disable the IPv4 binding with a command like
`Disable-NetAdapterBinding -ifAlias wg_zoredache -ComponentID ms_tcpip`.
But PreUp/Postup is blocked by default on Windows. Also some security
software on block Powershell executing as a form of anti-malware
protection. Also, simply running the command once isn't useful, since the
Interface seems to get recreated each time the tunnel or computer restarts.

I have more output of what I currently get, and what I am hoping for
in this gist.

- https://gist.github.com/zoredache/995bb5bea01f6fd8dced0fda708c97c8

Anyway, I am just posting this as a bit of a wishlist request. Please
consider adjusting the client to disable the IPv4 binding on the interface
it creates if no IPv4 addresses are in the tunnel configuration. Or perhaps
provide some other method to have a true IPv6 only wireguard interface on
Windows.

Thanks,

Zoredache

From houmie at gmail.com  Mon Dec 27 07:42:27 2021
From: houmie at gmail.com (Houman)
Date: Mon, 27 Dec 2021 11:12:27 +0330
Subject: iOS WireGuard Kit integration with my project can't be distributed to
 AppStore
Message-ID: <CABBZOskdS0P-3hx91JXkGkE5qaTdx_3B+gYGE4+44N9ZQg8Yhw@mail.gmail.com>

I have followed the instructions from WireGuard Kit Integration to
create a simple client for myself.
https://github.com/WireGuard/wireguard-apple#wireguardkit-integration
I got it perfectly working.

But when I try to release it to the AppStore via Xcode -> Release ->
Archive I get into this situation
where the 'distribute app' option to the AppStore is missing.
Strangely the archive appears only after a few seconds under 'Other
Items' instead of under iOS Apps.

This has been reported previously and explained here:
https://stackoverflow.com/questions/55467829/xcode-10-1-doesnt-show-distribute-app-option-when-archiving-cant-archive
on Stackoverflow, but none of the solutions work for me.

Judging from this answer here:
https://stackoverflow.com/a/67650750/92153 It seems that this is
indeed an issue with Wireguard Framework.

Can someone please advise what to do?
Many Thanks,

From leon at is.currently.online  Tue Dec 28 23:45:21 2021
From: leon at is.currently.online (leon at is.currently.online)
Date: Wed, 29 Dec 2021 00:45:21 +0100
Subject: [RFC PATCH 0/4] Introduce per-peer MTU setting
Message-ID: <20211228234524.633509-1-leon@is.currently.online>

From: Leon Schuermann <leon at is.currently.online>

This patch series is an attempt to integrate a per-peer MTU setting
into WireGuard. With matching changes to the wireguard-tools,
individual MTU values can be set and retrieved for each registered
peer.

While Linux supports setting an MTU metric for specific FIB route
entries [which I've only found out after implementing this :)], and
thus allows to lower the MTU for individual peers, this appears to
disable regular path MTU discovery (PMTUD) entirely on the
route. While regular PMTUD does not work over the tunnel link, it
should still be usable on the rest of the route.

Furthermore, with the goal of eventually introducing an in-band
per-peer PMTUD mechanism, keeping an internal per-peer MTU value does
not require modifying the FIB and thus potentially interfere with
userspace.

In an effort to solve this issues, this patch series introduces a
rather generic framework for implementing these kinds of dynamic MTU
policies. By providing a hook in the netdevice implementation to
decide on the applicable MTU, very flexible designs can be built. I
suppose that these changes are rather controversial, or at least
require some more discussion. I'm sending this patchset to the
WireGuard development list in hopes to get some initial feedback on
the idea and implementation and would like to eventually submit the
non-WireGuard changes directly to the netdev ML.

The patches are currently based on v5.10, as that happens to be
what I was developing on. I'll gladly rebase to the latest
revision / wireguard-devel if requested.

Thanks!

Leon

Leon Schuermann (4):
  netdevice: add ndo_lookup_mtu for dynamically determining MTU
  net/ipv4: respect MTU determined by `ndo_lookup_mtu`
  net/ipv6: respect MTU determined by `ndo_lookup_mtu`
  net/wireguard: add per-peer MTU setting

 drivers/net/wireguard/allowedips.c |  2 +-
 drivers/net/wireguard/allowedips.h |  2 +-
 drivers/net/wireguard/device.c     | 20 ++++++++++++++++--
 drivers/net/wireguard/netlink.c    |  8 +++++++
 drivers/net/wireguard/peer.c       |  1 +
 drivers/net/wireguard/peer.h       |  1 +
 drivers/net/wireguard/queueing.h   |  2 +-
 include/linux/netdevice.h          | 12 +++++++++++
 include/net/ip.h                   | 34 ++++++++++++++++++++++--------
 include/net/ip6_route.h            | 14 ++++++++++--
 include/uapi/linux/wireguard.h     |  5 +++++
 net/ipv4/ip_forward.c              |  2 +-
 net/netfilter/nf_flow_table_core.c |  2 +-
 13 files changed, 87 insertions(+), 18 deletions(-)


base-commit: 2c85ebc57b3e1817b6ce1a6b703928e113a90442
-- 
2.33.1


From leon at is.currently.online  Tue Dec 28 23:45:23 2021
From: leon at is.currently.online (leon at is.currently.online)
Date: Wed, 29 Dec 2021 00:45:23 +0100
Subject: [RFC PATCH 1/4] netdevice: add ndo_lookup_mtu for dynamically
 determining MTU
In-Reply-To: <20211228234524.633509-1-leon@is.currently.online>
References: <20211228234524.633509-1-leon@is.currently.online>
Message-ID: <20211228234524.633509-2-leon@is.currently.online>

From: Leon Schuermann <leon at is.currently.online>

Add an optional function `ndo_lookup_mtu` to the `struct
net_device_ops`. This function can be used to allow other parts of the
network stack to let the destination netdevice determine the allowed
packet MTU. This is done on a per-packet basis, providing the `struct
sk_buff` holding the packet contents.

The information obtained through this method may be cached by other
parts of the network stack, such as for instance the path MTU
discovery (PMTUD) mechanism. It is not guaranteed that this function
will be called for every packet, not even that is called on a single
packet of a given flow. When this function is not implemented or when
it returns -ENODATA no statement about the permitted MTU is made and
the networking stack will resort to the device MTU values. These
properties make this mechanism capable of providing a "suggestion" for
a packet's MTU, deviating from the default device MTU.

The device is allowed to announce MTU values lower or higher than the
minimum and maximum device MTU respectively. Whether such MTU values
will be respected is up to the implementation.

Still, even with this being a non-mandatory to implement or respect
mechanism, it has some interesting consequences. Being able to inspect
the entire packet buffer, the destination netdevice implementation can
control MTUs on a flow granularity. For instance, it could be used to
allow two devices on a shared Ethernet segment to communicate with
each other using a large (> 1500 byte) MTU, while using a lower MTU
for other devices.

The immediate motivation for these changes provide another example of
this mechanism being useful: when using WireGuard, peers can reside
behind paths of varying MTU restrictions. PMTUD does not work across
these tunnel links however, as WireGuard cannot accept unauthenticated
ICMP responses. Thus it will continue to send too large packets over
lower-MTU links. With this mechanism WireGuard can, on a per-peer
granularity, reduce the MTU, without limiting the overall device
MTU. Furthermore, it can employ in-band PMTUD mechanisms to resolve
these values automatically. While an MTU metric can be set for
specific FIB routes and thus lower the MTU for individual peers, as a
consequence this completely disables PMTUD on the entire route. While
regular PMTUD does not work over the tunnel link, it should still be
usable on the rest of the route. Furthermore, when employing an
in-band per-peer PMTUD mechanism, modifying the FIB to store the
detected MTU is inelegant at best.

Signed-off-by: Leon Schuermann <leon at is.currently.online>
---
 include/linux/netdevice.h | 12 ++++++++++++
 1 file changed, 12 insertions(+)

diff --git a/include/linux/netdevice.h b/include/linux/netdevice.h
index 7c3da0e1ea9d..d9d59b756f57 100644
--- a/include/linux/netdevice.h
+++ b/include/linux/netdevice.h
@@ -1279,6 +1279,16 @@ struct netdev_net_notifier {
  * struct net_device *(*ndo_get_peer_dev)(struct net_device *dev);
  *	If a device is paired with a peer device, return the peer instance.
  *	The caller must be under RCU read context.
+ * int (*ndo_lookup_mtu)(const struct sk_buff *skb,
+ *			 const struct net_device *dev);
+ *	For devices supporting dynamic lookup of the MTU for individual
+ *	skb packets, this function returns the MTU for the passed skb.
+ *	A return value of -ENODATA must be treated as if the device does
+ *	not support this feature. It is not guaranteed that this function will
+ *	be called for every packet presented to the ndo_start_xmit function.
+ *	A device must always accept packets of the announced min/max device MTU.
+ *	This function may be used to potentially allow MTU sizes lower/higher
+ *	than the min/max device MTU respectively.
  */
 struct net_device_ops {
 	int			(*ndo_init)(struct net_device *dev);
@@ -1487,6 +1497,8 @@ struct net_device_ops {
 	int			(*ndo_tunnel_ctl)(struct net_device *dev,
 						  struct ip_tunnel_parm *p, int cmd);
 	struct net_device *	(*ndo_get_peer_dev)(struct net_device *dev);
+	int			(*ndo_lookup_mtu)(const struct sk_buff *skb,
+						  const struct net_device *dev);
 };
 
 /**
-- 
2.33.1


From leon at is.currently.online  Tue Dec 28 23:45:25 2021
From: leon at is.currently.online (leon at is.currently.online)
Date: Wed, 29 Dec 2021 00:45:25 +0100
Subject: [RFC PATCH 2/4] net/ipv4: respect MTU determined by `ndo_lookup_mtu`
In-Reply-To: <20211228234524.633509-1-leon@is.currently.online>
References: <20211228234524.633509-1-leon@is.currently.online>
Message-ID: <20211228234524.633509-3-leon@is.currently.online>

From: Leon Schuermann <leon at is.currently.online>

This integrates the newly introduced dynamic MTU lookup mechanism with
the IPv4 stack. It will attempt to query the destination netdevice for
the individual packet MTU and, if not found or the mechanism is not
implemented, fall back to the device MTU.

`ndo_lookup_mtu` will not be queried and respected for every
packet. For instance, flow offloading with netfilter will only take
the device MTU into account.

Signed-off-by: Leon Schuermann <leon at is.currently.online>
---
 include/net/ip.h                   | 34 ++++++++++++++++++++++--------
 net/ipv4/ip_forward.c              |  2 +-
 net/netfilter/nf_flow_table_core.c |  2 +-
 3 files changed, 27 insertions(+), 11 deletions(-)

diff --git a/include/net/ip.h b/include/net/ip.h
index 2d6b985d11cc..5232d0c07dea 100644
--- a/include/net/ip.h
+++ b/include/net/ip.h
@@ -433,34 +433,50 @@ static inline bool ip_sk_ignore_df(const struct sock *sk)
 }
 
 static inline unsigned int ip_dst_mtu_maybe_forward(const struct dst_entry *dst,
-						    bool forwarding)
+						    bool forwarding,
+						    const struct sk_buff *skb)
 {
-	struct net *net = dev_net(dst->dev);
+	int err;
 	unsigned int mtu;
+	struct net_device *dev = dst->dev;
 
-	if (net->ipv4.sysctl_ip_fwd_use_pmtu ||
-	    ip_mtu_locked(dst) ||
-	    !forwarding)
-		return dst_mtu(dst);
+	err = -ENODATA;
+	if (skb && dev->netdev_ops->ndo_lookup_mtu)
+		err = dev->netdev_ops->ndo_lookup_mtu(skb, dev);
+	mtu = (err >= 0) ? err : READ_ONCE(dst->dev->mtu);
+
+	if (dev_net(dev)->ipv4.sysctl_ip_fwd_use_pmtu ||
+		ip_mtu_locked(dst) ||
+		!forwarding)
+		return min(mtu, dst_mtu(dst));
 
 	/* 'forwarding = true' case should always honour route mtu */
 	mtu = dst_metric_raw(dst, RTAX_MTU);
 	if (mtu)
 		return mtu;
 
-	return min(READ_ONCE(dst->dev->mtu), IP_MAX_MTU);
+	return min(mtu, IP_MAX_MTU);
 }
 
 static inline unsigned int ip_skb_dst_mtu(struct sock *sk,
 					  const struct sk_buff *skb)
 {
+	int err;
+	unsigned int mtu;
+	struct net_device *dev = skb_dst(skb)->dev;
+
 	if (!sk || !sk_fullsock(sk) || ip_sk_use_pmtu(sk)) {
 		bool forwarding = IPCB(skb)->flags & IPSKB_FORWARDED;
 
-		return ip_dst_mtu_maybe_forward(skb_dst(skb), forwarding);
+		return ip_dst_mtu_maybe_forward(skb_dst(skb), forwarding, skb);
 	}
 
-	return min(READ_ONCE(skb_dst(skb)->dev->mtu), IP_MAX_MTU);
+	err = -ENODATA;
+	if (dev->netdev_ops->ndo_lookup_mtu)
+		err = dev->netdev_ops->ndo_lookup_mtu(skb, dev);
+	mtu = (err >= 0) ? err : READ_ONCE(dev->mtu);
+
+	return min(mtu, IP_MAX_MTU);
 }
 
 struct dst_metrics *ip_fib_metrics_init(struct net *net, struct nlattr *fc_mx,
diff --git a/net/ipv4/ip_forward.c b/net/ipv4/ip_forward.c
index 00ec819f949b..7a7ec3643c37 100644
--- a/net/ipv4/ip_forward.c
+++ b/net/ipv4/ip_forward.c
@@ -127,7 +127,7 @@ int ip_forward(struct sk_buff *skb)
 		goto sr_failed;
 
 	IPCB(skb)->flags |= IPSKB_FORWARDED;
-	mtu = ip_dst_mtu_maybe_forward(&rt->dst, true);
+	mtu = ip_dst_mtu_maybe_forward(&rt->dst, true, skb);
 	if (ip_exceeds_mtu(skb, mtu)) {
 		IP_INC_STATS(net, IPSTATS_MIB_FRAGFAILS);
 		icmp_send(skb, ICMP_DEST_UNREACH, ICMP_FRAG_NEEDED,
diff --git a/net/netfilter/nf_flow_table_core.c b/net/netfilter/nf_flow_table_core.c
index 513f78db3cb2..95bd7a87066a 100644
--- a/net/netfilter/nf_flow_table_core.c
+++ b/net/netfilter/nf_flow_table_core.c
@@ -87,7 +87,7 @@ static int flow_offload_fill_route(struct flow_offload *flow,
 
 	switch (flow_tuple->l3proto) {
 	case NFPROTO_IPV4:
-		flow_tuple->mtu = ip_dst_mtu_maybe_forward(dst, true);
+		flow_tuple->mtu = ip_dst_mtu_maybe_forward(dst, true, NULL);
 		break;
 	case NFPROTO_IPV6:
 		flow_tuple->mtu = ip6_dst_mtu_forward(dst);
-- 
2.33.1


From leon at is.currently.online  Tue Dec 28 23:45:27 2021
From: leon at is.currently.online (leon at is.currently.online)
Date: Wed, 29 Dec 2021 00:45:27 +0100
Subject: [RFC PATCH 3/4] net/ipv6: respect MTU determined by `ndo_lookup_mtu`
In-Reply-To: <20211228234524.633509-1-leon@is.currently.online>
References: <20211228234524.633509-1-leon@is.currently.online>
Message-ID: <20211228234524.633509-4-leon@is.currently.online>

From: Leon Schuermann <leon at is.currently.online>

This integrates the newly introduced dynamic MTU lookup mechanism with
the IPv6 stack. It will attempt to query the destination netdevice for
the individual packet MTU and, if not found or the mechanism is not
implemented, fall back to the device MTU.

`ndo_lookup_mtu` will not be queried and respected for every
packet. For instance, flow offloading with netfilter will only take
the device MTU into account.

Signed-off-by: Leon Schuermann <leon at is.currently.online>
---
 include/net/ip6_route.h | 14 ++++++++++++--
 1 file changed, 12 insertions(+), 2 deletions(-)

diff --git a/include/net/ip6_route.h b/include/net/ip6_route.h
index 2a5277758379..97e304291d1f 100644
--- a/include/net/ip6_route.h
+++ b/include/net/ip6_route.h
@@ -264,11 +264,21 @@ int ip6_fragment(struct net *net, struct sock *sk, struct sk_buff *skb,
 
 static inline int ip6_skb_dst_mtu(struct sk_buff *skb)
 {
+	int mtu;
+	int err;
+	struct net_device *dev = skb_dst(skb)->dev;
 	struct ipv6_pinfo *np = skb->sk && !dev_recursion_level() ?
 				inet6_sk(skb->sk) : NULL;
 
-	return (np && np->pmtudisc >= IPV6_PMTUDISC_PROBE) ?
-	       skb_dst(skb)->dev->mtu : dst_mtu(skb_dst(skb));
+	err = -ENODATA;
+	if (dev->netdev_ops->ndo_lookup_mtu)
+		err = dev->netdev_ops->ndo_lookup_mtu(skb, dev);
+	mtu = (err >= 0) ? err : READ_ONCE(dev->mtu);
+
+	if (np && np->pmtudisc < IPV6_PMTUDISC_PROBE)
+		mtu = min_t(int, mtu, dst_mtu(skb_dst(skb)));
+
+	return mtu;
 }
 
 static inline bool ip6_sk_accept_pmtu(const struct sock *sk)
-- 
2.33.1


From leon at is.currently.online  Tue Dec 28 23:45:29 2021
From: leon at is.currently.online (leon at is.currently.online)
Date: Wed, 29 Dec 2021 00:45:29 +0100
Subject: [RFC PATCH 4/4] net/wireguard: add per-peer MTU setting
In-Reply-To: <20211228234524.633509-1-leon@is.currently.online>
References: <20211228234524.633509-1-leon@is.currently.online>
Message-ID: <20211228234524.633509-5-leon@is.currently.online>

From: Leon Schuermann <leon at is.currently.online>

WireGuard is, as of now, unable to detect an insufficient link MTU and
adjust the MTU of transmitted frames accordingly, as ICMP messages
which would indicate this are unauthenticated. This poses a problem in
an environment where multiple peers connect to a shared endpoint,
through various links with different MTU constraints. A workaround is
to reduce the shared endpoint's MTU to the largest MTU supported by
every peer link. Naturally, this adds additional overhead for all
peers with more relaxed MTU constraints.

Thus this introduces a per-peer MTU setting into WireGuard. This value
can be set statically through the userspace API. It is announced to
the rest of the network stack through the `ndo_lookup_mtu` module,
which is (partially) respected by both IPv4 and IPv6.

While Linux supports setting an MTU metric for specific FIB route
entries and thus allows to lower the MTU for individual peers, this
causes regular path MTU discovery (PMTUD) to be completely disabled on
the entire route. While regular PMTUD does not work over the tunnel
link, it should still be usable on the rest of the route. Furthermore,
keeping an internal per-peer MTU value paves the way for integrating
an in-band PMTUD mechanism, as it does not require modifying the FIB
to reflect new discovered MTU values.

Signed-off-by: Leon Schuermann <leon at is.currently.online>
---
 drivers/net/wireguard/allowedips.c |  2 +-
 drivers/net/wireguard/allowedips.h |  2 +-
 drivers/net/wireguard/device.c     | 20 ++++++++++++++++++--
 drivers/net/wireguard/netlink.c    |  8 ++++++++
 drivers/net/wireguard/peer.c       |  1 +
 drivers/net/wireguard/peer.h       |  1 +
 drivers/net/wireguard/queueing.h   |  2 +-
 include/uapi/linux/wireguard.h     |  5 +++++
 8 files changed, 36 insertions(+), 5 deletions(-)

diff --git a/drivers/net/wireguard/allowedips.c b/drivers/net/wireguard/allowedips.c
index 3725e9cd85f4..8870a96b223b 100644
--- a/drivers/net/wireguard/allowedips.c
+++ b/drivers/net/wireguard/allowedips.c
@@ -354,7 +354,7 @@ int wg_allowedips_read_node(struct allowedips_node *node, u8 ip[16], u8 *cidr)
 
 /* Returns a strong reference to a peer */
 struct wg_peer *wg_allowedips_lookup_dst(struct allowedips *table,
-					 struct sk_buff *skb)
+					 const struct sk_buff *skb)
 {
 	if (skb->protocol == htons(ETH_P_IP))
 		return lookup(table->root4, 32, &ip_hdr(skb)->daddr);
diff --git a/drivers/net/wireguard/allowedips.h b/drivers/net/wireguard/allowedips.h
index e5c83cafcef4..366f42c04e25 100644
--- a/drivers/net/wireguard/allowedips.h
+++ b/drivers/net/wireguard/allowedips.h
@@ -48,7 +48,7 @@ int wg_allowedips_read_node(struct allowedips_node *node, u8 ip[16], u8 *cidr);
 
 /* These return a strong reference to a peer: */
 struct wg_peer *wg_allowedips_lookup_dst(struct allowedips *table,
-					 struct sk_buff *skb);
+					 const struct sk_buff *skb);
 struct wg_peer *wg_allowedips_lookup_src(struct allowedips *table,
 					 struct sk_buff *skb);
 
diff --git a/drivers/net/wireguard/device.c b/drivers/net/wireguard/device.c
index c9f65e96ccb0..52460a04cffb 100644
--- a/drivers/net/wireguard/device.c
+++ b/drivers/net/wireguard/device.c
@@ -211,11 +211,27 @@ static netdev_tx_t wg_xmit(struct sk_buff *skb, struct net_device *dev)
 	return ret;
 }
 
+static int wg_lookup_mtu(const struct sk_buff *skb, const struct net_device *dev)
+{
+	struct wg_device *wg = netdev_priv(dev);
+	struct wg_peer *peer;
+
+	if (unlikely(!wg_check_packet_protocol(skb)))
+		return -ENODATA;
+
+	peer = wg_allowedips_lookup_dst(&wg->peer_allowedips, skb);
+	if (unlikely(!peer))
+		return -ENODATA;
+
+	return (peer->peer_mtu != 0) ? peer->peer_mtu : -ENODATA;
+}
+
 static const struct net_device_ops netdev_ops = {
 	.ndo_open		= wg_open,
 	.ndo_stop		= wg_stop,
-	.ndo_start_xmit		= wg_xmit,
-	.ndo_get_stats64	= ip_tunnel_get_stats64
+	.ndo_start_xmit	= wg_xmit,
+	.ndo_get_stats64	= ip_tunnel_get_stats64,
+	.ndo_lookup_mtu	= wg_lookup_mtu,
 };
 
 static void wg_destruct(struct net_device *dev)
diff --git a/drivers/net/wireguard/netlink.c b/drivers/net/wireguard/netlink.c
index d0f3b6d7f408..ad636f893027 100644
--- a/drivers/net/wireguard/netlink.c
+++ b/drivers/net/wireguard/netlink.c
@@ -40,6 +40,7 @@ static const struct nla_policy peer_policy[WGPEER_A_MAX + 1] = {
 	[WGPEER_A_RX_BYTES]				= { .type = NLA_U64 },
 	[WGPEER_A_TX_BYTES]				= { .type = NLA_U64 },
 	[WGPEER_A_ALLOWEDIPS]				= { .type = NLA_NESTED },
+	[WGPEER_A_PEER_MTU]				= { .type = NLA_U32 },
 	[WGPEER_A_PROTOCOL_VERSION]			= { .type = NLA_U32 }
 };
 
@@ -142,6 +143,7 @@ get_peer(struct wg_peer *peer, struct sk_buff *skb, struct dump_ctx *ctx)
 				      WGPEER_A_UNSPEC) ||
 		    nla_put_u64_64bit(skb, WGPEER_A_RX_BYTES, peer->rx_bytes,
 				      WGPEER_A_UNSPEC) ||
+		    nla_put_u32(skb, WGPEER_A_PEER_MTU, peer->peer_mtu) ||
 		    nla_put_u32(skb, WGPEER_A_PROTOCOL_VERSION, 1))
 			goto err;
 
@@ -480,6 +482,12 @@ static int set_peer(struct wg_device *wg, struct nlattr **attrs)
 			wg_packet_send_keepalive(peer);
 	}
 
+	if (attrs[WGPEER_A_PEER_MTU]) {
+		const u32 peer_mtu = nla_get_u32(attrs[WGPEER_A_PEER_MTU]);
+
+		peer->peer_mtu = peer_mtu;
+	}
+
 	if (netif_running(wg->dev))
 		wg_packet_send_staged_packets(peer);
 
diff --git a/drivers/net/wireguard/peer.c b/drivers/net/wireguard/peer.c
index b3b6370e6b95..c21704176216 100644
--- a/drivers/net/wireguard/peer.c
+++ b/drivers/net/wireguard/peer.c
@@ -46,6 +46,7 @@ struct wg_peer *wg_peer_create(struct wg_device *wg,
 		goto err_3;
 
 	peer->internal_id = atomic64_inc_return(&peer_counter);
+	peer->peer_mtu = 0;
 	peer->serial_work_cpu = nr_cpumask_bits;
 	wg_cookie_init(&peer->latest_cookie);
 	wg_timers_init(peer);
diff --git a/drivers/net/wireguard/peer.h b/drivers/net/wireguard/peer.h
index 23af40922997..2ba51f29e3da 100644
--- a/drivers/net/wireguard/peer.h
+++ b/drivers/net/wireguard/peer.h
@@ -64,6 +64,7 @@ struct wg_peer {
 	u64 internal_id;
 	struct napi_struct napi;
 	bool is_dead;
+	u32 peer_mtu;
 };
 
 struct wg_peer *wg_peer_create(struct wg_device *wg,
diff --git a/drivers/net/wireguard/queueing.h b/drivers/net/wireguard/queueing.h
index dfb674e03076..f7fc7a32abf6 100644
--- a/drivers/net/wireguard/queueing.h
+++ b/drivers/net/wireguard/queueing.h
@@ -66,7 +66,7 @@ struct packet_cb {
 #define PACKET_CB(skb) ((struct packet_cb *)((skb)->cb))
 #define PACKET_PEER(skb) (PACKET_CB(skb)->keypair->entry.peer)
 
-static inline bool wg_check_packet_protocol(struct sk_buff *skb)
+static inline bool wg_check_packet_protocol(const struct sk_buff *skb)
 {
 	__be16 real_protocol = ip_tunnel_parse_protocol(skb);
 	return real_protocol && skb->protocol == real_protocol;
diff --git a/include/uapi/linux/wireguard.h b/include/uapi/linux/wireguard.h
index ae88be14c947..b86ab0320fa5 100644
--- a/include/uapi/linux/wireguard.h
+++ b/include/uapi/linux/wireguard.h
@@ -49,6 +49,7 @@
  *                    ...
  *                ...
  *            WGPEER_A_PROTOCOL_VERSION: NLA_U32
+ *            WGPEER_A_PEER_MTU: NLA_U32
  *        0: NLA_NESTED
  *            ...
  *        ...
@@ -111,6 +112,9 @@
  *                                       most recent protocol will be used when
  *                                       this is unset. Otherwise, must be set
  *                                       to 1.
+ *            WGPEER_A_PEER_MTU: per-peer TX MTU (IP header + payload), or 0 to
+ *                                       use the interface MTU. Applies only if
+ *                                       lower than the interface MTU.
  *        0: NLA_NESTED
  *            ...
  *        ...
@@ -180,6 +184,7 @@ enum wgpeer_attribute {
 	WGPEER_A_TX_BYTES,
 	WGPEER_A_ALLOWEDIPS,
 	WGPEER_A_PROTOCOL_VERSION,
+	WGPEER_A_PEER_MTU,
 	__WGPEER_A_LAST
 };
 #define WGPEER_A_MAX (__WGPEER_A_LAST - 1)
-- 
2.33.1