From daniel at makrotopia.org  Sun Jul  4 14:15:18 2021
From: daniel at makrotopia.org (Daniel Golle)
Date: Sun, 4 Jul 2021 15:15:18 +0100
Subject: passing-through TOS/DSCP marking
In-Reply-To: <87h7hf139u.fsf@toke.dk>
References: <87v96dpepz.fsf@toke.dk>
 <0102017a18f77a7e-85cc3154-dbac-4a9f-a0c5-acba247919a6-000000@eu-west-1.amazonses.com>
 <YMsYrpAm/29Q91GX@makrotopia.org> <87sg1gptky.fsf@toke.dk>
 <877disdre0.fsf@toke.dk>
 <CAHmME9qnqixzDKegMprkT8xNXHfbrhGZfwAROp6vjrQLJEo87g@mail.gmail.com>
 <YNCHs1wKWEoNxUyX@makrotopia.org> <877dinths3.fsf@toke.dk>
 <YNyopHsKX2m5HSRr@makrotopia.org> <87h7hf139u.fsf@toke.dk>
Message-ID: <YOHCdnZoAKBhNIF8@makrotopia.org>

Hi Toke,

thank you for the ongoing efforts and support on this issue.

On Wed, Jun 30, 2021 at 10:55:09PM +0200, Toke H?iland-J?rgensen wrote:
> Daniel Golle <daniel at makrotopia.org> writes:
> > ...
> >> >
> >> > In terms of toolchain: LLVM/Clang is a very bulky beast, I gave up on
> >> > that and started working on integrating GCC-10's BPF target in our build
> >> > system...
> >> 
> >> I saw that, but I have no idea if GCC's BPF target support will support
> >> this. My tentative guess would be no, unfortunately :(
> >
> > Probably you are right. When building the BPF object with GCC, the
> > result is:
> > root at OpenWrt:/usr/lib/bpf# preserve-dscp wg0 eth0
> > libbpf: elf: skipping unrecognized data section(4) .stab
> > libbpf: elf: skipping relo section(5) .rel.stab for section(4) .stab
> > libbpf: elf: skipping unrecognized data section(13) .comment
> > libbpf: BTF is required, but is missing or corrupted.
> > Couldn't open file: preserve_dscp_kern.o
> 
> Hmm, for this example it should be possible to make it run without BTF.
> I'm only using that for the map definition, so that could be changed to
> the old format; you could try this patch:
> 
> diff --git a/preserve-dscp/preserve_dscp_kern.c b/preserve-dscp/preserve_dscp_kern.c
> index 24120cb8a3ff..08248e1f0e41 100644
> --- a/preserve-dscp/preserve_dscp_kern.c
> +++ b/preserve-dscp/preserve_dscp_kern.c
> @@ -9,12 +9,12 @@
>   * otherwise clean up stale entries. Instead, we just rely on the LRU mechanism
>   * to evict old entries as the map fills up.
>   */
> -struct {
> -       __uint(type, BPF_MAP_TYPE_LRU_HASH);
> -       __type(key, __u32);
> -       __type(value, __u8);
> -       __uint(max_entries, 16384);
> -} flow_dscps SEC(".maps");
> +struct bpf_map_def SEC("maps") flow_dscps = {
> +       .type           = BPF_MAP_TYPE_LRU_HASH,
> +       .key_size       = sizeof(__u32),
> +       .value_size     = sizeof(__u8),
> +       .max_entries    = 16384,
> +};
>  
>  const volatile static int ip_only = 0;
> 

That change gives me the next error:
libbpf: map '' (legacy): static maps are not supported

Also speaks for itself...

> > Using the LLVM/Clang compiled object also doesn't work:
> > root at OpenWrt:/usr/lib/bpf# preserve-dscp wg0 eth0
> > libbpf: Error in bpf_create_map_xattr(flow_dscps):Operation not permitted(-1). Retrying without BTF.
> > libbpf: map 'flow_dscps': failed to create: Operation not permitted(-1)
> > libbpf: permission error while running as root; try raising 'ulimit -l'? current value: 512.0 KiB
> > libbpf: failed to load object 'preserve_dscp_kern.o'
> > Failed to load object
> >
> > Probably Kernel 5.4.124 is too old...?
> 
> Here I think the hint is in the error message ;)

Yep, I realized I had to increase it to ulimit to 2048...
With that at least the LLVM/Clang generated BPF object seems to load
properly, and I can load and unload it as expected.


> 
> >> An alternative to getting LLVM built as part of the OpenWrt toolchain is
> >> to just use the host clang to build the BPF binaries. It doesn't
> >> actually need to be cross-compiled with a special compiler, the BPF byte
> >> code format is the same on all architectures except for endianness, so
> >> just passing that to the host clang should theoretically be enough...
> >
> > I believe that having a way to build BPF objects compatible with the
> > target built-into our toolchain would be a huge step forward.
> > And given that gcc already get's pretty far, I think it'd be worth
> > fixing/patching what ever is missing (I haven't even tried GCC-11 yet)
> 
> For this example that might work (as noted above), but for other things
> BTF is a hard requirement, and I don't believe GCC supports that at all,
> sadly :(

It looks like this has changed very recently:

https://gcc.gnu.org/git/?p=gcc.git;a=commit;h=d5cf2b5db325fd5c053ca7bc8d6a54a06cd71124


> 
> > Find my staging tree including 'preserve-dscp' ready to play with:
> >
> > https://git.openwrt.org/?p=openwrt/staging/dangole.git;a=shortlog;h=refs/heads/gcc10-bpf
> >
> > Select 'Enable experimental features by default', but note that toolchain
> > doesn't build when selecting Linux 5.10 for x86, so you need to un-select
> > 'Use testing Kernel' if building for x86.
> > And have a look at the patch for allow building bpf-examples BPF objects
> > with GCC in package/network/utils/bpf-examples/patches
> >
> >
> >> 
> >> > In terms of kernel support: recent kernels don't build yet because of
> >> > gelf_getsymshndx, so we got to update libelf first for that. Recent
> >> > libelf doesn't seem to be an option yet on many of the build hosts we
> >> > currently support (Darwin and such).
> >> >
> >> > In terms of library support: our build of libbpf comes from Linux
> >> > release tarballs. There isn't yet a release supporting bpf_tc_attach,
> >> > the easiest would be to wait for Linux 5.13 to be released.
> >> 
> >> I used the libbpf TC loading support for convenience, but it's possible
> >> to load it using 'tc' as well without too much trouble (right now the
> >> userspace component sets a config variable before loading the program,
> >> but it can be restructured to not need that).
> >> 
> >> Alternatively, the bpf-examples repository is setup with a libbpf
> >> submodule that it can link statically against, so you could use that for
> >> now?
> >
> > I've updated to 5.13 + patches on top, so now it builds :)
> 
> Alright, that works.
> 
> > Library-embedding is a no-go for OpenWrt. Having different ABI-versions
> > of libraries installed simultanously works, so we can just ship with
> > a more recent version of libbpf.
> 
> Yeah, I wasn't suggesting it as a permanent solution, just so you could
> test it out :)

In the long run it would be great to have a somehow standardized and
reproducible way to build packages containing targetted BPF objects.
As having LLVM/Clang built-into OpenWrt has shown to be a huge amount
of work, I'm looking forward to GCC-12 which will support BTF from how
it looks by now...

> 
> >> > I (of course ;) also tried and spend almost a day looking for a
> >> > quick-and-dirty path for temporary deployment, so I could at least give
> >> > feedback -- bpf-examples also isn't exactly made to be cross-compiled
> >> > manually, so I have failed with that as well so far.
> >> 
> >> Heh, no, it isn't, really. Anything in particular you need to make this
> >> easier? We already added some bits to xdp-tools for supporting
> >> cross-compilation (and that shares some lineage with bpf-examples), so
> >> porting those over should not be too difficult.
> >
> > I found my way around, see the packaging for bpf-examples in the tree
> > (link above, at path stated above)
> 
> Right, I see. 

I have managed to test your solution and it seems to do the job.
Remaining issues:
 * What to do if there are many tunnels all sharing the same upstream
   interface? In this case I'm thinking of doing:
   preserve-dscp wg0 eth0
   preserve-dscp wg1 eth0
   preserve-dscp wg2 eth0
   ...
   But I'm unsure whether this is indented or if further details need
   to be implemented in order to make that work.

 * Once a wireguard interface goes down, one cannot unload the
   remaining program on the upstream interface, as
   preserve-dscp wg0 eth0 --unload
   would fail in case of 'wg0' having gone missing.
   What do you suggest to do in this case?


> 
> >> 
> >> See: https://github.com/xdp-project/xdp-tools/pull/78 and
> >> https://github.com/xdp-project/xdp-tools/issues/74
> >> 
> >> Unfortunately I don't have a lot of time to poke more at this right now,
> >> but feel free to open up an issue / pull request to the bpf-examples
> >> repository with any changes you need :)
> >
> > I guess I'll just go ahead then and package xdp-tools :)
> 
> That would be awesome! xdp-tools will definitely need BTF, though, so
> I'm afraid it'll need to be compiled with LLVM at this stage...

I'll probably move on doing other things for a while then and get back
to it once GCC-12 has been released...


Cheers


Daniel

From rcmcdonald91 at gmail.com  Sun Jul  4 20:57:20 2021
From: rcmcdonald91 at gmail.com (Christian McDonald)
Date: Sun, 4 Jul 2021 16:57:20 -0400
Subject: 'wg syncconf' not removing keep alive once set
Message-ID: <CADTMz0LJfBUtnKonOhvzs9ZzFPAiuvAG0qD79kATW=1hOgo_zg@mail.gmail.com>

If wg syncconf is called using a .conf that sets a keep alive, the
keep alive can't then be removed by subsequent calls to syncconf using
a conf that lacks the keep alive value. This seems counterintuitive.
The only way to disable the keep alive once set is to set the value to
zero.

-- 
R. Christian McDonald
E: rcmcdonald91 at gmail.com

From toke at toke.dk  Mon Jul  5 15:21:25 2021
From: toke at toke.dk (Toke =?utf-8?Q?H=C3=B8iland-J=C3=B8rgensen?=)
Date: Mon, 05 Jul 2021 17:21:25 +0200
Subject: passing-through TOS/DSCP marking
In-Reply-To: <YOHCdnZoAKBhNIF8@makrotopia.org>
References: <87v96dpepz.fsf@toke.dk>
 <0102017a18f77a7e-85cc3154-dbac-4a9f-a0c5-acba247919a6-000000@eu-west-1.amazonses.com>
 <YMsYrpAm/29Q91GX@makrotopia.org> <87sg1gptky.fsf@toke.dk>
 <877disdre0.fsf@toke.dk>
 <CAHmME9qnqixzDKegMprkT8xNXHfbrhGZfwAROp6vjrQLJEo87g@mail.gmail.com>
 <YNCHs1wKWEoNxUyX@makrotopia.org> <877dinths3.fsf@toke.dk>
 <YNyopHsKX2m5HSRr@makrotopia.org> <87h7hf139u.fsf@toke.dk>
 <YOHCdnZoAKBhNIF8@makrotopia.org>
Message-ID: <87zgv0yefe.fsf@toke.dk>

Daniel Golle <daniel at makrotopia.org> writes:

> Hi Toke,
>
> thank you for the ongoing efforts and support on this issue.

You're welcome! :)

> On Wed, Jun 30, 2021 at 10:55:09PM +0200, Toke H?iland-J?rgensen wrote:
>> Daniel Golle <daniel at makrotopia.org> writes:
>> > ...
>> >> >
>> >> > In terms of toolchain: LLVM/Clang is a very bulky beast, I gave up on
>> >> > that and started working on integrating GCC-10's BPF target in our build
>> >> > system...
>> >> 
>> >> I saw that, but I have no idea if GCC's BPF target support will support
>> >> this. My tentative guess would be no, unfortunately :(
>> >
>> > Probably you are right. When building the BPF object with GCC, the
>> > result is:
>> > root at OpenWrt:/usr/lib/bpf# preserve-dscp wg0 eth0
>> > libbpf: elf: skipping unrecognized data section(4) .stab
>> > libbpf: elf: skipping relo section(5) .rel.stab for section(4) .stab
>> > libbpf: elf: skipping unrecognized data section(13) .comment
>> > libbpf: BTF is required, but is missing or corrupted.
>> > Couldn't open file: preserve_dscp_kern.o
>> 
>> Hmm, for this example it should be possible to make it run without BTF.
>> I'm only using that for the map definition, so that could be changed to
>> the old format; you could try this patch:
>> 
>> diff --git a/preserve-dscp/preserve_dscp_kern.c b/preserve-dscp/preserve_dscp_kern.c
>> index 24120cb8a3ff..08248e1f0e41 100644
>> --- a/preserve-dscp/preserve_dscp_kern.c
>> +++ b/preserve-dscp/preserve_dscp_kern.c
>> @@ -9,12 +9,12 @@
>>   * otherwise clean up stale entries. Instead, we just rely on the LRU mechanism
>>   * to evict old entries as the map fills up.
>>   */
>> -struct {
>> -       __uint(type, BPF_MAP_TYPE_LRU_HASH);
>> -       __type(key, __u32);
>> -       __type(value, __u8);
>> -       __uint(max_entries, 16384);
>> -} flow_dscps SEC(".maps");
>> +struct bpf_map_def SEC("maps") flow_dscps = {
>> +       .type           = BPF_MAP_TYPE_LRU_HASH,
>> +       .key_size       = sizeof(__u32),
>> +       .value_size     = sizeof(__u8),
>> +       .max_entries    = 16384,
>> +};
>>  
>>  const volatile static int ip_only = 0;
>> 
>
> That change gives me the next error:
> libbpf: map '' (legacy): static maps are not supported
>
> Also speaks for itself...

Ah, right, the ip_only config also needs to be moved to an old-style
map, then...

>> > Using the LLVM/Clang compiled object also doesn't work:
>> > root at OpenWrt:/usr/lib/bpf# preserve-dscp wg0 eth0
>> > libbpf: Error in bpf_create_map_xattr(flow_dscps):Operation not permitted(-1). Retrying without BTF.
>> > libbpf: map 'flow_dscps': failed to create: Operation not permitted(-1)
>> > libbpf: permission error while running as root; try raising 'ulimit -l'? current value: 512.0 KiB
>> > libbpf: failed to load object 'preserve_dscp_kern.o'
>> > Failed to load object
>> >
>> > Probably Kernel 5.4.124 is too old...?
>> 
>> Here I think the hint is in the error message ;)
>
> Yep, I realized I had to increase it to ulimit to 2048...
> With that at least the LLVM/Clang generated BPF object seems to load
> properly, and I can load and unload it as expected.

Awesome!

>> >> An alternative to getting LLVM built as part of the OpenWrt toolchain is
>> >> to just use the host clang to build the BPF binaries. It doesn't
>> >> actually need to be cross-compiled with a special compiler, the BPF byte
>> >> code format is the same on all architectures except for endianness, so
>> >> just passing that to the host clang should theoretically be enough...
>> >
>> > I believe that having a way to build BPF objects compatible with the
>> > target built-into our toolchain would be a huge step forward.
>> > And given that gcc already get's pretty far, I think it'd be worth
>> > fixing/patching what ever is missing (I haven't even tried GCC-11 yet)
>> 
>> For this example that might work (as noted above), but for other things
>> BTF is a hard requirement, and I don't believe GCC supports that at all,
>> sadly :(
>
> It looks like this has changed very recently:
>
> https://gcc.gnu.org/git/?p=gcc.git;a=commit;h=d5cf2b5db325fd5c053ca7bc8d6a54a06cd71124

Uhh, exciting! Will be interesting to see how compatible this will be;
and how BPF upstream will deal with multiple compilers :)

>> > Find my staging tree including 'preserve-dscp' ready to play with:
>> >
>> > https://git.openwrt.org/?p=openwrt/staging/dangole.git;a=shortlog;h=refs/heads/gcc10-bpf
>> >
>> > Select 'Enable experimental features by default', but note that toolchain
>> > doesn't build when selecting Linux 5.10 for x86, so you need to un-select
>> > 'Use testing Kernel' if building for x86.
>> > And have a look at the patch for allow building bpf-examples BPF objects
>> > with GCC in package/network/utils/bpf-examples/patches
>> >
>> >
>> >> 
>> >> > In terms of kernel support: recent kernels don't build yet because of
>> >> > gelf_getsymshndx, so we got to update libelf first for that. Recent
>> >> > libelf doesn't seem to be an option yet on many of the build hosts we
>> >> > currently support (Darwin and such).
>> >> >
>> >> > In terms of library support: our build of libbpf comes from Linux
>> >> > release tarballs. There isn't yet a release supporting bpf_tc_attach,
>> >> > the easiest would be to wait for Linux 5.13 to be released.
>> >> 
>> >> I used the libbpf TC loading support for convenience, but it's possible
>> >> to load it using 'tc' as well without too much trouble (right now the
>> >> userspace component sets a config variable before loading the program,
>> >> but it can be restructured to not need that).
>> >> 
>> >> Alternatively, the bpf-examples repository is setup with a libbpf
>> >> submodule that it can link statically against, so you could use that for
>> >> now?
>> >
>> > I've updated to 5.13 + patches on top, so now it builds :)
>> 
>> Alright, that works.
>> 
>> > Library-embedding is a no-go for OpenWrt. Having different ABI-versions
>> > of libraries installed simultanously works, so we can just ship with
>> > a more recent version of libbpf.
>> 
>> Yeah, I wasn't suggesting it as a permanent solution, just so you could
>> test it out :)
>
> In the long run it would be great to have a somehow standardized and
> reproducible way to build packages containing targetted BPF objects.
> As having LLVM/Clang built-into OpenWrt has shown to be a huge amount
> of work, I'm looking forward to GCC-12 which will support BTF from how
> it looks by now...
>
>> 
>> >> > I (of course ;) also tried and spend almost a day looking for a
>> >> > quick-and-dirty path for temporary deployment, so I could at least give
>> >> > feedback -- bpf-examples also isn't exactly made to be cross-compiled
>> >> > manually, so I have failed with that as well so far.
>> >> 
>> >> Heh, no, it isn't, really. Anything in particular you need to make this
>> >> easier? We already added some bits to xdp-tools for supporting
>> >> cross-compilation (and that shares some lineage with bpf-examples), so
>> >> porting those over should not be too difficult.
>> >
>> > I found my way around, see the packaging for bpf-examples in the tree
>> > (link above, at path stated above)
>> 
>> Right, I see. 
>
> I have managed to test your solution and it seems to do the job.
> Remaining issues:
>  * What to do if there are many tunnels all sharing the same upstream
>    interface? In this case I'm thinking of doing:
>    preserve-dscp wg0 eth0
>    preserve-dscp wg1 eth0
>    preserve-dscp wg2 eth0
>    ...
>    But I'm unsure whether this is indented or if further details need
>    to be implemented in order to make that work.

Hmm, not sure whether that will work out of the box, actually. Would
definitely be doable to make the userspace utility understand how to do
this properly, though. There's nothing in principle preventing this from
working; the loader should just be smart enough to do incremental
loading of multiple "ingress" programs while still sharing the map
between all of them.

The only potential operational issue with using it on multiple wg
interfaces is if they share IP space; because in that case you might
have packets from different tunnels ending up with identical hashes,
confusing the egress side. Fixing this would require the outer BPF
program to know about wg endpoint addresses and map the packets back to
their inner ifindexes using that. But as long as the wireguard tunnels
are using different IP subnets (or mostly forwarding traffic without the
inner addresses as sources or destinations), the hash collision
probability should not be bigger than just traffic on a single tunnel, I
suppose.

One particular thing to watch out for here is IPv6 link-local traffic;
sine wg doesn't generate link-local addresses automatically, they are
commonly configured with (the same) static address (like fe80::1 or
fe80::2), which would make link-local traffic identical across wg
interfaces. But this is only used for particular setups (I use it for
running Babel over wg, for instance), just make sure it won't be an
issue for your deployment scenario :)

>  * Once a wireguard interface goes down, one cannot unload the
>    remaining program on the upstream interface, as
>    preserve-dscp wg0 eth0 --unload
>    would fail in case of 'wg0' having gone missing.
>    What do you suggest to do in this case?

Just fixing the userspace utility to deal with this case properly as
well is probably the easiest. How are you thinking you'd deploy this?
Via ifup hooks on openwrt, or something different?

>> >> See: https://github.com/xdp-project/xdp-tools/pull/78 and
>> >> https://github.com/xdp-project/xdp-tools/issues/74
>> >> 
>> >> Unfortunately I don't have a lot of time to poke more at this right now,
>> >> but feel free to open up an issue / pull request to the bpf-examples
>> >> repository with any changes you need :)
>> >
>> > I guess I'll just go ahead then and package xdp-tools :)
>> 
>> That would be awesome! xdp-tools will definitely need BTF, though, so
>> I'm afraid it'll need to be compiled with LLVM at this stage...
>
> I'll probably move on doing other things for a while then and get back
> to it once GCC-12 has been released...

OK, SGTM.

-Toke

From daniel at makrotopia.org  Mon Jul  5 16:05:31 2021
From: daniel at makrotopia.org (Daniel Golle)
Date: Mon, 5 Jul 2021 17:05:31 +0100
Subject: passing-through TOS/DSCP marking
In-Reply-To: <87zgv0yefe.fsf@toke.dk>
References: <YMsYrpAm/29Q91GX@makrotopia.org> <87sg1gptky.fsf@toke.dk>
 <877disdre0.fsf@toke.dk>
 <CAHmME9qnqixzDKegMprkT8xNXHfbrhGZfwAROp6vjrQLJEo87g@mail.gmail.com>
 <YNCHs1wKWEoNxUyX@makrotopia.org> <877dinths3.fsf@toke.dk>
 <YNyopHsKX2m5HSRr@makrotopia.org> <87h7hf139u.fsf@toke.dk>
 <YOHCdnZoAKBhNIF8@makrotopia.org> <87zgv0yefe.fsf@toke.dk>
Message-ID: <YOMtyz6GJdjJ4xuy@makrotopia.org>

On Mon, Jul 05, 2021 at 05:21:25PM +0200, Toke H?iland-J?rgensen wrote:
> Daniel Golle <daniel at makrotopia.org> writes:
> ...
> > I have managed to test your solution and it seems to do the job.
> > Remaining issues:
> >  * What to do if there are many tunnels all sharing the same upstream
> >    interface? In this case I'm thinking of doing:
> >    preserve-dscp wg0 eth0
> >    preserve-dscp wg1 eth0
> >    preserve-dscp wg2 eth0
> >    ...
> >    But I'm unsure whether this is indented or if further details need
> >    to be implemented in order to make that work.
> 
> Hmm, not sure whether that will work out of the box, actually. Would
> definitely be doable to make the userspace utility understand how to do
> this properly, though. There's nothing in principle preventing this from
> working; the loader should just be smart enough to do incremental
> loading of multiple "ingress" programs while still sharing the map
> between all of them.

You make it at least sound easy :)

> 
> The only potential operational issue with using it on multiple wg
> interfaces is if they share IP space; because in that case you might
> have packets from different tunnels ending up with identical hashes,
> confusing the egress side. Fixing this would require the outer BPF
> program to know about wg endpoint addresses and map the packets back to
> their inner ifindexes using that. But as long as the wireguard tunnels
> are using different IP subnets (or mostly forwarding traffic without the
> inner addresses as sources or destinations), the hash collision
> probability should not be bigger than just traffic on a single tunnel, I
> suppose.
> 
> One particular thing to watch out for here is IPv6 link-local traffic;
> sine wg doesn't generate link-local addresses automatically, they are
> commonly configured with (the same) static address (like fe80::1 or
> fe80::2), which would make link-local traffic identical across wg
> interfaces. But this is only used for particular setups (I use it for
> running Babel over wg, for instance), just make sure it won't be an
> issue for your deployment scenario :)

All this is good to know, but from what I can see now shouldn't be
a problem in our deployment -- it's multiple wireguard links which are
(using fwmark and ip rules) routed over several uplinks. We then use
mwan3 to balance most of the gateway traffic accross the available
wireguard interfaces, using MASQ/SNAT on each tunnel which has a
unique transfer network assigned, and no IPv6 at all.
Hence it should be ok to go under the restrictions you described.

> >  * Once a wireguard interface goes down, one cannot unload the
> >    remaining program on the upstream interface, as
> >    preserve-dscp wg0 eth0 --unload
> >    would fail in case of 'wg0' having gone missing.
> >    What do you suggest to do in this case?
> 
> Just fixing the userspace utility to deal with this case properly as
> well is probably the easiest. How are you thinking you'd deploy this?
> Via ifup hooks on openwrt, or something different?

Yes, I use ifup hooks configured in an init script for procd and have
it tied to the wireguard config sections in /etc/config/network:

https://git.openwrt.org/?p=openwrt/staging/dangole.git;a=blob;f=package/network/utils/bpf-examples/files/wireguard-preserve-dscp.init;h=f1e5e25e663308e057285e2bd8e3bcb9560bdd54;hb=5923a78d74be3f05e734b0be0a832a87be8d369b#l56

Passing multiple inner interfaces to one call to the to-be-modified
preserve-dscp tool could be achieved by some shell magic dealing with
the configuration...

We will have to restart the filter for all inner interfaces in case of
one being added or removed, right?

And maybe I'll come up with some state tracking so orphaned filters can
be removed after configuration changes...



Cheers


Daniel

From toke at toke.dk  Mon Jul  5 16:59:10 2021
From: toke at toke.dk (Toke =?utf-8?Q?H=C3=B8iland-J=C3=B8rgensen?=)
Date: Mon, 05 Jul 2021 18:59:10 +0200
Subject: passing-through TOS/DSCP marking
In-Reply-To: <YOMtyz6GJdjJ4xuy@makrotopia.org>
References: <YMsYrpAm/29Q91GX@makrotopia.org> <87sg1gptky.fsf@toke.dk>
 <877disdre0.fsf@toke.dk>
 <CAHmME9qnqixzDKegMprkT8xNXHfbrhGZfwAROp6vjrQLJEo87g@mail.gmail.com>
 <YNCHs1wKWEoNxUyX@makrotopia.org> <877dinths3.fsf@toke.dk>
 <YNyopHsKX2m5HSRr@makrotopia.org> <87h7hf139u.fsf@toke.dk>
 <YOHCdnZoAKBhNIF8@makrotopia.org> <87zgv0yefe.fsf@toke.dk>
 <YOMtyz6GJdjJ4xuy@makrotopia.org>
Message-ID: <87v95oy9wh.fsf@toke.dk>

Daniel Golle <daniel at makrotopia.org> writes:

> On Mon, Jul 05, 2021 at 05:21:25PM +0200, Toke H?iland-J?rgensen wrote:
>> Daniel Golle <daniel at makrotopia.org> writes:
>> ...
>> > I have managed to test your solution and it seems to do the job.
>> > Remaining issues:
>> >  * What to do if there are many tunnels all sharing the same upstream
>> >    interface? In this case I'm thinking of doing:
>> >    preserve-dscp wg0 eth0
>> >    preserve-dscp wg1 eth0
>> >    preserve-dscp wg2 eth0
>> >    ...
>> >    But I'm unsure whether this is indented or if further details need
>> >    to be implemented in order to make that work.
>> 
>> Hmm, not sure whether that will work out of the box, actually. Would
>> definitely be doable to make the userspace utility understand how to do
>> this properly, though. There's nothing in principle preventing this from
>> working; the loader should just be smart enough to do incremental
>> loading of multiple "ingress" programs while still sharing the map
>> between all of them.
>
> You make it at least sound easy :)

I'd say the implementation is relatively straight-forward for anyone
familiar with how BPF works; figuring out how it's *supposed* to work is
the hard bit ;)

>> The only potential operational issue with using it on multiple wg
>> interfaces is if they share IP space; because in that case you might
>> have packets from different tunnels ending up with identical hashes,
>> confusing the egress side. Fixing this would require the outer BPF
>> program to know about wg endpoint addresses and map the packets back to
>> their inner ifindexes using that. But as long as the wireguard tunnels
>> are using different IP subnets (or mostly forwarding traffic without the
>> inner addresses as sources or destinations), the hash collision
>> probability should not be bigger than just traffic on a single tunnel, I
>> suppose.
>> 
>> One particular thing to watch out for here is IPv6 link-local traffic;
>> sine wg doesn't generate link-local addresses automatically, they are
>> commonly configured with (the same) static address (like fe80::1 or
>> fe80::2), which would make link-local traffic identical across wg
>> interfaces. But this is only used for particular setups (I use it for
>> running Babel over wg, for instance), just make sure it won't be an
>> issue for your deployment scenario :)
>
> All this is good to know, but from what I can see now shouldn't be
> a problem in our deployment -- it's multiple wireguard links which are
> (using fwmark and ip rules) routed over several uplinks. We then use
> mwan3 to balance most of the gateway traffic accross the available
> wireguard interfaces, using MASQ/SNAT on each tunnel which has a
> unique transfer network assigned, and no IPv6 at all.
> Hence it should be ok to go under the restrictions you described.

Alright, so the wireguard-to-physical interfaces is always many-to-one?
I.e., each wireguard interface is always routed out the same physical
interface, but there may be multiple wg interfaces sharing the same
uplink?

I'm asking because in that case it does make sense to keep separate
instances of the whole setup per physical interface to limit hash
collisions; otherwise, the lookup table could also be made global and
shared between all physical interfaces, so you'd avoid having to specify
the relationship explicitly...

>> >  * Once a wireguard interface goes down, one cannot unload the
>> >    remaining program on the upstream interface, as
>> >    preserve-dscp wg0 eth0 --unload
>> >    would fail in case of 'wg0' having gone missing.
>> >    What do you suggest to do in this case?
>> 
>> Just fixing the userspace utility to deal with this case properly as
>> well is probably the easiest. How are you thinking you'd deploy this?
>> Via ifup hooks on openwrt, or something different?
>
> Yes, I use ifup hooks configured in an init script for procd and have
> it tied to the wireguard config sections in /etc/config/network:
>
> https://git.openwrt.org/?p=openwrt/staging/dangole.git;a=blob;f=package/network/utils/bpf-examples/files/wireguard-preserve-dscp.init;h=f1e5e25e663308e057285e2bd8e3bcb9560bdd54;hb=5923a78d74be3f05e734b0be0a832a87be8d369b#l56
>
> Passing multiple inner interfaces to one call to the to-be-modified
> preserve-dscp tool could be achieved by some shell magic dealing with
> the configuration...

Not necessary: it's perfectly fine to attach them one at a time.

> We will have to restart the filter for all inner interfaces in case of
> one being added or removed, right?

Nope, that's no necessary either. We can just re-attach the same filter
program to each additional interface.

> And maybe I'll come up with some state tracking so orphaned filters can
> be removed after configuration changes...

The userspace loader could be made to detect this and automatically
clean up the program on the physical interface after the last internal
interface goes away. At least as long as we can rely on an ifdown hook
this will be fairly straight-forward (just requires a lock to not be
racy). Detecting it after interfaces are automatically removed from the
kernel is a bit more cumbersome as it would require some way to trigger
the garbage collection.

-Toke

From daniel at makrotopia.org  Mon Jul  5 17:26:32 2021
From: daniel at makrotopia.org (Daniel Golle)
Date: Mon, 5 Jul 2021 18:26:32 +0100
Subject: passing-through TOS/DSCP marking
In-Reply-To: <87v95oy9wh.fsf@toke.dk>
References: <877disdre0.fsf@toke.dk>
 <CAHmME9qnqixzDKegMprkT8xNXHfbrhGZfwAROp6vjrQLJEo87g@mail.gmail.com>
 <YNCHs1wKWEoNxUyX@makrotopia.org> <877dinths3.fsf@toke.dk>
 <YNyopHsKX2m5HSRr@makrotopia.org> <87h7hf139u.fsf@toke.dk>
 <YOHCdnZoAKBhNIF8@makrotopia.org> <87zgv0yefe.fsf@toke.dk>
 <YOMtyz6GJdjJ4xuy@makrotopia.org> <87v95oy9wh.fsf@toke.dk>
Message-ID: <YONAyApKnaECA5RB@makrotopia.org>

Hi Toke,

On Mon, Jul 05, 2021 at 06:59:10PM +0200, Toke H?iland-J?rgensen wrote:
> Daniel Golle <daniel at makrotopia.org> writes:
> > ...
> >> The only potential operational issue with using it on multiple wg
> >> interfaces is if they share IP space; because in that case you might
> >> have packets from different tunnels ending up with identical hashes,
> >> confusing the egress side. Fixing this would require the outer BPF
> >> program to know about wg endpoint addresses and map the packets back to
> >> their inner ifindexes using that. But as long as the wireguard tunnels
> >> are using different IP subnets (or mostly forwarding traffic without the
> >> inner addresses as sources or destinations), the hash collision
> >> probability should not be bigger than just traffic on a single tunnel, I
> >> suppose.
> >> 
> >> One particular thing to watch out for here is IPv6 link-local traffic;
> >> sine wg doesn't generate link-local addresses automatically, they are
> >> commonly configured with (the same) static address (like fe80::1 or
> >> fe80::2), which would make link-local traffic identical across wg
> >> interfaces. But this is only used for particular setups (I use it for
> >> running Babel over wg, for instance), just make sure it won't be an
> >> issue for your deployment scenario :)
> >
> > All this is good to know, but from what I can see now shouldn't be
> > a problem in our deployment -- it's multiple wireguard links which are
> > (using fwmark and ip rules) routed over several uplinks. We then use
> > mwan3 to balance most of the gateway traffic accross the available
> > wireguard interfaces, using MASQ/SNAT on each tunnel which has a
> > unique transfer network assigned, and no IPv6 at all.
> > Hence it should be ok to go under the restrictions you described.
> 
> Alright, so the wireguard-to-physical interfaces is always many-to-one?
> I.e., each wireguard interface is always routed out the same physical
> interface, but there may be multiple wg interfaces sharing the same
> uplink?

Well, on the access concentrator in the datacentre this is the case:
All wireguard tunnels are using the same interface.

On the remote system there are *many* tunnels to the same access
concentrator, each routed over a different uplink interface.
So there it's a 1:1 mapping, each wgX has it's distinct ethX.
(and there the current solution already works fine)

> 
> I'm asking because in that case it does make sense to keep separate
> instances of the whole setup per physical interface to limit hash
> collisions; otherwise, the lookup table could also be made global and
> shared between all physical interfaces, so you'd avoid having to specify
> the relationship explicitly...
> 
> >> >  * Once a wireguard interface goes down, one cannot unload the
> >> >    remaining program on the upstream interface, as
> >> >    preserve-dscp wg0 eth0 --unload
> >> >    would fail in case of 'wg0' having gone missing.
> >> >    What do you suggest to do in this case?
> >> 
> >> Just fixing the userspace utility to deal with this case properly as
> >> well is probably the easiest. How are you thinking you'd deploy this?
> >> Via ifup hooks on openwrt, or something different?
> >
> > Yes, I use ifup hooks configured in an init script for procd and have
> > it tied to the wireguard config sections in /etc/config/network:
> >
> > https://git.openwrt.org/?p=openwrt/staging/dangole.git;a=blob;f=package/network/utils/bpf-examples/files/wireguard-preserve-dscp.init;h=f1e5e25e663308e057285e2bd8e3bcb9560bdd54;hb=5923a78d74be3f05e734b0be0a832a87be8d369b#l56
> >
> > Passing multiple inner interfaces to one call to the to-be-modified
> > preserve-dscp tool could be achieved by some shell magic dealing with
> > the configuration...
> 
> Not necessary: it's perfectly fine to attach them one at a time.

So assume we changed the userspace tool to accept multiple inner
interfaces, let's say we called:
preserve-dscp wg0 wg1 wg2 eth0

And then, at some later point in time we want to add 'wg3'.
So calling
preserve-dscp wg0 wg1 wg2 wg3 eth0
could just work without interrupting ongoing service on wg0..wg2?
That would definitely require the userspace tool to track some local
state and store it in /var/lib/foo/...? Or am I getting something
wrong here?

> 
> > We will have to restart the filter for all inner interfaces in case of
> > one being added or removed, right?
> 
> Nope, that's no necessary either. We can just re-attach the same filter
> program to each additional interface.

So given the above example, I could then just call
preserve-dscp wg3 eth0
to add eth3 while wg0..wg2 keep working?

> 
> > And maybe I'll come up with some state tracking so orphaned filters can
> > be removed after configuration changes...
> 
> The userspace loader could be made to detect this and automatically
> clean up the program on the physical interface after the last internal
> interface goes away. At least as long as we can rely on an ifdown hook
> this will be fairly straight-forward (just requires a lock to not be
> racy). Detecting it after interfaces are automatically removed from the
> kernel is a bit more cumbersome as it would require some way to trigger
> the garbage collection.

I'll look into that and how we are intending to handle that in
general in OpenWrt. John was working on that I believe, I'll ask him
first.

From toke at toke.dk  Mon Jul  5 21:20:40 2021
From: toke at toke.dk (Toke =?utf-8?Q?H=C3=B8iland-J=C3=B8rgensen?=)
Date: Mon, 05 Jul 2021 23:20:40 +0200
Subject: passing-through TOS/DSCP marking
In-Reply-To: <YONAyApKnaECA5RB@makrotopia.org>
References: <877disdre0.fsf@toke.dk>
 <CAHmME9qnqixzDKegMprkT8xNXHfbrhGZfwAROp6vjrQLJEo87g@mail.gmail.com>
 <YNCHs1wKWEoNxUyX@makrotopia.org> <877dinths3.fsf@toke.dk>
 <YNyopHsKX2m5HSRr@makrotopia.org> <87h7hf139u.fsf@toke.dk>
 <YOHCdnZoAKBhNIF8@makrotopia.org> <87zgv0yefe.fsf@toke.dk>
 <YOMtyz6GJdjJ4xuy@makrotopia.org> <87v95oy9wh.fsf@toke.dk>
 <YONAyApKnaECA5RB@makrotopia.org>
Message-ID: <87pmvwxxsn.fsf@toke.dk>

Daniel Golle <daniel at makrotopia.org> writes:

> Hi Toke,
>
> On Mon, Jul 05, 2021 at 06:59:10PM +0200, Toke H?iland-J?rgensen wrote:
>> Daniel Golle <daniel at makrotopia.org> writes:
>> > ...
>> >> The only potential operational issue with using it on multiple wg
>> >> interfaces is if they share IP space; because in that case you might
>> >> have packets from different tunnels ending up with identical hashes,
>> >> confusing the egress side. Fixing this would require the outer BPF
>> >> program to know about wg endpoint addresses and map the packets back to
>> >> their inner ifindexes using that. But as long as the wireguard tunnels
>> >> are using different IP subnets (or mostly forwarding traffic without the
>> >> inner addresses as sources or destinations), the hash collision
>> >> probability should not be bigger than just traffic on a single tunnel, I
>> >> suppose.
>> >> 
>> >> One particular thing to watch out for here is IPv6 link-local traffic;
>> >> sine wg doesn't generate link-local addresses automatically, they are
>> >> commonly configured with (the same) static address (like fe80::1 or
>> >> fe80::2), which would make link-local traffic identical across wg
>> >> interfaces. But this is only used for particular setups (I use it for
>> >> running Babel over wg, for instance), just make sure it won't be an
>> >> issue for your deployment scenario :)
>> >
>> > All this is good to know, but from what I can see now shouldn't be
>> > a problem in our deployment -- it's multiple wireguard links which are
>> > (using fwmark and ip rules) routed over several uplinks. We then use
>> > mwan3 to balance most of the gateway traffic accross the available
>> > wireguard interfaces, using MASQ/SNAT on each tunnel which has a
>> > unique transfer network assigned, and no IPv6 at all.
>> > Hence it should be ok to go under the restrictions you described.
>> 
>> Alright, so the wireguard-to-physical interfaces is always many-to-one?
>> I.e., each wireguard interface is always routed out the same physical
>> interface, but there may be multiple wg interfaces sharing the same
>> uplink?
>
> Well, on the access concentrator in the datacentre this is the case:
> All wireguard tunnels are using the same interface.
>
> On the remote system there are *many* tunnels to the same access
> concentrator, each routed over a different uplink interface.
> So there it's a 1:1 mapping, each wgX has it's distinct ethX.
> (and there the current solution already works fine)

Alright. The important thing here is that no single wg interface splits
its traffic over multiple uplinks, so that's all fine :)

>> I'm asking because in that case it does make sense to keep separate
>> instances of the whole setup per physical interface to limit hash
>> collisions; otherwise, the lookup table could also be made global and
>> shared between all physical interfaces, so you'd avoid having to specify
>> the relationship explicitly...
>> 
>> >> >  * Once a wireguard interface goes down, one cannot unload the
>> >> >    remaining program on the upstream interface, as
>> >> >    preserve-dscp wg0 eth0 --unload
>> >> >    would fail in case of 'wg0' having gone missing.
>> >> >    What do you suggest to do in this case?
>> >> 
>> >> Just fixing the userspace utility to deal with this case properly as
>> >> well is probably the easiest. How are you thinking you'd deploy this?
>> >> Via ifup hooks on openwrt, or something different?
>> >
>> > Yes, I use ifup hooks configured in an init script for procd and have
>> > it tied to the wireguard config sections in /etc/config/network:
>> >
>> > https://git.openwrt.org/?p=openwrt/staging/dangole.git;a=blob;f=package/network/utils/bpf-examples/files/wireguard-preserve-dscp.init;h=f1e5e25e663308e057285e2bd8e3bcb9560bdd54;hb=5923a78d74be3f05e734b0be0a832a87be8d369b#l56
>> >
>> > Passing multiple inner interfaces to one call to the to-be-modified
>> > preserve-dscp tool could be achieved by some shell magic dealing with
>> > the configuration...
>> 
>> Not necessary: it's perfectly fine to attach them one at a time.
>
> So assume we changed the userspace tool to accept multiple inner
> interfaces, let's say we called:
> preserve-dscp wg0 wg1 wg2 eth0
>
> And then, at some later point in time we want to add 'wg3'.
> So calling
> preserve-dscp wg0 wg1 wg2 wg3 eth0
> could just work without interrupting ongoing service on wg0..wg2?
> That would definitely require the userspace tool to track some local
> state and store it in /var/lib/foo/...? Or am I getting something
> wrong here?

Not necessarily; we can extract the information we need from the kernel:

The whole thing works by having one BPF program that extracts the DSCP
value on the wg interface and puts it into a map, keyed by the packet
hash; and then the other program on the physical interface reads that
same map and looks up packet hashes in it. So what this means it that
the critical thing is that the two BPF programs share the same map, or
the information won't be there.

In other words, when loading the DSCP-parsing program on a second wg
interface, the essential bit is that it ends up using the same map as
the already-loaded program on the physical interface. Either by loading
a second copy of the BPF program and re-using the existing map, or by
just attaching the same program to multiple interfaces.

As for how we find the map (or program) to attach to the second
interface, we can either "pin" the reference in a special bpffs and load
it from there, or we can use the kernel introspection APIs to discover
which map ID is currently used by the program on the physical interface,
and then open that map by ID (if just reusing the map), or iterate
through existing BPF programs looking for one which uses this map, then
attaching that to a second interface. Using bpffs is slightly more
convenient (doesn't require iterating programs), but has the drawback
that it adds a dependency on the bpffs itself, and that it needs an
explicit removal step (unlinking the pinned reference).

No matter which mechanism is used, the loading of the parsing program on
a second wg interface is completely transparent to any program already
running: it'll just be another source of data going into the map that
the physical interface program can then read. Think of it as multiple
clients writing to the same database: as long as they agree on which
database they're using, clients can come and go without interfering with
each other.

>> > We will have to restart the filter for all inner interfaces in case of
>> > one being added or removed, right?
>> 
>> Nope, that's no necessary either. We can just re-attach the same filter
>> program to each additional interface.
>
> So given the above example, I could then just call
> preserve-dscp wg3 eth0
> to add eth3 while wg0..wg2 keep working?

Yup, see above.

>> > And maybe I'll come up with some state tracking so orphaned filters can
>> > be removed after configuration changes...
>> 
>> The userspace loader could be made to detect this and automatically
>> clean up the program on the physical interface after the last internal
>> interface goes away. At least as long as we can rely on an ifdown hook
>> this will be fairly straight-forward (just requires a lock to not be
>> racy). Detecting it after interfaces are automatically removed from the
>> kernel is a bit more cumbersome as it would require some way to trigger
>> the garbage collection.
>
> I'll look into that and how we are intending to handle that in
> general in OpenWrt. John was working on that I believe, I'll ask him
> first.

OK, cool.

-Toke

From nextgens at freenetproject.org  Tue Jul  6 07:00:14 2021
From: nextgens at freenetproject.org (Florent Daigniere)
Date: Tue, 6 Jul 2021 07:00:14 +0000
Subject: passing-through TOS/DSCP marking
In-Reply-To: <CAHmME9omR=NXkJj-AJviWhcDDY=X9vQbLxn5WbeamUGhVWHhJw@mail.gmail.com>
References: <YMn7qQ/7s/CciRNy@makrotopia.org>
 <CAHmME9omR=NXkJj-AJviWhcDDY=X9vQbLxn5WbeamUGhVWHhJw@mail.gmail.com>
Message-ID: <0102017a7a9e0329-2cf98ec6-3874-4f0e-80f0-663792b43e3b-000000@eu-west-1.amazonses.com>

On Wed, 2021-06-16 at 18:28 +0200, Jason A. Donenfeld wrote:
> WireGuard does not copy the inner DSCP mark to the outside, aside from
> the ECN bits, in order to avoid a data leak.
> 
> Jason

Hi Jason,

Is there any room for revisiting this design decision? We are talking
about 6 bits of metadata per packet here...

Which realistic threats are we trying to protect against?

The solutions that don't involve code changes all have significant
drawbacks:
- awesome BPF-based magic will be Linux only
- multiple tunnels are not always practical and arguably worse traffic
correlation-wise.

I still use a patched wireguard to protect traffic from a voip app on an
android handset using wifi here... and while I have a solution that's
good enough for my requirements, I do think that the community would
benefit from having something that works better out of the box (and on
all platforms).

Florent


From luizluca at gmail.com  Tue Jul  6 20:08:00 2021
From: luizluca at gmail.com (Luiz Angelo Daros de Luca)
Date: Tue, 6 Jul 2021 17:08:00 -0300
Subject: passing-through TOS/DSCP marking
In-Reply-To: <0102017a7a9e0329-2cf98ec6-3874-4f0e-80f0-663792b43e3b-000000@eu-west-1.amazonses.com>
References: <YMn7qQ/7s/CciRNy@makrotopia.org>
 <CAHmME9omR=NXkJj-AJviWhcDDY=X9vQbLxn5WbeamUGhVWHhJw@mail.gmail.com>
 <0102017a7a9e0329-2cf98ec6-3874-4f0e-80f0-663792b43e3b-000000@eu-west-1.amazonses.com>
Message-ID: <CAJq09z5fhE=Rj_wTs5x1zYSu3e0+tU-huymEVV5Y6Bfyu_7fJw@mail.gmail.com>

I see talented people spending resources trying to go around a
wireguard design decision. I understand that wireguard try to keep it
as safe as possible and as simple as possible. However, passing some
traffic information to the encrypted packet is a requirement for some
setups. Wouldn't it be better to have it provided by wireguard but
disabled by default? If the change don't kill a kitty, it will not
harm the security as users are already doing the same but through a
much harder way.

My 2 cents.

---
     Luiz Angelo Daros de Luca
            luizluca at gmail.com

From huzhengmian at gmail.com  Thu Jul  8 04:52:50 2021
From: huzhengmian at gmail.com (zhengmian hu)
Date: Thu, 8 Jul 2021 00:52:50 -0400
Subject: Wrong route on macOS
Message-ID: <CAAntei-TBN5VUpnHLNExgg_4V80zU+PMAHcjVOqfmDbeJ3G1aA@mail.gmail.com>

I expect no data to be sent to the server when I only visit the local
address. However, this is not true for wireguard on macOS.

Detailed setup:
App:
Downloaded from app store. App version: 1.0.13 (24), Go backend
version: f9b48a96.

Server:
```
[Interface]
Address = 10.110.210.1/24
SaveConfig = true
ListenPort = 51820
PrivateKey = xxxxSERVER_PRIVATExxxx

[Peer]
PublicKey = xxxxCLIENT_PUBLICxxxx
AllowedIPs = 10.110.210.7/32
Endpoint = xxxx
```

Client:
```
[Interface]
PrivateKey = xxxxCLIENT_PRIVATExxxx
Address = 10.110.210.7/32

[Peer]
PublicKey = xxxxSERVER_PUBLICxxxx
AllowedIPs = 10.110.210.1/24
Endpoint = xxxSERVER_IPxxx:51820
PersistentKeepalive = 25
```

Test:
```
? traceroute 10.110.210.7
traceroute to 10.110.210.7 (10.110.210.7), 64 hops max, 52 byte packets
 1  10.110.210.1 (10.110.210.1)  22.118 ms  19.846 ms  19.929 ms
 2  10.110.210.7 (10.110.210.7)  40.627 ms  40.264 ms  43.752 ms
```

Route:
```
? netstat -rn -f inet
Routing tables

Internet:
Destination        Gateway            Flags        Netif Expire
default            192.168.0.1        UGScg          en0
default            link#17            UCSIg        utun2
10.110.210/24      link#17            UCS          utun2
10.110.210.7       10.110.210.7       UH           utun2
127                127.0.0.1          UCS            lo0
127.0.0.1          127.0.0.1          UH             lo0
...
255.255.255.255/32 link#17            UCSI         utun2
```

Expected behaviour:
When using `traceroute 10.110.210.7`, the server `10.110.210.1` doesn't show up.

Other:
I have tried the same configuration on Windows, and it works fine.

From huzhengmian at gmail.com  Thu Jul  8 05:07:32 2021
From: huzhengmian at gmail.com (zhengmian hu)
Date: Thu, 8 Jul 2021 01:07:32 -0400
Subject: Wrong route on macOS
In-Reply-To: <CAAntei-TBN5VUpnHLNExgg_4V80zU+PMAHcjVOqfmDbeJ3G1aA@mail.gmail.com>
References: <CAAntei-TBN5VUpnHLNExgg_4V80zU+PMAHcjVOqfmDbeJ3G1aA@mail.gmail.com>
Message-ID: <CAAntei9qYK-cJgGzj9y0RTSLAqG54EnO2_9GDCpOmA5htNQcdg@mail.gmail.com>

The macOS version is 11.4 (20F71).
I also found that when the client config contains two peers, one of
them just doesn't work.

zhengmian hu <huzhengmian at gmail.com> ?2021?7?8??? ??12:52???
>
> I expect no data to be sent to the server when I only visit the local
> address. However, this is not true for wireguard on macOS.
>
> Detailed setup:
> App:
> Downloaded from app store. App version: 1.0.13 (24), Go backend
> version: f9b48a96.
>
> Server:
> ```
> [Interface]
> Address = 10.110.210.1/24
> SaveConfig = true
> ListenPort = 51820
> PrivateKey = xxxxSERVER_PRIVATExxxx
>
> [Peer]
> PublicKey = xxxxCLIENT_PUBLICxxxx
> AllowedIPs = 10.110.210.7/32
> Endpoint = xxxx
> ```
>
> Client:
> ```
> [Interface]
> PrivateKey = xxxxCLIENT_PRIVATExxxx
> Address = 10.110.210.7/32
>
> [Peer]
> PublicKey = xxxxSERVER_PUBLICxxxx
> AllowedIPs = 10.110.210.1/24
> Endpoint = xxxSERVER_IPxxx:51820
> PersistentKeepalive = 25
> ```
>
> Test:
> ```
> ? traceroute 10.110.210.7
> traceroute to 10.110.210.7 (10.110.210.7), 64 hops max, 52 byte packets
>  1  10.110.210.1 (10.110.210.1)  22.118 ms  19.846 ms  19.929 ms
>  2  10.110.210.7 (10.110.210.7)  40.627 ms  40.264 ms  43.752 ms
> ```
>
> Route:
> ```
> ? netstat -rn -f inet
> Routing tables
>
> Internet:
> Destination        Gateway            Flags        Netif Expire
> default            192.168.0.1        UGScg          en0
> default            link#17            UCSIg        utun2
> 10.110.210/24      link#17            UCS          utun2
> 10.110.210.7       10.110.210.7       UH           utun2
> 127                127.0.0.1          UCS            lo0
> 127.0.0.1          127.0.0.1          UH             lo0
> ...
> 255.255.255.255/32 link#17            UCSI         utun2
> ```
>
> Expected behaviour:
> When using `traceroute 10.110.210.7`, the server `10.110.210.1` doesn't show up.
>
> Other:
> I have tried the same configuration on Windows, and it works fine.

From rdunlap at infradead.org  Fri Jul  9 02:17:43 2021
From: rdunlap at infradead.org (Randy Dunlap)
Date: Thu,  8 Jul 2021 19:17:43 -0700
Subject: [PATCH 2/6] hw_random: rename 'mod_init' & 'mod_exit' functions to be
 module-specific
In-Reply-To: <20210709021747.32737-1-rdunlap@infradead.org>
References: <20210709021747.32737-1-rdunlap@infradead.org>
Message-ID: <20210709021747.32737-3-rdunlap@infradead.org>

Rename module_init & module_exit functions that are named
"mod_init" and "mod_exit" so that they are unique in both the
System.map file and in initcall_debug output instead of showing
up as almost anonymous "mod_init".

This is helpful for debugging and in determining how long certain
module_init calls take to execute.

Signed-off-by: Randy Dunlap <rdunlap at infradead.org>
Cc: Arnd Bergmann <arnd at arndb.de>
Cc: Greg Kroah-Hartman <gregkh at linuxfoundation.org>
Cc: Andres Salomon <dilinger at queued.net>
Cc: linux-geode at lists.infradead.org
Cc: Matt Mackall <mpm at selenic.com>
Cc: Herbert Xu <herbert at gondor.apana.org.au>
Cc: "David S. Miller" <davem at davemloft.net>
Cc: linux-crypto at vger.kernel.org
Cc: Jason A. Donenfeld <Jason at zx2c4.com>
---
 drivers/char/hw_random/amd-rng.c   |    8 ++++----
 drivers/char/hw_random/geode-rng.c |    8 ++++----
 drivers/char/hw_random/intel-rng.c |    8 ++++----
 drivers/char/hw_random/via-rng.c   |    8 ++++----
 4 files changed, 16 insertions(+), 16 deletions(-)

--- linux-next-20210708.orig/drivers/char/hw_random/amd-rng.c
+++ linux-next-20210708/drivers/char/hw_random/amd-rng.c
@@ -124,7 +124,7 @@ static struct hwrng amd_rng = {
 	.read		= amd_rng_read,
 };
 
-static int __init mod_init(void)
+static int __init amd_rng_mod_init(void)
 {
 	int err;
 	struct pci_dev *pdev = NULL;
@@ -188,7 +188,7 @@ out:
 	return err;
 }
 
-static void __exit mod_exit(void)
+static void __exit amd_rng_mod_exit(void)
 {
 	struct amd768_priv *priv;
 
@@ -203,8 +203,8 @@ static void __exit mod_exit(void)
 	kfree(priv);
 }
 
-module_init(mod_init);
-module_exit(mod_exit);
+module_init(amd_rng_mod_init);
+module_exit(amd_rng_mod_exit);
 
 MODULE_AUTHOR("The Linux Kernel team");
 MODULE_DESCRIPTION("H/W RNG driver for AMD chipsets");
--- linux-next-20210708.orig/drivers/char/hw_random/geode-rng.c
+++ linux-next-20210708/drivers/char/hw_random/geode-rng.c
@@ -83,7 +83,7 @@ static struct hwrng geode_rng = {
 };
 
 
-static int __init mod_init(void)
+static int __init geode_rng_init(void)
 {
 	int err = -ENODEV;
 	struct pci_dev *pdev = NULL;
@@ -124,7 +124,7 @@ err_unmap:
 	goto out;
 }
 
-static void __exit mod_exit(void)
+static void __exit geode_rng_exit(void)
 {
 	void __iomem *mem = (void __iomem *)geode_rng.priv;
 
@@ -132,8 +132,8 @@ static void __exit mod_exit(void)
 	iounmap(mem);
 }
 
-module_init(mod_init);
-module_exit(mod_exit);
+module_init(geode_rng_init);
+module_exit(geode_rng_exit);
 
 MODULE_DESCRIPTION("H/W RNG driver for AMD Geode LX CPUs");
 MODULE_LICENSE("GPL");
--- linux-next-20210708.orig/drivers/char/hw_random/intel-rng.c
+++ linux-next-20210708/drivers/char/hw_random/intel-rng.c
@@ -325,7 +325,7 @@ PFX "RNG, try using the 'no_fwh_detect'
 }
 
 
-static int __init mod_init(void)
+static int __init intel_rng_mod_init(void)
 {
 	int err = -ENODEV;
 	int i;
@@ -403,7 +403,7 @@ out:
 
 }
 
-static void __exit mod_exit(void)
+static void __exit intel_rng_mod_exit(void)
 {
 	void __iomem *mem = (void __iomem *)intel_rng.priv;
 
@@ -411,8 +411,8 @@ static void __exit mod_exit(void)
 	iounmap(mem);
 }
 
-module_init(mod_init);
-module_exit(mod_exit);
+module_init(intel_rng_mod_init);
+module_exit(intel_rng_mod_exit);
 
 MODULE_DESCRIPTION("H/W RNG driver for Intel chipsets");
 MODULE_LICENSE("GPL");
--- linux-next-20210708.orig/drivers/char/hw_random/via-rng.c
+++ linux-next-20210708/drivers/char/hw_random/via-rng.c
@@ -192,7 +192,7 @@ static struct hwrng via_rng = {
 };
 
 
-static int __init mod_init(void)
+static int __init via_rng_mod_init(void)
 {
 	int err;
 
@@ -209,13 +209,13 @@ static int __init mod_init(void)
 out:
 	return err;
 }
-module_init(mod_init);
+module_init(via_rng_mod_init);
 
-static void __exit mod_exit(void)
+static void __exit via_rng_mod_exit(void)
 {
 	hwrng_unregister(&via_rng);
 }
-module_exit(mod_exit);
+module_exit(via_rng_mod_exit);
 
 static struct x86_cpu_id __maybe_unused via_rng_cpu_id[] = {
 	X86_MATCH_FEATURE(X86_FEATURE_XSTORE, NULL),

From rdunlap at infradead.org  Fri Jul  9 02:17:45 2021
From: rdunlap at infradead.org (Randy Dunlap)
Date: Thu,  8 Jul 2021 19:17:45 -0700
Subject: [PATCH 4/6] MOST: cdev: rename 'mod_init' & 'mod_exit' functions to
 be module-specific
In-Reply-To: <20210709021747.32737-1-rdunlap@infradead.org>
References: <20210709021747.32737-1-rdunlap@infradead.org>
Message-ID: <20210709021747.32737-5-rdunlap@infradead.org>

Rename module_init & module_exit functions that are named
"mod_init" and "mod_exit" so that they are unique in both the
System.map file and in initcall_debug output instead of showing
up as almost anonymous "mod_init".

This is helpful for debugging and in determining how long certain
module_init calls take to execute.

Signed-off-by: Randy Dunlap <rdunlap at infradead.org>
Cc: Christian Gromm <christian.gromm at microchip.com>
Cc: Greg Kroah-Hartman <gregkh at linuxfoundation.org>
---
 drivers/most/most_cdev.c |    8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)

--- linux-next-20210708.orig/drivers/most/most_cdev.c
+++ linux-next-20210708/drivers/most/most_cdev.c
@@ -486,7 +486,7 @@ static struct cdev_component comp = {
 	},
 };
 
-static int __init mod_init(void)
+static int __init most_cdev_init(void)
 {
 	int err;
 
@@ -518,7 +518,7 @@ dest_ida:
 	return err;
 }
 
-static void __exit mod_exit(void)
+static void __exit most_cdev_exit(void)
 {
 	struct comp_channel *c, *tmp;
 
@@ -534,8 +534,8 @@ static void __exit mod_exit(void)
 	class_destroy(comp.class);
 }
 
-module_init(mod_init);
-module_exit(mod_exit);
+module_init(most_cdev_init);
+module_exit(most_cdev_exit);
 MODULE_AUTHOR("Christian Gromm <christian.gromm at microchip.com>");
 MODULE_LICENSE("GPL");
 MODULE_DESCRIPTION("character device component for mostcore");

From rdunlap at infradead.org  Fri Jul  9 02:17:44 2021
From: rdunlap at infradead.org (Randy Dunlap)
Date: Thu,  8 Jul 2021 19:17:44 -0700
Subject: [PATCH 3/6] lib: crypto: rename 'mod_init' & 'mod_exit' functions to
 be module-specific
In-Reply-To: <20210709021747.32737-1-rdunlap@infradead.org>
References: <20210709021747.32737-1-rdunlap@infradead.org>
Message-ID: <20210709021747.32737-4-rdunlap@infradead.org>

Rename module_init & module_exit functions that are named
"mod_init" and "mod_exit" so that they are unique in both the
System.map file and in initcall_debug output instead of showing
up as almost anonymous "mod_init".

This is helpful for debugging and in determining how long certain
module_init calls take to execute.

Signed-off-by: Randy Dunlap <rdunlap at infradead.org>
Cc: Herbert Xu <herbert at gondor.apana.org.au>
Cc: "David S. Miller" <davem at davemloft.net>
Cc: linux-crypto at vger.kernel.org
Cc: Jason A. Donenfeld <Jason at zx2c4.com>
---
 lib/crypto/blake2s.c          |    8 ++++----
 lib/crypto/chacha20poly1305.c |    8 ++++----
 lib/crypto/curve25519.c       |    8 ++++----
 3 files changed, 12 insertions(+), 12 deletions(-)

--- linux-next-20210708.orig/lib/crypto/blake2s.c
+++ linux-next-20210708/lib/crypto/blake2s.c
@@ -73,7 +73,7 @@ void blake2s256_hmac(u8 *out, const u8 *
 }
 EXPORT_SYMBOL(blake2s256_hmac);
 
-static int __init mod_init(void)
+static int __init blake2s_mod_init(void)
 {
 	if (!IS_ENABLED(CONFIG_CRYPTO_MANAGER_DISABLE_TESTS) &&
 	    WARN_ON(!blake2s_selftest()))
@@ -81,12 +81,12 @@ static int __init mod_init(void)
 	return 0;
 }
 
-static void __exit mod_exit(void)
+static void __exit blake2s_mod_exit(void)
 {
 }
 
-module_init(mod_init);
-module_exit(mod_exit);
+module_init(blake2s_mod_init);
+module_exit(blake2s_mod_exit);
 MODULE_LICENSE("GPL v2");
 MODULE_DESCRIPTION("BLAKE2s hash function");
 MODULE_AUTHOR("Jason A. Donenfeld <Jason at zx2c4.com>");
--- linux-next-20210708.orig/lib/crypto/chacha20poly1305.c
+++ linux-next-20210708/lib/crypto/chacha20poly1305.c
@@ -354,7 +354,7 @@ bool chacha20poly1305_decrypt_sg_inplace
 }
 EXPORT_SYMBOL(chacha20poly1305_decrypt_sg_inplace);
 
-static int __init mod_init(void)
+static int __init chacha20poly1305_init(void)
 {
 	if (!IS_ENABLED(CONFIG_CRYPTO_MANAGER_DISABLE_TESTS) &&
 	    WARN_ON(!chacha20poly1305_selftest()))
@@ -362,12 +362,12 @@ static int __init mod_init(void)
 	return 0;
 }
 
-static void __exit mod_exit(void)
+static void __exit chacha20poly1305_exit(void)
 {
 }
 
-module_init(mod_init);
-module_exit(mod_exit);
+module_init(chacha20poly1305_init);
+module_exit(chacha20poly1305_exit);
 MODULE_LICENSE("GPL v2");
 MODULE_DESCRIPTION("ChaCha20Poly1305 AEAD construction");
 MODULE_AUTHOR("Jason A. Donenfeld <Jason at zx2c4.com>");
--- linux-next-20210708.orig/lib/crypto/curve25519.c
+++ linux-next-20210708/lib/crypto/curve25519.c
@@ -13,7 +13,7 @@
 #include <linux/module.h>
 #include <linux/init.h>
 
-static int __init mod_init(void)
+static int __init curve25519_init(void)
 {
 	if (!IS_ENABLED(CONFIG_CRYPTO_MANAGER_DISABLE_TESTS) &&
 	    WARN_ON(!curve25519_selftest()))
@@ -21,12 +21,12 @@ static int __init mod_init(void)
 	return 0;
 }
 
-static void __exit mod_exit(void)
+static void __exit curve25519_exit(void)
 {
 }
 
-module_init(mod_init);
-module_exit(mod_exit);
+module_init(curve25519_init);
+module_exit(curve25519_exit);
 
 MODULE_LICENSE("GPL v2");
 MODULE_DESCRIPTION("Curve25519 scalar multiplication");

From rdunlap at infradead.org  Fri Jul  9 02:17:42 2021
From: rdunlap at infradead.org (Randy Dunlap)
Date: Thu,  8 Jul 2021 19:17:42 -0700
Subject: [PATCH 1/6] arm: crypto: rename 'mod_init' & 'mod_exit' functions to
 be module-specific
In-Reply-To: <20210709021747.32737-1-rdunlap@infradead.org>
References: <20210709021747.32737-1-rdunlap@infradead.org>
Message-ID: <20210709021747.32737-2-rdunlap@infradead.org>

Rename module_init & module_exit functions that are named
"mod_init" and "mod_exit" so that they are unique in both the
System.map file and in initcall_debug output instead of showing
up as almost anonymous "mod_init".

This is helpful for debugging and in determining how long certain
module_init calls take to execute.

Signed-off-by: Randy Dunlap <rdunlap at infradead.org>
Cc: Jason A. Donenfeld <Jason at zx2c4.com>
Cc: Russell King <linux at armlinux.org.uk>
Cc: linux-arm-kernel at lists.infradead.org
Cc: patches at armlinux.org.uk
---
 arch/arm/crypto/curve25519-glue.c |    8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)

--- linux-next-20210708.orig/arch/arm/crypto/curve25519-glue.c
+++ linux-next-20210708/arch/arm/crypto/curve25519-glue.c
@@ -112,7 +112,7 @@ static struct kpp_alg curve25519_alg = {
 	.max_size		= curve25519_max_size,
 };
 
-static int __init mod_init(void)
+static int __init arm_curve25519_init(void)
 {
 	if (elf_hwcap & HWCAP_NEON) {
 		static_branch_enable(&have_neon);
@@ -122,14 +122,14 @@ static int __init mod_init(void)
 	return 0;
 }
 
-static void __exit mod_exit(void)
+static void __exit arm_curve25519_exit(void)
 {
 	if (IS_REACHABLE(CONFIG_CRYPTO_KPP) && elf_hwcap & HWCAP_NEON)
 		crypto_unregister_kpp(&curve25519_alg);
 }
 
-module_init(mod_init);
-module_exit(mod_exit);
+module_init(arm_curve25519_init);
+module_exit(arm_curve25519_exit);
 
 MODULE_ALIAS_CRYPTO("curve25519");
 MODULE_ALIAS_CRYPTO("curve25519-neon");

From rdunlap at infradead.org  Fri Jul  9 02:17:41 2021
From: rdunlap at infradead.org (Randy Dunlap)
Date: Thu,  8 Jul 2021 19:17:41 -0700
Subject: [PATCH 0/6] treewide: rename 'mod_init' & 'mod_exit' functions to be
 module-specific
Message-ID: <20210709021747.32737-1-rdunlap@infradead.org>

There are multiple (16) modules which use a module_init() function
with the name 'mod_init' and a module_exit() function with the name
'mod_exit'. This can lead to confusion or ambiguity when reading
crashes/oops/bugs etc. and when reading an initcall_debug log.

Example 1: (System.map file)

ffffffff83446d10 t mod_init
ffffffff83446d18 t mod_init
ffffffff83446d20 t mod_init
...
ffffffff83454665 t mod_init
ffffffff834548a4 t mod_init
ffffffff83454a53 t mod_init
...
ffffffff8345bd42 t mod_init
...
ffffffff8345c916 t mod_init
ffffffff8345c92a t mod_init
ffffffff8345c93e t mod_init
ffffffff8345c952 t mod_init
ffffffff8345c966 t mod_init
...
ffffffff834672c9 t mod_init

Example 2: (boot log when using 'initcall_debug')

[    0.252157] initcall mod_init+0x0/0x8 returned 0 after 0 usecs
[    0.252180] initcall mod_init+0x0/0x8 returned 0 after 0 usecs
[    0.252202] initcall mod_init+0x0/0x8 returned 0 after 0 usecs
...
[    0.892907] initcall mod_init+0x0/0x23f returned -19 after 104 usecs
[    0.913788] initcall mod_init+0x0/0x1af returned -19 after 9 usecs
[    0.934353] initcall mod_init+0x0/0x49 returned -19 after 0 usecs
...
[    1.454870] initcall mod_init+0x0/0x66 returned 0 after 72 usecs
...
[    1.455527] initcall mod_init+0x0/0x14 returned 0 after 0 usecs
[    1.455531] initcall mod_init+0x0/0x14 returned 0 after 0 usecs
[    1.455536] initcall mod_init+0x0/0x14 returned 0 after 0 usecs
[    1.455541] initcall mod_init+0x0/0x14 returned 0 after 0 usecs
[    1.455545] initcall mod_init+0x0/0x52 returned 0 after 0 usecs
...
[    1.588162] initcall mod_init+0x0/0xef returned 0 after 45 usecs


Cc: Jason A. Donenfeld <Jason at zx2c4.com>
Cc: Russell King <linux at armlinux.org.uk>
Cc: linux-arm-kernel at lists.infradead.org
Cc: Arnd Bergmann <arnd at arndb.de>
Cc: Greg Kroah-Hartman <gregkh at linuxfoundation.org>
Cc: Andres Salomon <dilinger at queued.net>
Cc: linux-geode at lists.infradead.org
Cc: Matt Mackall <mpm at selenic.com>
Cc: Herbert Xu <herbert at gondor.apana.org.au>
Cc: linux-crypto at vger.kernel.org
Cc: Christian Gromm <christian.gromm at microchip.com>
Cc: Krzysztof Halasa <khc at pm.waw.pl>
Cc: netdev at vger.kernel.org
Cc: "David S. Miller" <davem at davemloft.net>
Cc: Jakub Kicinski <kuba at kernel.org>
Cc: Martin Schiller <ms at dev.tdt.de>
Cc: linux-x25 at vger.kernel.org
Cc: wireguard at lists.zx2c4.com

[PATCH 1/6] arm: crypto: rename 'mod_init' & 'mod_exit' functions to be module-specific
[PATCH 2/6] hw_random: rename 'mod_init' & 'mod_exit' functions to be module-specific
[PATCH 3/6] lib: crypto: rename 'mod_init' & 'mod_exit' functions to be module-specific
[PATCH 4/6] MOST: cdev: rename 'mod_init' & 'mod_exit' functions to be module-specific
[PATCH 5/6] net: hdlc: rename 'mod_init' & 'mod_exit' functions to be module-specific
[PATCH 6/6] net: wireguard: rename 'mod_init' & 'mod_exit' functions to be module-specific

 arch/arm/crypto/curve25519-glue.c  |    8 ++++----
 drivers/char/hw_random/amd-rng.c   |    8 ++++----
 drivers/char/hw_random/geode-rng.c |    8 ++++----
 drivers/char/hw_random/intel-rng.c |    8 ++++----
 drivers/char/hw_random/via-rng.c   |    8 ++++----
 drivers/most/most_cdev.c           |    8 ++++----
 drivers/net/wan/hdlc_cisco.c       |    8 ++++----
 drivers/net/wan/hdlc_fr.c          |    8 ++++----
 drivers/net/wan/hdlc_ppp.c         |    8 ++++----
 drivers/net/wan/hdlc_raw.c         |    8 ++++----
 drivers/net/wan/hdlc_raw_eth.c     |    8 ++++----
 drivers/net/wan/hdlc_x25.c         |    8 ++++----
 drivers/net/wireguard/main.c       |    8 ++++----
 lib/crypto/blake2s.c               |    8 ++++----
 lib/crypto/chacha20poly1305.c      |    8 ++++----
 lib/crypto/curve25519.c            |    8 ++++----
 16 files changed, 64 insertions(+), 64 deletions(-)

From rdunlap at infradead.org  Fri Jul  9 02:17:46 2021
From: rdunlap at infradead.org (Randy Dunlap)
Date: Thu,  8 Jul 2021 19:17:46 -0700
Subject: [PATCH 5/6] net: hdlc: rename 'mod_init' & 'mod_exit' functions to be
 module-specific
In-Reply-To: <20210709021747.32737-1-rdunlap@infradead.org>
References: <20210709021747.32737-1-rdunlap@infradead.org>
Message-ID: <20210709021747.32737-6-rdunlap@infradead.org>

Rename module_init & module_exit functions that are named
"mod_init" and "mod_exit" so that they are unique in both the
System.map file and in initcall_debug output instead of showing
up as almost anonymous "mod_init".

This is helpful for debugging and in determining how long certain
module_init calls take to execute.

Signed-off-by: Randy Dunlap <rdunlap at infradead.org>
Cc: Krzysztof Halasa <khc at pm.waw.pl>
Cc: netdev at vger.kernel.org
Cc: "David S. Miller" <davem at davemloft.net>
Cc: Jakub Kicinski <kuba at kernel.org>
Cc: Martin Schiller <ms at dev.tdt.de>
Cc: linux-x25 at vger.kernel.org
---
 drivers/net/wan/hdlc_cisco.c   |    8 ++++----
 drivers/net/wan/hdlc_fr.c      |    8 ++++----
 drivers/net/wan/hdlc_ppp.c     |    8 ++++----
 drivers/net/wan/hdlc_raw.c     |    8 ++++----
 drivers/net/wan/hdlc_raw_eth.c |    8 ++++----
 drivers/net/wan/hdlc_x25.c     |    8 ++++----
 6 files changed, 24 insertions(+), 24 deletions(-)

--- linux-next-20210708.orig/drivers/net/wan/hdlc_cisco.c
+++ linux-next-20210708/drivers/net/wan/hdlc_cisco.c
@@ -364,19 +364,19 @@ static int cisco_ioctl(struct net_device
 	return -EINVAL;
 }
 
-static int __init mod_init(void)
+static int __init hdlc_cisco_init(void)
 {
 	register_hdlc_protocol(&proto);
 	return 0;
 }
 
-static void __exit mod_exit(void)
+static void __exit hdlc_cisco_exit(void)
 {
 	unregister_hdlc_protocol(&proto);
 }
 
-module_init(mod_init);
-module_exit(mod_exit);
+module_init(hdlc_cisco_init);
+module_exit(hdlc_cisco_exit);
 
 MODULE_AUTHOR("Krzysztof Halasa <khc at pm.waw.pl>");
 MODULE_DESCRIPTION("Cisco HDLC protocol support for generic HDLC");
--- linux-next-20210708.orig/drivers/net/wan/hdlc_fr.c
+++ linux-next-20210708/drivers/net/wan/hdlc_fr.c
@@ -1279,19 +1279,19 @@ static int fr_ioctl(struct net_device *d
 	return -EINVAL;
 }
 
-static int __init mod_init(void)
+static int __init hdlc_fr_init(void)
 {
 	register_hdlc_protocol(&proto);
 	return 0;
 }
 
-static void __exit mod_exit(void)
+static void __exit hdlc_fr_exit(void)
 {
 	unregister_hdlc_protocol(&proto);
 }
 
-module_init(mod_init);
-module_exit(mod_exit);
+module_init(hdlc_fr_init);
+module_exit(hdlc_fr_exit);
 
 MODULE_AUTHOR("Krzysztof Halasa <khc at pm.waw.pl>");
 MODULE_DESCRIPTION("Frame-Relay protocol support for generic HDLC");
--- linux-next-20210708.orig/drivers/net/wan/hdlc_ppp.c
+++ linux-next-20210708/drivers/net/wan/hdlc_ppp.c
@@ -705,20 +705,20 @@ static int ppp_ioctl(struct net_device *
 	return -EINVAL;
 }
 
-static int __init mod_init(void)
+static int __init hdlc_ppp_init(void)
 {
 	skb_queue_head_init(&tx_queue);
 	register_hdlc_protocol(&proto);
 	return 0;
 }
 
-static void __exit mod_exit(void)
+static void __exit hdlc_ppp_exit(void)
 {
 	unregister_hdlc_protocol(&proto);
 }
 
-module_init(mod_init);
-module_exit(mod_exit);
+module_init(hdlc_ppp_init);
+module_exit(hdlc_ppp_exit);
 
 MODULE_AUTHOR("Krzysztof Halasa <khc at pm.waw.pl>");
 MODULE_DESCRIPTION("PPP protocol support for generic HDLC");
--- linux-next-20210708.orig/drivers/net/wan/hdlc_raw.c
+++ linux-next-20210708/drivers/net/wan/hdlc_raw.c
@@ -90,7 +90,7 @@ static int raw_ioctl(struct net_device *
 }
 
 
-static int __init mod_init(void)
+static int __init hdlc_raw_init(void)
 {
 	register_hdlc_protocol(&proto);
 	return 0;
@@ -98,14 +98,14 @@ static int __init mod_init(void)
 
 
 
-static void __exit mod_exit(void)
+static void __exit hdlc_raw_exit(void)
 {
 	unregister_hdlc_protocol(&proto);
 }
 
 
-module_init(mod_init);
-module_exit(mod_exit);
+module_init(hdlc_raw_init);
+module_exit(hdlc_raw_exit);
 
 MODULE_AUTHOR("Krzysztof Halasa <khc at pm.waw.pl>");
 MODULE_DESCRIPTION("Raw HDLC protocol support for generic HDLC");
--- linux-next-20210708.orig/drivers/net/wan/hdlc_raw_eth.c
+++ linux-next-20210708/drivers/net/wan/hdlc_raw_eth.c
@@ -110,7 +110,7 @@ static int raw_eth_ioctl(struct net_devi
 }
 
 
-static int __init mod_init(void)
+static int __init hdlc_eth_init(void)
 {
 	register_hdlc_protocol(&proto);
 	return 0;
@@ -118,14 +118,14 @@ static int __init mod_init(void)
 
 
 
-static void __exit mod_exit(void)
+static void __exit hdlc_eth_exit(void)
 {
 	unregister_hdlc_protocol(&proto);
 }
 
 
-module_init(mod_init);
-module_exit(mod_exit);
+module_init(hdlc_eth_init);
+module_exit(hdlc_eth_exit);
 
 MODULE_AUTHOR("Krzysztof Halasa <khc at pm.waw.pl>");
 MODULE_DESCRIPTION("Ethernet encapsulation support for generic HDLC");
--- linux-next-20210708.orig/drivers/net/wan/hdlc_x25.c
+++ linux-next-20210708/drivers/net/wan/hdlc_x25.c
@@ -365,19 +365,19 @@ static int x25_ioctl(struct net_device *
 	return -EINVAL;
 }
 
-static int __init mod_init(void)
+static int __init hdlc_x25_init(void)
 {
 	register_hdlc_protocol(&proto);
 	return 0;
 }
 
-static void __exit mod_exit(void)
+static void __exit hdlc_x25_exit(void)
 {
 	unregister_hdlc_protocol(&proto);
 }
 
-module_init(mod_init);
-module_exit(mod_exit);
+module_init(hdlc_x25_init);
+module_exit(hdlc_x25_exit);
 
 MODULE_AUTHOR("Krzysztof Halasa <khc at pm.waw.pl>");
 MODULE_DESCRIPTION("X.25 protocol support for generic HDLC");

From rdunlap at infradead.org  Fri Jul  9 02:17:47 2021
From: rdunlap at infradead.org (Randy Dunlap)
Date: Thu,  8 Jul 2021 19:17:47 -0700
Subject: [PATCH 6/6] net: wireguard: rename 'mod_init' & 'mod_exit' functions
 to be module-specific
In-Reply-To: <20210709021747.32737-1-rdunlap@infradead.org>
References: <20210709021747.32737-1-rdunlap@infradead.org>
Message-ID: <20210709021747.32737-7-rdunlap@infradead.org>

Rename module_init & module_exit functions that are named
"mod_init" and "mod_exit" so that they are unique in both the
System.map file and in initcall_debug output instead of showing
up as almost anonymous "mod_init".

This is helpful for debugging and in determining how long certain
module_init calls take to execute.

Signed-off-by: Randy Dunlap <rdunlap at infradead.org>
Cc: Jason A. Donenfeld <Jason at zx2c4.com>
Cc: wireguard at lists.zx2c4.com
Cc: netdev at vger.kernel.org
---
 drivers/net/wireguard/main.c |    8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)

--- linux-next-20210708.orig/drivers/net/wireguard/main.c
+++ linux-next-20210708/drivers/net/wireguard/main.c
@@ -17,7 +17,7 @@
 #include <linux/genetlink.h>
 #include <net/rtnetlink.h>
 
-static int __init mod_init(void)
+static int __init wireguard_init(void)
 {
 	int ret;
 
@@ -60,7 +60,7 @@ err_allowedips:
 	return ret;
 }
 
-static void __exit mod_exit(void)
+static void __exit wireguard_exit(void)
 {
 	wg_genetlink_uninit();
 	wg_device_uninit();
@@ -68,8 +68,8 @@ static void __exit mod_exit(void)
 	wg_allowedips_slab_uninit();
 }
 
-module_init(mod_init);
-module_exit(mod_exit);
+module_init(wireguard_init);
+module_exit(wireguard_exit);
 MODULE_LICENSE("GPL v2");
 MODULE_DESCRIPTION("WireGuard secure network tunnel");
 MODULE_AUTHOR("Jason A. Donenfeld <Jason at zx2c4.com>");

From Jason at zx2c4.com  Fri Jul  9 10:21:39 2021
From: Jason at zx2c4.com (Jason A. Donenfeld)
Date: Fri, 9 Jul 2021 12:21:39 +0200
Subject: [PATCH 6/6] net: wireguard: rename 'mod_init' & 'mod_exit'
 functions to be module-specific
In-Reply-To: <20210709021747.32737-7-rdunlap@infradead.org>
References: <20210709021747.32737-1-rdunlap@infradead.org>
 <20210709021747.32737-7-rdunlap@infradead.org>
Message-ID: <CAHmME9rUJndFC-6KAxGL3w6Ka4WgyP_m5DS+1_vuHdQowonSOg@mail.gmail.com>

Hi Randy,

The commit subject line should be:

wireguard: main: rename 'mod_init' & 'mod_exit' functions to be module-specific

And:

On Fri, Jul 9, 2021 at 4:17 AM Randy Dunlap <rdunlap at infradead.org> wrote:
> -static int __init mod_init(void)
> +static int __init wireguard_init(void)

wg_mod_init

> -static void __exit mod_exit(void)
> +static void __exit wireguard_exit(void)

wg_mod_exit

Thanks,
Jason

From rdunlap at infradead.org  Sun Jul 11 22:31:46 2021
From: rdunlap at infradead.org (Randy Dunlap)
Date: Sun, 11 Jul 2021 15:31:46 -0700
Subject: [PATCH 4/6 v2] MOST: cdev: rename 'mod_init' & 'mod_exit' functions
 to be module-specific
In-Reply-To: <20210711223148.5250-1-rdunlap@infradead.org>
References: <20210711223148.5250-1-rdunlap@infradead.org>
Message-ID: <20210711223148.5250-5-rdunlap@infradead.org>

Rename module_init & module_exit functions that are named
"mod_init" and "mod_exit" so that they are unique in both the
System.map file and in initcall_debug output instead of showing
up as almost anonymous "mod_init".

This is helpful for debugging and in determining how long certain
module_init calls take to execute.

Signed-off-by: Randy Dunlap <rdunlap at infradead.org>
Cc: Christian Gromm <christian.gromm at microchip.com>
Cc: Greg Kroah-Hartman <gregkh at linuxfoundation.org>
---
v2: no change

 drivers/most/most_cdev.c |    8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)

--- linux-next-20210708.orig/drivers/most/most_cdev.c
+++ linux-next-20210708/drivers/most/most_cdev.c
@@ -486,7 +486,7 @@ static struct cdev_component comp = {
 	},
 };
 
-static int __init mod_init(void)
+static int __init most_cdev_init(void)
 {
 	int err;
 
@@ -518,7 +518,7 @@ dest_ida:
 	return err;
 }
 
-static void __exit mod_exit(void)
+static void __exit most_cdev_exit(void)
 {
 	struct comp_channel *c, *tmp;
 
@@ -534,8 +534,8 @@ static void __exit mod_exit(void)
 	class_destroy(comp.class);
 }
 
-module_init(mod_init);
-module_exit(mod_exit);
+module_init(most_cdev_init);
+module_exit(most_cdev_exit);
 MODULE_AUTHOR("Christian Gromm <christian.gromm at microchip.com>");
 MODULE_LICENSE("GPL");
 MODULE_DESCRIPTION("character device component for mostcore");

From rdunlap at infradead.org  Sun Jul 11 22:31:45 2021
From: rdunlap at infradead.org (Randy Dunlap)
Date: Sun, 11 Jul 2021 15:31:45 -0700
Subject: [PATCH 3/6 v2] lib: crypto: rename 'mod_init' & 'mod_exit' functions
 to be module-specific
In-Reply-To: <20210711223148.5250-1-rdunlap@infradead.org>
References: <20210711223148.5250-1-rdunlap@infradead.org>
Message-ID: <20210711223148.5250-4-rdunlap@infradead.org>

Rename module_init & module_exit functions that are named
"mod_init" and "mod_exit" so that they are unique in both the
System.map file and in initcall_debug output instead of showing
up as almost anonymous "mod_init".

This is helpful for debugging and in determining how long certain
module_init calls take to execute.

Signed-off-by: Randy Dunlap <rdunlap at infradead.org>
Cc: Herbert Xu <herbert at gondor.apana.org.au>
Cc: "David S. Miller" <davem at davemloft.net>
Cc: linux-crypto at vger.kernel.org
Cc: Jason A. Donenfeld <Jason at zx2c4.com>
---
v2: no change

 lib/crypto/blake2s.c          |    8 ++++----
 lib/crypto/chacha20poly1305.c |    8 ++++----
 lib/crypto/curve25519.c       |    8 ++++----
 3 files changed, 12 insertions(+), 12 deletions(-)

--- linux-next-20210708.orig/lib/crypto/blake2s.c
+++ linux-next-20210708/lib/crypto/blake2s.c
@@ -73,7 +73,7 @@ void blake2s256_hmac(u8 *out, const u8 *
 }
 EXPORT_SYMBOL(blake2s256_hmac);
 
-static int __init mod_init(void)
+static int __init blake2s_mod_init(void)
 {
 	if (!IS_ENABLED(CONFIG_CRYPTO_MANAGER_DISABLE_TESTS) &&
 	    WARN_ON(!blake2s_selftest()))
@@ -81,12 +81,12 @@ static int __init mod_init(void)
 	return 0;
 }
 
-static void __exit mod_exit(void)
+static void __exit blake2s_mod_exit(void)
 {
 }
 
-module_init(mod_init);
-module_exit(mod_exit);
+module_init(blake2s_mod_init);
+module_exit(blake2s_mod_exit);
 MODULE_LICENSE("GPL v2");
 MODULE_DESCRIPTION("BLAKE2s hash function");
 MODULE_AUTHOR("Jason A. Donenfeld <Jason at zx2c4.com>");
--- linux-next-20210708.orig/lib/crypto/chacha20poly1305.c
+++ linux-next-20210708/lib/crypto/chacha20poly1305.c
@@ -354,7 +354,7 @@ bool chacha20poly1305_decrypt_sg_inplace
 }
 EXPORT_SYMBOL(chacha20poly1305_decrypt_sg_inplace);
 
-static int __init mod_init(void)
+static int __init chacha20poly1305_init(void)
 {
 	if (!IS_ENABLED(CONFIG_CRYPTO_MANAGER_DISABLE_TESTS) &&
 	    WARN_ON(!chacha20poly1305_selftest()))
@@ -362,12 +362,12 @@ static int __init mod_init(void)
 	return 0;
 }
 
-static void __exit mod_exit(void)
+static void __exit chacha20poly1305_exit(void)
 {
 }
 
-module_init(mod_init);
-module_exit(mod_exit);
+module_init(chacha20poly1305_init);
+module_exit(chacha20poly1305_exit);
 MODULE_LICENSE("GPL v2");
 MODULE_DESCRIPTION("ChaCha20Poly1305 AEAD construction");
 MODULE_AUTHOR("Jason A. Donenfeld <Jason at zx2c4.com>");
--- linux-next-20210708.orig/lib/crypto/curve25519.c
+++ linux-next-20210708/lib/crypto/curve25519.c
@@ -13,7 +13,7 @@
 #include <linux/module.h>
 #include <linux/init.h>
 
-static int __init mod_init(void)
+static int __init curve25519_init(void)
 {
 	if (!IS_ENABLED(CONFIG_CRYPTO_MANAGER_DISABLE_TESTS) &&
 	    WARN_ON(!curve25519_selftest()))
@@ -21,12 +21,12 @@ static int __init mod_init(void)
 	return 0;
 }
 
-static void __exit mod_exit(void)
+static void __exit curve25519_exit(void)
 {
 }
 
-module_init(mod_init);
-module_exit(mod_exit);
+module_init(curve25519_init);
+module_exit(curve25519_exit);
 
 MODULE_LICENSE("GPL v2");
 MODULE_DESCRIPTION("Curve25519 scalar multiplication");

From rdunlap at infradead.org  Sun Jul 11 22:31:47 2021
From: rdunlap at infradead.org (Randy Dunlap)
Date: Sun, 11 Jul 2021 15:31:47 -0700
Subject: [PATCH 5/6 v2] net: hdlc: rename 'mod_init' & 'mod_exit' functions to
 be module-specific
In-Reply-To: <20210711223148.5250-1-rdunlap@infradead.org>
References: <20210711223148.5250-1-rdunlap@infradead.org>
Message-ID: <20210711223148.5250-6-rdunlap@infradead.org>

Rename module_init & module_exit functions that are named
"mod_init" and "mod_exit" so that they are unique in both the
System.map file and in initcall_debug output instead of showing
up as almost anonymous "mod_init".

This is helpful for debugging and in determining how long certain
module_init calls take to execute.

Signed-off-by: Randy Dunlap <rdunlap at infradead.org>
Cc: Krzysztof Halasa <khc at pm.waw.pl>
Cc: netdev at vger.kernel.org
Cc: "David S. Miller" <davem at davemloft.net>
Cc: Jakub Kicinski <kuba at kernel.org>
Cc: Martin Schiller <ms at dev.tdt.de>
Cc: linux-x25 at vger.kernel.org
---
v2: no change

 drivers/net/wan/hdlc_cisco.c   |    8 ++++----
 drivers/net/wan/hdlc_fr.c      |    8 ++++----
 drivers/net/wan/hdlc_ppp.c     |    8 ++++----
 drivers/net/wan/hdlc_raw.c     |    8 ++++----
 drivers/net/wan/hdlc_raw_eth.c |    8 ++++----
 drivers/net/wan/hdlc_x25.c     |    8 ++++----
 6 files changed, 24 insertions(+), 24 deletions(-)

--- linux-next-20210708.orig/drivers/net/wan/hdlc_cisco.c
+++ linux-next-20210708/drivers/net/wan/hdlc_cisco.c
@@ -364,19 +364,19 @@ static int cisco_ioctl(struct net_device
 	return -EINVAL;
 }
 
-static int __init mod_init(void)
+static int __init hdlc_cisco_init(void)
 {
 	register_hdlc_protocol(&proto);
 	return 0;
 }
 
-static void __exit mod_exit(void)
+static void __exit hdlc_cisco_exit(void)
 {
 	unregister_hdlc_protocol(&proto);
 }
 
-module_init(mod_init);
-module_exit(mod_exit);
+module_init(hdlc_cisco_init);
+module_exit(hdlc_cisco_exit);
 
 MODULE_AUTHOR("Krzysztof Halasa <khc at pm.waw.pl>");
 MODULE_DESCRIPTION("Cisco HDLC protocol support for generic HDLC");
--- linux-next-20210708.orig/drivers/net/wan/hdlc_fr.c
+++ linux-next-20210708/drivers/net/wan/hdlc_fr.c
@@ -1279,19 +1279,19 @@ static int fr_ioctl(struct net_device *d
 	return -EINVAL;
 }
 
-static int __init mod_init(void)
+static int __init hdlc_fr_init(void)
 {
 	register_hdlc_protocol(&proto);
 	return 0;
 }
 
-static void __exit mod_exit(void)
+static void __exit hdlc_fr_exit(void)
 {
 	unregister_hdlc_protocol(&proto);
 }
 
-module_init(mod_init);
-module_exit(mod_exit);
+module_init(hdlc_fr_init);
+module_exit(hdlc_fr_exit);
 
 MODULE_AUTHOR("Krzysztof Halasa <khc at pm.waw.pl>");
 MODULE_DESCRIPTION("Frame-Relay protocol support for generic HDLC");
--- linux-next-20210708.orig/drivers/net/wan/hdlc_ppp.c
+++ linux-next-20210708/drivers/net/wan/hdlc_ppp.c
@@ -705,20 +705,20 @@ static int ppp_ioctl(struct net_device *
 	return -EINVAL;
 }
 
-static int __init mod_init(void)
+static int __init hdlc_ppp_init(void)
 {
 	skb_queue_head_init(&tx_queue);
 	register_hdlc_protocol(&proto);
 	return 0;
 }
 
-static void __exit mod_exit(void)
+static void __exit hdlc_ppp_exit(void)
 {
 	unregister_hdlc_protocol(&proto);
 }
 
-module_init(mod_init);
-module_exit(mod_exit);
+module_init(hdlc_ppp_init);
+module_exit(hdlc_ppp_exit);
 
 MODULE_AUTHOR("Krzysztof Halasa <khc at pm.waw.pl>");
 MODULE_DESCRIPTION("PPP protocol support for generic HDLC");
--- linux-next-20210708.orig/drivers/net/wan/hdlc_raw.c
+++ linux-next-20210708/drivers/net/wan/hdlc_raw.c
@@ -90,7 +90,7 @@ static int raw_ioctl(struct net_device *
 }
 
 
-static int __init mod_init(void)
+static int __init hdlc_raw_init(void)
 {
 	register_hdlc_protocol(&proto);
 	return 0;
@@ -98,14 +98,14 @@ static int __init mod_init(void)
 
 
 
-static void __exit mod_exit(void)
+static void __exit hdlc_raw_exit(void)
 {
 	unregister_hdlc_protocol(&proto);
 }
 
 
-module_init(mod_init);
-module_exit(mod_exit);
+module_init(hdlc_raw_init);
+module_exit(hdlc_raw_exit);
 
 MODULE_AUTHOR("Krzysztof Halasa <khc at pm.waw.pl>");
 MODULE_DESCRIPTION("Raw HDLC protocol support for generic HDLC");
--- linux-next-20210708.orig/drivers/net/wan/hdlc_raw_eth.c
+++ linux-next-20210708/drivers/net/wan/hdlc_raw_eth.c
@@ -110,7 +110,7 @@ static int raw_eth_ioctl(struct net_devi
 }
 
 
-static int __init mod_init(void)
+static int __init hdlc_eth_init(void)
 {
 	register_hdlc_protocol(&proto);
 	return 0;
@@ -118,14 +118,14 @@ static int __init mod_init(void)
 
 
 
-static void __exit mod_exit(void)
+static void __exit hdlc_eth_exit(void)
 {
 	unregister_hdlc_protocol(&proto);
 }
 
 
-module_init(mod_init);
-module_exit(mod_exit);
+module_init(hdlc_eth_init);
+module_exit(hdlc_eth_exit);
 
 MODULE_AUTHOR("Krzysztof Halasa <khc at pm.waw.pl>");
 MODULE_DESCRIPTION("Ethernet encapsulation support for generic HDLC");
--- linux-next-20210708.orig/drivers/net/wan/hdlc_x25.c
+++ linux-next-20210708/drivers/net/wan/hdlc_x25.c
@@ -365,19 +365,19 @@ static int x25_ioctl(struct net_device *
 	return -EINVAL;
 }
 
-static int __init mod_init(void)
+static int __init hdlc_x25_init(void)
 {
 	register_hdlc_protocol(&proto);
 	return 0;
 }
 
-static void __exit mod_exit(void)
+static void __exit hdlc_x25_exit(void)
 {
 	unregister_hdlc_protocol(&proto);
 }
 
-module_init(mod_init);
-module_exit(mod_exit);
+module_init(hdlc_x25_init);
+module_exit(hdlc_x25_exit);
 
 MODULE_AUTHOR("Krzysztof Halasa <khc at pm.waw.pl>");
 MODULE_DESCRIPTION("X.25 protocol support for generic HDLC");

From rdunlap at infradead.org  Sun Jul 11 22:31:42 2021
From: rdunlap at infradead.org (Randy Dunlap)
Date: Sun, 11 Jul 2021 15:31:42 -0700
Subject: [PATCH 0/6 v2] treewide: rename 'mod_init' & 'mod_exit' functions to
 be module-specific
Message-ID: <20210711223148.5250-1-rdunlap@infradead.org>

There are multiple (16) modules which use a module_init() function
with the name 'mod_init' and a module_exit() function with the name
'mod_exit'. This can lead to confusion or ambiguity when reading
crashes/oops/bugs etc. and when reading an initcall_debug log.

Example 1: (System.map file)

ffffffff83446d10 t mod_init
ffffffff83446d18 t mod_init
ffffffff83446d20 t mod_init
...
ffffffff83454665 t mod_init
ffffffff834548a4 t mod_init
ffffffff83454a53 t mod_init
...
ffffffff8345bd42 t mod_init
...
ffffffff8345c916 t mod_init
ffffffff8345c92a t mod_init
ffffffff8345c93e t mod_init
ffffffff8345c952 t mod_init
ffffffff8345c966 t mod_init
...
ffffffff834672c9 t mod_init

Example 2: (boot log when using 'initcall_debug')

[    0.252157] initcall mod_init+0x0/0x8 returned 0 after 0 usecs
[    0.252180] initcall mod_init+0x0/0x8 returned 0 after 0 usecs
[    0.252202] initcall mod_init+0x0/0x8 returned 0 after 0 usecs
...
[    0.892907] initcall mod_init+0x0/0x23f returned -19 after 104 usecs
[    0.913788] initcall mod_init+0x0/0x1af returned -19 after 9 usecs
[    0.934353] initcall mod_init+0x0/0x49 returned -19 after 0 usecs
...
[    1.454870] initcall mod_init+0x0/0x66 returned 0 after 72 usecs
...
[    1.455527] initcall mod_init+0x0/0x14 returned 0 after 0 usecs
[    1.455531] initcall mod_init+0x0/0x14 returned 0 after 0 usecs
[    1.455536] initcall mod_init+0x0/0x14 returned 0 after 0 usecs
[    1.455541] initcall mod_init+0x0/0x14 returned 0 after 0 usecs
[    1.455545] initcall mod_init+0x0/0x52 returned 0 after 0 usecs
...
[    1.588162] initcall mod_init+0x0/0xef returned 0 after 45 usecs


v2: wireguard: changes per Jason
    arm/crypto/curve25519-glue: add Russell's Acked-by

Cc: Jason A. Donenfeld <Jason at zx2c4.com>
Cc: Russell King <linux at armlinux.org.uk>
Cc: linux-arm-kernel at lists.infradead.org
Cc: Arnd Bergmann <arnd at arndb.de>
Cc: Greg Kroah-Hartman <gregkh at linuxfoundation.org>
Cc: Andres Salomon <dilinger at queued.net>
Cc: linux-geode at lists.infradead.org
Cc: Matt Mackall <mpm at selenic.com>
Cc: Herbert Xu <herbert at gondor.apana.org.au>
Cc: linux-crypto at vger.kernel.org
Cc: Christian Gromm <christian.gromm at microchip.com>
Cc: Krzysztof Halasa <khc at pm.waw.pl>
Cc: netdev at vger.kernel.org
Cc: "David S. Miller" <davem at davemloft.net>
Cc: Jakub Kicinski <kuba at kernel.org>
Cc: Martin Schiller <ms at dev.tdt.de>
Cc: linux-x25 at vger.kernel.org
Cc: wireguard at lists.zx2c4.com

[PATCH 1/6 v2] arm: crypto: rename 'mod_init' & 'mod_exit' functions to be module-specific
[PATCH 2/6 v2] hw_random: rename 'mod_init' & 'mod_exit' functions to be module-specific
[PATCH 3/6 v2] lib: crypto: rename 'mod_init' & 'mod_exit' functions to be module-specific
[PATCH 4/6 v2] MOST: cdev: rename 'mod_init' & 'mod_exit' functions to be module-specific
[PATCH 5/6 v2] net: hdlc: rename 'mod_init' & 'mod_exit' functions to be module-specific
[PATCH 6/6 v2] net: wireguard: rename 'mod_init' & 'mod_exit' functions to be module-specific

 arch/arm/crypto/curve25519-glue.c  |    8 ++++----
 drivers/char/hw_random/amd-rng.c   |    8 ++++----
 drivers/char/hw_random/geode-rng.c |    8 ++++----
 drivers/char/hw_random/intel-rng.c |    8 ++++----
 drivers/char/hw_random/via-rng.c   |    8 ++++----
 drivers/most/most_cdev.c           |    8 ++++----
 drivers/net/wan/hdlc_cisco.c       |    8 ++++----
 drivers/net/wan/hdlc_fr.c          |    8 ++++----
 drivers/net/wan/hdlc_ppp.c         |    8 ++++----
 drivers/net/wan/hdlc_raw.c         |    8 ++++----
 drivers/net/wan/hdlc_raw_eth.c     |    8 ++++----
 drivers/net/wan/hdlc_x25.c         |    8 ++++----
 drivers/net/wireguard/main.c       |    8 ++++----
 lib/crypto/blake2s.c               |    8 ++++----
 lib/crypto/chacha20poly1305.c      |    8 ++++----
 lib/crypto/curve25519.c            |    8 ++++----
 16 files changed, 64 insertions(+), 64 deletions(-)

From rdunlap at infradead.org  Sun Jul 11 22:31:48 2021
From: rdunlap at infradead.org (Randy Dunlap)
Date: Sun, 11 Jul 2021 15:31:48 -0700
Subject: [PATCH 6/6 v2] wireguard: main: rename 'mod_init' & 'mod_exit'
 functions to be module-specific
In-Reply-To: <20210711223148.5250-1-rdunlap@infradead.org>
References: <20210711223148.5250-1-rdunlap@infradead.org>
Message-ID: <20210711223148.5250-7-rdunlap@infradead.org>

Rename module_init & module_exit functions that are named
"mod_init" and "mod_exit" so that they are unique in both the
System.map file and in initcall_debug output instead of showing
up as almost anonymous "mod_init".

This is helpful for debugging and in determining how long certain
module_init calls take to execute.

Signed-off-by: Randy Dunlap <rdunlap at infradead.org>
Cc: Jason A. Donenfeld <Jason at zx2c4.com>
Cc: wireguard at lists.zx2c4.com
Cc: netdev at vger.kernel.org
---
v2: change Subject: prefixes and function names (thanks, Jason)

 drivers/net/wireguard/main.c |    8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)

--- linux-next-20210708.orig/drivers/net/wireguard/main.c
+++ linux-next-20210708/drivers/net/wireguard/main.c
@@ -17,7 +17,7 @@
 #include <linux/genetlink.h>
 #include <net/rtnetlink.h>
 
-static int __init mod_init(void)
+static int __init wg_mod_init(void)
 {
 	int ret;
 
@@ -60,7 +60,7 @@ err_allowedips:
 	return ret;
 }
 
-static void __exit mod_exit(void)
+static void __exit wg_mod_exit(void)
 {
 	wg_genetlink_uninit();
 	wg_device_uninit();
@@ -68,8 +68,8 @@ static void __exit mod_exit(void)
 	wg_allowedips_slab_uninit();
 }
 
-module_init(mod_init);
-module_exit(mod_exit);
+module_init(wg_mod_init);
+module_exit(wg_mod_exit);
 MODULE_LICENSE("GPL v2");
 MODULE_DESCRIPTION("WireGuard secure network tunnel");
 MODULE_AUTHOR("Jason A. Donenfeld <Jason at zx2c4.com>");

From rdunlap at infradead.org  Sun Jul 11 22:31:44 2021
From: rdunlap at infradead.org (Randy Dunlap)
Date: Sun, 11 Jul 2021 15:31:44 -0700
Subject: [PATCH 2/6 v2] hw_random: rename 'mod_init' & 'mod_exit' functions to
 be module-specific
In-Reply-To: <20210711223148.5250-1-rdunlap@infradead.org>
References: <20210711223148.5250-1-rdunlap@infradead.org>
Message-ID: <20210711223148.5250-3-rdunlap@infradead.org>

Rename module_init & module_exit functions that are named
"mod_init" and "mod_exit" so that they are unique in both the
System.map file and in initcall_debug output instead of showing
up as almost anonymous "mod_init".

This is helpful for debugging and in determining how long certain
module_init calls take to execute.

Signed-off-by: Randy Dunlap <rdunlap at infradead.org>
Cc: Arnd Bergmann <arnd at arndb.de>
Cc: Greg Kroah-Hartman <gregkh at linuxfoundation.org>
Cc: Andres Salomon <dilinger at queued.net>
Cc: linux-geode at lists.infradead.org
Cc: Matt Mackall <mpm at selenic.com>
Cc: Herbert Xu <herbert at gondor.apana.org.au>
Cc: "David S. Miller" <davem at davemloft.net>
Cc: linux-crypto at vger.kernel.org
Cc: Jason A. Donenfeld <Jason at zx2c4.com>
---
v2: no change

 drivers/char/hw_random/amd-rng.c   |    8 ++++----
 drivers/char/hw_random/geode-rng.c |    8 ++++----
 drivers/char/hw_random/intel-rng.c |    8 ++++----
 drivers/char/hw_random/via-rng.c   |    8 ++++----
 4 files changed, 16 insertions(+), 16 deletions(-)

--- linux-next-20210708.orig/drivers/char/hw_random/amd-rng.c
+++ linux-next-20210708/drivers/char/hw_random/amd-rng.c
@@ -124,7 +124,7 @@ static struct hwrng amd_rng = {
 	.read		= amd_rng_read,
 };
 
-static int __init mod_init(void)
+static int __init amd_rng_mod_init(void)
 {
 	int err;
 	struct pci_dev *pdev = NULL;
@@ -188,7 +188,7 @@ out:
 	return err;
 }
 
-static void __exit mod_exit(void)
+static void __exit amd_rng_mod_exit(void)
 {
 	struct amd768_priv *priv;
 
@@ -203,8 +203,8 @@ static void __exit mod_exit(void)
 	kfree(priv);
 }
 
-module_init(mod_init);
-module_exit(mod_exit);
+module_init(amd_rng_mod_init);
+module_exit(amd_rng_mod_exit);
 
 MODULE_AUTHOR("The Linux Kernel team");
 MODULE_DESCRIPTION("H/W RNG driver for AMD chipsets");
--- linux-next-20210708.orig/drivers/char/hw_random/geode-rng.c
+++ linux-next-20210708/drivers/char/hw_random/geode-rng.c
@@ -83,7 +83,7 @@ static struct hwrng geode_rng = {
 };
 
 
-static int __init mod_init(void)
+static int __init geode_rng_init(void)
 {
 	int err = -ENODEV;
 	struct pci_dev *pdev = NULL;
@@ -124,7 +124,7 @@ err_unmap:
 	goto out;
 }
 
-static void __exit mod_exit(void)
+static void __exit geode_rng_exit(void)
 {
 	void __iomem *mem = (void __iomem *)geode_rng.priv;
 
@@ -132,8 +132,8 @@ static void __exit mod_exit(void)
 	iounmap(mem);
 }
 
-module_init(mod_init);
-module_exit(mod_exit);
+module_init(geode_rng_init);
+module_exit(geode_rng_exit);
 
 MODULE_DESCRIPTION("H/W RNG driver for AMD Geode LX CPUs");
 MODULE_LICENSE("GPL");
--- linux-next-20210708.orig/drivers/char/hw_random/intel-rng.c
+++ linux-next-20210708/drivers/char/hw_random/intel-rng.c
@@ -325,7 +325,7 @@ PFX "RNG, try using the 'no_fwh_detect'
 }
 
 
-static int __init mod_init(void)
+static int __init intel_rng_mod_init(void)
 {
 	int err = -ENODEV;
 	int i;
@@ -403,7 +403,7 @@ out:
 
 }
 
-static void __exit mod_exit(void)
+static void __exit intel_rng_mod_exit(void)
 {
 	void __iomem *mem = (void __iomem *)intel_rng.priv;
 
@@ -411,8 +411,8 @@ static void __exit mod_exit(void)
 	iounmap(mem);
 }
 
-module_init(mod_init);
-module_exit(mod_exit);
+module_init(intel_rng_mod_init);
+module_exit(intel_rng_mod_exit);
 
 MODULE_DESCRIPTION("H/W RNG driver for Intel chipsets");
 MODULE_LICENSE("GPL");
--- linux-next-20210708.orig/drivers/char/hw_random/via-rng.c
+++ linux-next-20210708/drivers/char/hw_random/via-rng.c
@@ -192,7 +192,7 @@ static struct hwrng via_rng = {
 };
 
 
-static int __init mod_init(void)
+static int __init via_rng_mod_init(void)
 {
 	int err;
 
@@ -209,13 +209,13 @@ static int __init mod_init(void)
 out:
 	return err;
 }
-module_init(mod_init);
+module_init(via_rng_mod_init);
 
-static void __exit mod_exit(void)
+static void __exit via_rng_mod_exit(void)
 {
 	hwrng_unregister(&via_rng);
 }
-module_exit(mod_exit);
+module_exit(via_rng_mod_exit);
 
 static struct x86_cpu_id __maybe_unused via_rng_cpu_id[] = {
 	X86_MATCH_FEATURE(X86_FEATURE_XSTORE, NULL),

From wireguard at spam-free.eu  Wed Jul 14 05:49:35 2021
From: wireguard at spam-free.eu (Chris)
Date: Wed, 14 Jul 2021 07:49:35 +0200
Subject: Android: more than one tunnel
In-Reply-To: <296f17d0-3f0b-6021-7bb3-4b4bf576f6c7@spam-free.eu>
References: <mailman.0.1601114111.20018.wireguard@lists.zx2c4.com>
 <296f17d0-3f0b-6021-7bb3-4b4bf576f6c7@spam-free.eu>
Message-ID: <e8e94195-b1f0-df62-4393-929f20f762a3@spam-free.eu>

I have two distinct tunnels defined on Android.
However, I can only toggle between the two. I cannot have both activated at the 
same time.
I need both connections.
Am I missing anything?
Why not several tunnels with several wg servers???
Why is this being prevented?

BR
Chris


From wireguard at spam-free.eu  Wed Jul 14 09:54:42 2021
From: wireguard at spam-free.eu (Chris)
Date: Wed, 14 Jul 2021 11:54:42 +0200
Subject: wg-quick with default route fails on nfs root filesystem
Message-ID: <e4fef9da-ee7f-d1db-036e-b294785d9ee8@spam-free.eu>

When wg-quick detects a default route through the tunnel it does this through a 
new routing table with a default route.
However not to destroy the existing non-default routes these will looked up and 
used first. This results in the follwing policy rule entries:
(The priority numers may be different from system to system)

32764:??? from all lookup main suppress_prefixlength 0
32765:??? not from all fwmark 0xca6c lookup 51820

It is very important of course, that the suppress_prefix rule comes first in the 
list, before
the second rule introduces the new default route (preventig the wireguard 
traffic through it's own tunnel).

The way to archive this is done by the following command sequence:

ip -4 rule add not fwmark 51820 table 51820
ip -4 rule add table main suppress_prefixlength 0

The sequence of the commands is important as the latter command gets the higher 
priority (lower numer).

BUT:
In case your root filesystem needs the local network, the second command will 
not be reached as the
first command (setting the new default route) kills the root filesystem and the 
system stalls!!!!!!

One possible solution:
Instead of adding the suppress_prefixlength 0 command secondly it must be first.
The you must find the priority of that rule and the add the default route with 
the same priority.
A rule with same priority will be added AFTER the other rules.

Example:
ip -4 rule add table main suppress_prefixlength 0
PRIO=$(ip rule list from all|grep suppress_prefixlength|sed -e 
'{s/^\(.*\)\:.*/\1/;q}')
ip -4 rule add not fwmark 51820 table 51820 priority $PRIO

This will lead to the correct sequence:
32765:??? from all lookup main suppress_prefixlength 0
32765:??? not from all fwmark 0xca6c lookup 51820
(Note the same priority number)

There are probably better ways to cirumvent cutting off the root filesystem.

Chris


From eric at ericlight.com  Wed Jul 14 22:09:36 2021
From: eric at ericlight.com (Eric Light)
Date: Thu, 15 Jul 2021 10:09:36 +1200
Subject: Android: more than one tunnel
In-Reply-To: <e8e94195-b1f0-df62-4393-929f20f762a3@spam-free.eu>
References: <mailman.0.1601114111.20018.wireguard@lists.zx2c4.com>
 <296f17d0-3f0b-6021-7bb3-4b4bf576f6c7@spam-free.eu>
 <e8e94195-b1f0-df62-4393-929f20f762a3@spam-free.eu>
Message-ID: <d994accd-d1a9-4f9a-bbf9-d6b99036b177@www.fastmail.com>

Hi Chris!

I don't have any experience with the Android implementation specifically... but most likely, your two tunnels have overlapping AllowedIPs ranges.  When this happens, bringing up the second interface will override the routing created by the first interface.

Most commonly, this happens when someone configures both interfaces with AllowedIPs = 0.0.0.0/0.  To fix this problem, make sure there's no overlap of AllowedIPs in your tunnel definitions.

I hope this helps  :)

(my apologies if the problem is something Android-specific that I don't know about)

E

--------------------------------------------
Q: Why is this email five sentences or less?
A: http://five.sentenc.es

On Wed, 14 Jul 2021, at 17:49, Chris wrote:
> I have two distinct tunnels defined on Android.
> However, I can only toggle between the two. I cannot have both activated at the 
> same time.
> I need both connections.
> Am I missing anything?
> Why not several tunnels with several wg servers???
> Why is this being prevented?
> 
> BR
> Chris
> 
> 

From wireguard at spam-free.eu  Thu Jul 15 06:25:59 2021
From: wireguard at spam-free.eu (Chris)
Date: Thu, 15 Jul 2021 08:25:59 +0200
Subject: Android: more than one tunnel
In-Reply-To: <d994accd-d1a9-4f9a-bbf9-d6b99036b177@www.fastmail.com>
References: <mailman.0.1601114111.20018.wireguard@lists.zx2c4.com>
 <296f17d0-3f0b-6021-7bb3-4b4bf576f6c7@spam-free.eu>
 <e8e94195-b1f0-df62-4393-929f20f762a3@spam-free.eu>
 <d994accd-d1a9-4f9a-bbf9-d6b99036b177@www.fastmail.com>
Message-ID: <98318db6-2c3d-82a7-4eea-5e8a6b1e313e@spam-free.eu>

Hi Eric,

thanks for you thoughts.
However, this is not the case. I have the same setup on many machines and 
different OSes.
Apart from that: you can still have many tunnels running with overlapping 
routeing ranges.
What will happen within the routeing tables for the routeing decision is a 
different question.

On Android switching on one tunnel just switches off the other.

I fear, Simon's reply is correct:
"
...
There?s only one active service per user or profile. Starting a new service, 
automatically stops an existing service.
...
"
So, unfortunately, that would be an unavoidable limitation of Android.

Chris

On 15/07/2021 00:09, Eric Light wrote:
> Hi Chris!
>
> I don't have any experience with the Android implementation specifically... 
> but most likely, your two tunnels have overlapping AllowedIPs ranges. When 
> this happens, bringing up the second interface will override the routing 
> created by the first interface.
>
> Most commonly, this happens when someone configures both interfaces with 
> AllowedIPs = 0.0.0.0/0. To fix this problem, make sure there's no overlap of 
> AllowedIPs in your tunnel definitions.
>
> I hope this helps :)
>
> (my apologies if the problem is something Android-specific that I don't know 
> about)
>
> E
>
> --------------------------------------------
> Q: Why is this email five sentences or less?
> A: http://five.sentenc.es
>
> On Wed, 14 Jul 2021, at 17:49, Chris wrote:
>> I have two distinct tunnels defined on Android.
>> However, I can only toggle between the two. I cannot have both activated at the
>> same time.
>> I need both connections.
>> Am I missing anything?
>> Why not several tunnels with several wg servers???
>> Why is this being prevented?
>>
>> BR
>> Chris
>>
>>



From Jason at zx2c4.com  Thu Jul 29 11:00:48 2021
From: Jason at zx2c4.com (Jason A. Donenfeld)
Date: Thu, 29 Jul 2021 13:00:48 +0200
Subject: Problems with Windows client over PulseSecure VPN
In-Reply-To: <CAHmME9qtPLOTaY3UzyCY1C6guVZXvJjXHeFwT8=94d=Zpjj=Tw@mail.gmail.com>
References: <9f621ce6-ec3d-0641-c359-756d0ad36f65@gmail.com>
 <6a01b182-a98f-1736-676f-d0811f6de086@gmail.com>
 <CAHmME9qtPLOTaY3UzyCY1C6guVZXvJjXHeFwT8=94d=Zpjj=Tw@mail.gmail.com>
Message-ID: <CAHmME9r5kecTj1iBGyqjvNRKsHZPRrXmN+fDXkaX3y==VPoMqA@mail.gmail.com>

Hi Peter, Heiko, Christopher, and others,

An update on:

> I had a strange idea for how to fix this without requiring
> recompilation or removal of that code.
>
> 1) Enable DangerousScriptExecution:
> https://git.zx2c4.com/wireguard-windows/about/docs/adminregistry.md#hklmsoftwarewireguarddangerousscriptexecution
>
> 2) Add a PostUp line to your [Interface] section:
>
> PostUp = wg set %WIREGUARD_TUNNEL_NAME% listen-port 0

I just wanted to let you know that this problem has been entirely
fixed (I think?) in the "WireGuardNT" kernel driver project I've been
working on (and haven't yet announced aside from development
screenshots on Twitter), and therefore the above steps will no longer
be necessary. When that ships as part of the v0.4 series of the normal
wireguard-windows client, you won't need the "listen-port 0" hack
anymore, as the kernel driver uses a more clever trick than the one
used by wireguard-go. So please do watch this mailing list in the next
few weeks for an announcement of that project, as I'll be very
interested in some real world tests and confirmation of the fix.

Thanks,
Jason

From peter.whisker at gmail.com  Fri Jul 30 07:28:37 2021
From: peter.whisker at gmail.com (Peter Whisker)
Date: Fri, 30 Jul 2021 08:28:37 +0100
Subject: Problems with Windows client over PulseSecure VPN
In-Reply-To: <CAHmME9r5kecTj1iBGyqjvNRKsHZPRrXmN+fDXkaX3y==VPoMqA@mail.gmail.com>
References: <9f621ce6-ec3d-0641-c359-756d0ad36f65@gmail.com>
 <6a01b182-a98f-1736-676f-d0811f6de086@gmail.com>
 <CAHmME9qtPLOTaY3UzyCY1C6guVZXvJjXHeFwT8=94d=Zpjj=Tw@mail.gmail.com>
 <CAHmME9r5kecTj1iBGyqjvNRKsHZPRrXmN+fDXkaX3y==VPoMqA@mail.gmail.com>
Message-ID: <29f66915-7fe8-94cd-cb17-bfba49072929@gmail.com>

Thanks Jason.

Does this mean that the driver speed will improve as it will move out of 
user space into the Windows Kernel in a similar way to Linux?

Peter

On 29/07/2021 12:00, Jason A. Donenfeld wrote:
> Hi Peter, Heiko, Christopher, and others,
>
> An update on:
>
>> I had a strange idea for how to fix this without requiring
>> recompilation or removal of that code.
>>
>> 1) Enable DangerousScriptExecution:
>> https://git.zx2c4.com/wireguard-windows/about/docs/adminregistry.md#hklmsoftwarewireguarddangerousscriptexecution
>>
>> 2) Add a PostUp line to your [Interface] section:
>>
>> PostUp = wg set %WIREGUARD_TUNNEL_NAME% listen-port 0
> I just wanted to let you know that this problem has been entirely
> fixed (I think?) in the "WireGuardNT" kernel driver project I've been
> working on (and haven't yet announced aside from development
> screenshots on Twitter), and therefore the above steps will no longer
> be necessary. When that ships as part of the v0.4 series of the normal
> wireguard-windows client, you won't need the "listen-port 0" hack
> anymore, as the kernel driver uses a more clever trick than the one
> used by wireguard-go. So please do watch this mailing list in the next
> few weeks for an announcement of that project, as I'll be very
> interested in some real world tests and confirmation of the fix.
>
> Thanks,
> Jason

From Jason at zx2c4.com  Fri Jul 30 15:57:51 2021
From: Jason at zx2c4.com (Jason A. Donenfeld)
Date: Fri, 30 Jul 2021 17:57:51 +0200
Subject: Problems with Windows client over PulseSecure VPN
In-Reply-To: <29f66915-7fe8-94cd-cb17-bfba49072929@gmail.com>
References: <9f621ce6-ec3d-0641-c359-756d0ad36f65@gmail.com>
 <6a01b182-a98f-1736-676f-d0811f6de086@gmail.com>
 <CAHmME9qtPLOTaY3UzyCY1C6guVZXvJjXHeFwT8=94d=Zpjj=Tw@mail.gmail.com>
 <CAHmME9r5kecTj1iBGyqjvNRKsHZPRrXmN+fDXkaX3y==VPoMqA@mail.gmail.com>
 <29f66915-7fe8-94cd-cb17-bfba49072929@gmail.com>
Message-ID: <CAHmME9r1zkZ+mXkY+8jTfeVrGPeHZwObrBvcEQM_hEHZvYzPWw@mail.gmail.com>

On Fri, Jul 30, 2021 at 9:25 AM Peter Whisker <peter.whisker at gmail.com> wrote:
> Does this mean that the driver speed will improve as it will move out of
> user space into the Windows Kernel in a similar way to Linux?

Please save further inquiries about topics unrelated to pulsesecure
until future announcements are made, in an applicable thread.

From j19981106 at protonmail.com  Fri Jul  2 04:18:53 2021
From: j19981106 at protonmail.com (j19981106)
Date: Fri, 02 Jul 2021 04:18:53 -0000
Subject: The default value of the IPv6 MTU of the Windows client is incorrect.
Message-ID: <zubfbJXkPKhuyOst-DfICGsvB7bY6kLzN-Qqb9a2kykFXHAkejEME14tTEyUenUCc6LHPUTyW4ThXKUIWBE7hSj6cV9JFKxSZyvi73pUOl8=@protonmail.com>

I found that when the Windows client does not set the MTU value, the IPv6 MTU will be 65535. This will cause IPv6 to not work properly.

Use the `netsh interface ipv6 show subinterfaces` command to view the IPv6 MTU value.

When I checked the mailing list, I saw a question about the MTU value of Apple products, but I only saw a vague answer. Do Apple products still have this problem? Is it reasonable that the default MTU is 1420?

Mailing list regarding the MTU value of Apple products:
https://lists.zx2c4.com/pipermail/wireguard/2017-December/002201.html

From minipli at grsecurity.net  Tue Jul  6 13:32:23 2021
From: minipli at grsecurity.net (Mathias Krause)
Date: Tue, 06 Jul 2021 13:32:23 -0000
Subject: [PATCH 2/2] curve25519-x86_64: solve register constraints with
 reserved registers
In-Reply-To: <20210706132714.8220-1-minipli@grsecurity.net>
References: <20210706132714.8220-1-minipli@grsecurity.net>
Message-ID: <20210706132714.8220-3-minipli@grsecurity.net>

The register constraints for the inline assembly in fsqr() and fsqr2()
are pretty tight on what the compiler may assign to the remaining three
register variables. The clobber list only allows the following to be
used: RDI, RSI, RBP and R12. With RAP reserving R12 and a kernel having
CONFIG_FRAME_POINTER=y, claiming RBP, there are only two registers left
so the compiler rightfully complains about impossible constraints.

Provide alternatives that'll allow a memory reference for 'out' to solve
the allocation constraint dilemma for this configuration.

Signed-off-by: Mathias Krause <minipli at grsecurity.net>

---

Yes, the '+' constraint prefix doesn't need to be repeated for the
alternatives. In fact, it's invalid syntax to do so (see [1]). Also
"+rm" won't do the trick either, as in this case gcc still insists to
have a free register -- even if it would choose a memory operand in the
end.

[1] https://gcc.gnu.org/onlinedocs/gcc/Multi-Alternative.html#Multi-Alternative
---
 src/crypto/zinc/curve25519/curve25519-x86_64.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/src/crypto/zinc/curve25519/curve25519-x86_64.c b/src/crypto/zinc/curve25519/curve25519-x86_64.c
index 79716c425b0c..67f55affcf88 100644
--- a/src/crypto/zinc/curve25519/curve25519-x86_64.c
+++ b/src/crypto/zinc/curve25519/curve25519-x86_64.c
@@ -581,7 +581,7 @@ static inline void fsqr(u64 *out, const u64 *f, u64 *tmp)
 		"  cmovc %%rdx, %%rax;"
 		"  add %%rax, %%r8;"
 		"  movq %%r8, 0(%0);"
-	: "+&r" (tmp), "+&r" (f), "+&r" (out)
+	: "+&r,&r" (tmp), "+&r,&r" (f), "+&r,m" (out)
 	:
 	: "%rax", "%rcx", "%rdx", "%r8", "%r9", "%r10", "%r11", "%rbx", "%r13", "%r14", "%r15", "memory", "cc"
 	);
@@ -743,7 +743,7 @@ static inline void fsqr2(u64 *out, const u64 *f, u64 *tmp)
 		"  cmovc %%rdx, %%rax;"
 		"  add %%rax, %%r8;"
 		"  movq %%r8, 32(%0);"
-	: "+&r" (tmp), "+&r" (f), "+&r" (out)
+	: "+&r,&r" (tmp), "+&r,&r" (f), "+&r,m" (out)
 	:
 	: "%rax", "%rcx", "%rdx", "%r8", "%r9", "%r10", "%r11", "%rbx", "%r13", "%r14", "%r15", "memory", "cc"
 	);
-- 
2.20.1


From minipli at grsecurity.net  Tue Jul  6 13:33:11 2021
From: minipli at grsecurity.net (Mathias Krause)
Date: Tue, 06 Jul 2021 13:33:11 -0000
Subject: [PATCH 0/2] wireguard-linux-compat: grsecurity compat patches
Message-ID: <20210706132714.8220-1-minipli@grsecurity.net>

Hi Jason,

this mini series enhances compatibility of the wireguard-linux-compat
repo with grsecurity. I noticed some build breakage on our 4.14 kernel
which the following patches take care of.

Please apply!

Thanks,
Mathias


Mathias Krause (2):
  compat: better grsecurity compatibility
  curve25519-x86_64: solve register constraints with reserved registers

 src/compat/compat-asm.h                        | 4 ++--
 src/compat/compat.h                            | 4 ++--
 src/crypto/zinc/curve25519/curve25519-x86_64.c | 4 ++--
 3 files changed, 6 insertions(+), 6 deletions(-)

-- 
2.20.1


From minipli at grsecurity.net  Tue Jul  6 13:35:35 2021
From: minipli at grsecurity.net (Mathias Krause)
Date: Tue, 06 Jul 2021 13:35:35 -0000
Subject: [PATCH 1/2] compat: better grsecurity compatibility
In-Reply-To: <20210706132714.8220-1-minipli@grsecurity.net>
References: <20210706132714.8220-1-minipli@grsecurity.net>
Message-ID: <20210706132714.8220-2-minipli@grsecurity.net>

grsecurity kernels tend to carry additional backports and changes, like
commit b60b87fc2996 ("netlink: add ethernet address policy types") or
the SYM_FUNC_* changes. RAP nowadays hooks the latter, therefore no
diversion to RAP_ENTRY is needed any more.

Instead of relying on the kernel version test, also test for the macros
we're about to define to not already be defined to account for these
additional changes in the grsecurity patch without breaking
compatibility to the older public ones.

Also test for CONFIG_PAX instead of RAP_PLUGIN for the timer API related
changes as these don't depend on the RAP plugin to be enabled but just a
PaX/grsecurity patch to be applied. While there is no preprocessor knob
for the latter, use CONFIG_PAX as this will likely be enabled in every
kernel that uses the patch.

Signed-off-by: Mathias Krause <minipli at grsecurity.net>
---
 src/compat/compat-asm.h | 4 ++--
 src/compat/compat.h     | 4 ++--
 2 files changed, 4 insertions(+), 4 deletions(-)

diff --git a/src/compat/compat-asm.h b/src/compat/compat-asm.h
index fde21dabba4f..5bfdb9410933 100644
--- a/src/compat/compat-asm.h
+++ b/src/compat/compat-asm.h
@@ -22,7 +22,7 @@
 #endif
 
 /* PaX compatibility */
-#if defined(RAP_PLUGIN)
+#if defined(RAP_PLUGIN) && defined(RAP_ENTRY)
 #undef ENTRY
 #define ENTRY RAP_ENTRY
 #endif
@@ -51,7 +51,7 @@
 #undef pull
 #endif
 
-#if LINUX_VERSION_CODE < KERNEL_VERSION(5, 4, 76) && !defined(ISCENTOS8S)
+#if LINUX_VERSION_CODE < KERNEL_VERSION(5, 4, 76) && !defined(ISCENTOS8S) && !defined(SYM_FUNC_START)
 #define SYM_FUNC_START ENTRY
 #define SYM_FUNC_END ENDPROC
 #endif
diff --git a/src/compat/compat.h b/src/compat/compat.h
index b2041327d85c..da6912d871fa 100644
--- a/src/compat/compat.h
+++ b/src/compat/compat.h
@@ -830,7 +830,7 @@ static inline void skb_mark_not_on_list(struct sk_buff *skb)
 }
 #endif
 
-#if LINUX_VERSION_CODE < KERNEL_VERSION(4, 20, 0) && !defined(ISRHEL8)
+#if LINUX_VERSION_CODE < KERNEL_VERSION(4, 20, 0) && !defined(ISRHEL8) && !defined(NLA_POLICY_EXACT_LEN)
 #define NLA_POLICY_EXACT_LEN(_len) { .type = NLA_UNSPEC, .len = _len }
 #endif
 #if LINUX_VERSION_CODE < KERNEL_VERSION(5, 2, 0) && !defined(ISRHEL8)
@@ -1127,7 +1127,7 @@ static const struct header_ops ip_tunnel_header_ops = { .parse_protocol = ip_tun
 #undef __read_mostly
 #define __read_mostly
 #endif
-#if (defined(RAP_PLUGIN) || defined(CONFIG_CFI_CLANG)) && LINUX_VERSION_CODE < KERNEL_VERSION(4, 15, 0)
+#if (defined(CONFIG_PAX) || defined(CONFIG_CFI_CLANG)) && LINUX_VERSION_CODE < KERNEL_VERSION(4, 15, 0)
 #include <linux/timer.h>
 #define wg_expired_retransmit_handshake(a) wg_expired_retransmit_handshake(unsigned long timer)
 #define wg_expired_send_keepalive(a) wg_expired_send_keepalive(unsigned long timer)
-- 
2.20.1


From mspaethe at outlook.com  Fri Jul  9 05:48:09 2021
From: mspaethe at outlook.com (matt spaethe)
Date: Fri, 09 Jul 2021 05:48:09 -0000
Subject: wireguard command line, dumplog and GUI pop-up in 0.3.16
In-Reply-To: <DS7PR07MB7591F166B9EC0B30DB10D5B0CF189@DS7PR07MB7591.namprd07.prod.outlook.com>
References: <DS7PR07MB7591F166B9EC0B30DB10D5B0CF189@DS7PR07MB7591.namprd07.prod.outlook.com>
Message-ID: <DS7PR07MB759127E1827ED9A3CBB69D43CF189@DS7PR07MB7591.namprd07.prod.outlook.com>

Hi,

I installed wireguard 0.3.16 as a tunnel on Windows 10 IoT.? When I attempt to dump the log buffer to file, I get a pop up about usage of wireguard command line.? I then verified that I can still perform /dumplog on a second machine with 0.3.14.

I then went to a third machine with 0.3.14 and /dumplog worked successfully.? I then upgraded to 0.3.16 from inside WireGuard Manager on this machine.? In the same Windows shell, the command 'wireguard /dumplog log.txt' results in a pop up about command line usage.

I've been trying to run without Manager because I don't want users accidently shutting down the wireguard tunnel.? So to look at logs, I use the command line option /dumplog.

Thank you,
Matt

From jon at gips.com  Wed Jul 21 17:05:23 2021
From: jon at gips.com (Jonathan Gips)
Date: Wed, 21 Jul 2021 13:05:23 -0400
Subject: Kernel error accessing user memory 
Message-ID: <3E5B9FB3-741F-4953-BD1A-9EA83A6FEF8E@gips.com>

Hello WireGuard Maintainers,

I am seeing the following kernel error when running WireGuard as a kernel module on my Ubuntu 18.04 armv8 device: 

"Internal error: Accessing user space memory outside uaccess.h routines: 96000005 [#1] PREEMPT SMP?

Has anyone come across this? I have seen mention on the Nvidia forums that this issue can arise on newer armv8 CPUs when a kernel module directly accesses userspace memory, with the recommended fix being to use copy_from_user() and copy_to_user(). 

My device's configuration:
--------------------------
- Nvidia Jetson Nano
- CPU: Armv8, 64-bit
- OS: Ubuntu 18.04 / Bionic
- uname -a: Linux jetson-nano 4.9.201-tegra #1 SMP PREEMPT Fri Feb 19 08:40:32 PST 2021 aarch64 aarch64 aarch64 GNU/Linux

Apt package configuration:
--------------------------
- wireguard/bionic-updates,now 1.0.20200513-1~18.04.2 all [installed]
- wireguard-dkms/bionic-updates,now 1.0.20201112-1~18.04.1 all [installed,automatic]
- wireguard-tools/bionic-updates,now 1.0.20200513-1~18.04.2 arm64 [installed]

Full syslog entry for kernel error:
-----------------------------------
May 4 18:45:22 jetson-nano kernel: [628210.837536] Internal error: Accessing user space memory outside uaccess.h routines: 96000005 [#1] PREEMPT SMP
May 4 18:45:22 jetson-nano kernel: [628210.847539] Modules linked in: wireguard ipt_MASQUERADE nf_nat_masquerade_ipv4 nf_conntrack_netlink nfnetlink br_netfilter overlay iwlmvm mac80211 bnep btusb btrtl btbcm btintel iwlwifi zram spidev cfg80211 ipt_REJECT nf_reject_ipv4 xt_limit xt_tcpudp xt_addrtype xt_conntrack iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 ip6table_mangle userspace_alert nvgpu iptable_mangle ip6table_filter ip6_tables nf_conntrack_netbios_ns nf_conntrack_broadcast nf_nat_ftp nf_nat bluedroid_pm nf_conntrack_ftp nf_conntrack iptable_filter ip_tables x_tables
May 4 18:45:22 jetson-nano kernel: [628210.896578] CPU: 1 PID: 3832 Comm: exe Not tainted 4.9.201-tegra #1
May 4 18:45:22 jetson-nano kernel: [628210.902921] Hardware name: custom-jetson-carrier (DT)
May 4 18:45:22 jetson-nano kernel: [628210.908137] task: ffffffc0d02c0e00 task.stack: ffffffc00d7ac000
May 4 18:45:22 jetson-nano kernel: [628210.914140] PC is at do_last+0x54/0xfd0
May 4 18:45:22 jetson-nano kernel: [628210.918057] LR is at path_openat+0x90/0x378
May 4 18:45:22 jetson-nano kernel: [628210.922320] pc : [] lr : [] pstate: 20400145
May 4 18:45:22 jetson-nano kernel: [628210.929792] sp : ffffffc00d7afbc0
May 4 18:45:22 jetson-nano kernel: [628210.933188] x29: ffffffc00d7afbc0 x28: ffffffc0d02c0e00
May 4 18:45:22 jetson-nano kernel: [628210.938595] x27: ffffffc00d7afd48 x26: ffffffc087b23e40
May 4 18:45:22 jetson-nano kernel: [628210.944000] x25: ffffffc00d7afe7c x24: 0000000000000000
May 4 18:45:22 jetson-nano kernel: [628210.949403] x23: 0000000000000041 x22: 0000000000000002
May 4 18:45:22 jetson-nano kernel: [628210.954805] x21: ffffffc037fb9100 x20: ffffffc00d7afcfc
May 4 18:45:22 jetson-nano kernel: [628210.960208] x19: 0000000000020041 x18: 00000000000002c8
May 4 18:45:22 jetson-nano kernel: [628210.965611] x17: 000000000000006e x16: 0000000000000000
May 4 18:45:22 jetson-nano kernel: [628210.971013] x15: 00000000000000f9 x14: 0000000000000000
May 4 18:45:22 jetson-nano kernel: [628210.976416] x13: 0000000000000080 x12: 732f74752f626772
May 4 18:45:22 jetson-nano kernel: [628210.981818] x11: 5f63652f7070612f x10: d0d0d0d0d0d0d0d0
May 4 18:45:22 jetson-nano kernel: [628210.987221] x9 : 96a4abb42eef25a3 x8 : 70d6b989bf66a5a8
May 4 18:45:22 jetson-nano kernel: [628210.992625] x7 : 30017cce4247b231 x6 : e672123d9189800b
May 4 18:45:22 jetson-nano kernel: [628210.998027] x5 : 0000000000000018 x4 : fefefefefefefeff
May 4 18:45:22 jetson-nano kernel: [628211.003430] x3 : ffffffc00d7afcfc x2 : 0000000000000000
May 4 18:45:22 jetson-nano kernel: [628211.008834] x1 : 0000000000000000 x0 : 0000000000000041
May 4 18:45:22 jetson-nano kernel: [628211.014237]
May 4 18:45:22 jetson-nano kernel: [628211.015811] Process exe (pid: 3832, stack limit = 0xffffffc00d7ac000)
May 4 18:45:22 jetson-nano kernel: [628211.022327] Call trace:
May 4 18:45:22 jetson-nano kernel: [628211.024860] [] do_last+0x54/0xfd0
May 4 18:45:22 jetson-nano kernel: [628211.029818] [] path_openat+0x90/0x378
May 4 18:45:22 jetson-nano kernel: [628211.035122] [] do_filp_open+0x70/0xe8
May 4 18:45:22 jetson-nano kernel: [628211.040425] [] do_sys_open+0x174/0x258
May 4 18:45:22 jetson-nano kernel: [628211.045816] [] SyS_openat+0x3c/0x50
May 4 18:45:22 jetson-nano kernel: [628211.050947] [] el0_svc_naked+0x34/0x38
May 4 18:45:22 jetson-nano kernel: [628211.056341] ---[ end trace c646984c02898d73 ]---
May 4 18:45:43 jetson-nano kernel: [628231.843814] INFO: rcu_preempt detected stalls on CPUs/tasks:
May 4 18:45:43 jetson-nano kernel: [628231.849575] Tasks blocked on level-0 rcu_node (CPUs 0-3): P3832
May 4 18:45:43 jetson-nano kernel: [628231.855674] (detected by 1, t=5255 jiffies, g=39064575, c=39064574, q=28414)
May 4 18:45:43 jetson-nano kernel: [628231.862901] exe D 0 3832 6362 0x00000008
May 4 18:45:43 jetson-nano kernel: [628231.868476] Call trace:
May 4 18:45:43 jetson-nano kernel: [628231.871012] [] __switch_to+0x9c/0xc0
May 4 18:45:43 jetson-nano kernel: [628231.876233] [] __schedule+0x270/0x780
May 4 18:45:43 jetson-nano kernel: [628231.881539] [] schedule+0x40/0xa8
May 4 18:45:43 jetson-nano kernel: [628231.886497] [] schedule_timeout+0x1ec/0x420
May 4 18:45:43 jetson-nano kernel: [628231.892322] [] wait_for_common+0xa0/0x140
May 4 18:45:43 jetson-nano kernel: [628231.897975] [] wait_for_completion+0x28/0x38
May 4 18:45:43 jetson-nano kernel: [628231.903887] [] __wait_rcu_gp+0x104/0x138
May 4 18:45:43 jetson-nano kernel: [628231.909452] [] synchronize_rcu+0x80/0x90
May 4 18:45:43 jetson-nano kernel: [628231.915017] [] namespace_unlock+0x58/0x68
May 4 18:45:43 jetson-nano kernel: [628231.920668] [] drop_collected_mounts+0x78/0x88
May 4 18:45:43 jetson-nano kernel: [628231.926751] [] put_mnt_ns+0x3c/0x50
May 4 18:45:43 jetson-nano kernel: [628231.931880] [] free_nsproxy+0x24/0xe8
May 4 18:45:43 jetson-nano kernel: [628231.937182] [] switch_task_namespaces+0x70/0x80
May 4 18:45:43 jetson-nano kernel: [628231.943351] [] exit_task_namespaces+0x24/0x30
May 4 18:45:43 jetson-nano kernel: [628231.949349] [] do_exit+0x2b8/0xa50
May 4 18:45:43 jetson-nano kernel: [628231.954392] [] die+0x194/0x198
May 4 18:45:43 jetson-nano kernel: [628231.959089] [] do_page_fault+0x1ec/0x480
May 4 18:45:43 jetson-nano kernel: [628231.964651] [] do_translation_fault+0x6c/0x80
May 4 18:45:43 jetson-nano kernel: [628231.970648] [] do_mem_abort+0x54/0xb0
May 4 18:45:43 jetson-nano kernel: [628231.975950] [] el1_da+0x24/0xbc
May 4 18:45:43 jetson-nano kernel: [628231.980735] [] path_openat+0x90/0x378
May 4 18:45:43 jetson-nano kernel: [628231.986037] [] do_filp_open+0x70/0xe8
May 4 18:45:43 jetson-nano kernel: [628231.991340] [] do_sys_open+0x174/0x258
May 4 18:45:43 jetson-nano kernel: [628231.996729] [] SyS_openat+0x3c/0x50
May 4 18:45:43 jetson-nano kernel: [628232.001859] [] el0_svc_naked+0x34/0x38
May 4 18:45:43 jetson-nano kernel: [628232.007249] exe D 0 3832 6362 0x00000008
May 4 18:45:43 jetson-nano kernel: [628232.012826] Call trace:
May 4 18:45:43 jetson-nano kernel: [628232.015361] [] __switch_to+0x9c/0xc0
May 4 18:45:43 jetson-nano kernel: [628232.020581] [] __schedule+0x270/0x780
May 4 18:45:43 jetson-nano kernel: [628232.025885] [] schedule+0x40/0xa8
May 4 18:45:43 jetson-nano kernel: [628232.030842] [] schedule_timeout+0x1ec/0x420
May 4 18:45:43 jetson-nano kernel: [628232.036668] [] wait_for_common+0xa0/0x140
May 4 18:45:43 jetson-nano kernel: [628232.042319] [] wait_for_completion+0x28/0x38
May 4 18:45:43 jetson-nano kernel: [628232.048231] [] __wait_rcu_gp+0x104/0x138
May 4 18:45:43 jetson-nano kernel: [628232.053797] [] synchronize_rcu+0x80/0x90
May 4 18:45:43 jetson-nano kernel: [628232.059362] [] namespace_unlock+0x58/0x68
May 4 18:45:43 jetson-nano kernel: [628232.065013] [] drop_collected_mounts+0x78/0x88
May 4 18:45:43 jetson-nano kernel: [628232.071097] [] put_mnt_ns+0x3c/0x50
May 4 18:45:43 jetson-nano kernel: [628232.076228] [] free_nsproxy+0x24/0xe8
May 4 18:45:43 jetson-nano kernel: [628232.081531] [] switch_task_namespaces+0x70/0x80
May 4 18:45:43 jetson-nano kernel: [628232.087702] [] exit_task_namespaces+0x24/0x30
May 4 18:45:43 jetson-nano kernel: [628232.093700] [] do_exit+0x2b8/0xa50

Thanks,
Jon

From aelis.sagot at gmail.com  Wed Jul 21 21:20:28 2021
From: aelis.sagot at gmail.com (Aelis Sagot)
Date: Wed, 21 Jul 2021 22:20:28 +0100
Subject: Windows issue: impossible to add tunnels when install after setting
 program files to folder on other disk via reg value edit
Message-ID: <CAM2QSmYNbXpkz55Ts5WJhGL3+Excy95LqTbu=YS0voScVP1nFQ@mail.gmail.com>

Reproduction:
-Clean windows install
-Be administrator
-2 physical drives
-Change reg value:
HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows\CurrentVersion\ProgramFilesDir
to OtherDriveLetter:/programs
-Install wireguard on windows
-try to add a tunnel
error: "Unable to import configuration: The system cannot move the
file to a different disk drive"

I tried to move the tunnel file directly into
OtherDriveLetter:\programs\WireGuard\Data\Configurations and it is
detected in the GUI
But then I get the error: "open <path to tunnel>. dpapi: The system
cannot find the file specified." when trying to activate that tunnel

From bandry at ut1.org  Tue Jul 13 14:59:35 2021
From: bandry at ut1.org (Bruno UT1)
Date: Tue, 13 Jul 2021 14:59:35 -0000
Subject: Windows Client - issue with LimitedOperatorUI rights
Message-ID: <fb69332b-a210-179a-4692-995f8dc19b5b@ut1.org>

Hi !

I install Wireguard Client on Windows 10 systems.

My users are not administrators, so the registry key LimitedOperatorUI 
is set to 1.

My issue is that the client has to run at least once as administrator to 
work for "Network Configuration Operators".

My script adds users to the administrative group, create the registry 
key then installs Wireguard Client:

 ?- If i don't use the parameter |DO_NOT_LAUNCH, |I have a popup telling 
me I need administrator rights. I don't want that popup, be everything 
works after a reboot.

 ?- If I use the parameter |DO_NOT_LAUNCH,| ||I don't have the popup but 
Wireguard don't start, even after a reboot. If I run Wireguard as 
administrator, it's unlocked for other users.

 ?- A third possibility is to install the client after a reboot, so the 
rights are good, |DO_NOT_LAUNCH |is not needed and no popup appears. But 
I don't want to force that reboot (I use SCCM and it don't like reboots 
before install is complete).

So I think there is a first start issue that needs administrator rights 
even with LimitedOperatorUI? active.

Regards,

Bruno

ps: I send another message (06/28) for another issue. I received "Your 
message to WireGuard awaits moderator approval" but message isn't posted 
and i got no notification from the moderator. Am I doing something wrong ?
||


From denis at shahaya.com  Tue Jul  6 06:18:02 2021
From: denis at shahaya.com (Denis Brodbeck)
Date: Tue, 06 Jul 2021 06:18:02 -0000
Subject: wireguard-windows: client forgets after restart that there was a
 activated tunnel before and won't activate said tunnel anymore
Message-ID: <DB7PR09MB24606350B108C7CB8DA140BFAB1B9@DB7PR09MB2460.eurprd09.prod.outlook.com>

Dear WireGuard-Community,

TL;DR: wireguard forgets after restart that there was a activated tunnel before and won't activate said tunnel anymore unless an admin intervenes

I've deployed wireguard-windows on 50 domain-joined Windows 10 (20H1 x64) notebooks (WireGuard versions range from v0.3.14 - v.03.16) and need your assistance resolving some mysterious behavior.

Some of my users are experiencing random connectivity loses (this example here is the only time I witnessed said behaviour myself):
- admin setups and activates '20_EPNBLE-04' tunnel config
- service 'WireGuardManager' runs
- service 'WireGuardTunnel$20_EPNBLE-04' runs
- everything is fine for days/weeks -- users reboot usually daily
- user reboots / comes back from weekend
- service 'WireGuardManager' runs
- service 'WireGuardTunnel$20_EPNBLE-04' does not exist
- config '20_EPNBLE-04.conf.dpapi' under 'C:\Program Files\WireGuard\Data\Configurations' still exists, it's just not active any more
- config '20_EPNBLE-04.conf.dpapi' *stays* inactive (multiple reboots), unless an admin re-activates it via WireGuard UI

My 99% windows environment:
- users have no admin privileges
- 'LimitedOperatorUI' is disabled, so users have no privileges to mess with network or wireguard config or tunnel state
- all clients have the 'Windows Baseline Security' applied
- each notebook has a unique config file (above example: 20_EPNBLE-04)
- that config has been enabled via wireguard UI (before deployment by an administrative account) and works
- that tunnel works 99%, but sometimes, just sometimes, the tunnel service is gone after reboot
- I can't spot a pattern to which or when a client looses connectivity
- Most clients have no issues whatsoever, but maybe 20% of those clients had the previously described vpn tunnel loss, but until now no client had this issue twice
- WireGuard log doesn't show anything interesting, because that config file is obviously inactive, and after I click 'Activate' the tunnel works instantly

I read (parts of) the source code and tried to understand how 'WireGuardManager' keeps tabs on which of the available vpn config needs to be reestablished after reboot, but I didn't grasp the business logic yet (I'm a longtime Go developer myself, so reading is usually not the issue - but maybe I need another coffee :/).

Hope you have some pointers on how to resolve this.

Cheers
D. Brodbeck

From dwest at galois.com  Tue Jul  6 16:57:42 2021
From: dwest at galois.com (Dan West)
Date: Tue, 06 Jul 2021 16:57:42 -0000
Subject: MacOS client v1.0.13 multiple tunnel issue
Message-ID: <86DB3970-3B57-4BC3-99FE-F18D17BB7058@galois.com>

I upgraded my wireguard client from v1.0.12 to v1.0.13 on my MacBook and when I connect using multiple tunnels to different sites, it no longer works correctly.  I am not entirely sure if this is a problem with routing on the client side or if there is a bug preventing additional tunnels from connecting to other sites.  It seems that the first tunnel I connect to will work, but any subsequent tunnels will not.  So for example, if I connect to SiteA and then SiteB, SiteA will work and SiteB will not.  If I connect to SiteB and then SiteA, SiteB will work but SiteA will not.  This is tested by pinging the VPN's gateway address after establishing each connection.  Here is a redacted copy of my tunnel configs:

### Site A Config ###

[Interface]
PrivateKey = xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx=
Address = 10.1.1.100/24
DNS = 10.1.1.1, example.com

[Peer]
PublicKey = yyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyy=
AllowedIPs = 10.1.1.0/16, aa.aaa.aaa.aa/25 #Public IP space for Site-A DMZ
Endpoint = site-a.example.com:32123

### Site B Config ###

[Interface]
PrivateKey = xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx=
Address = 10.2.1.100/24
DNS = 10.2.1.1, example.com

[Peer]
PublicKey = yyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyy=
AllowedIPs = 10.2.1.0/16, bb.bbb.bbb.bb/25 #Public IP space for Site-B DMZ
Endpoint = site-b.example.com:32123

It is worth noting that this has been tested using both the ?On-Demand? function as well as the MacOS built-in Network management tool.  The version history says ?Jun 17, 2021 - 1.0.13 - This version includes internal fixes and API changes for newer iOS versions?, so the actual changes are a little opaque. 

I ran a TCPdump on Site-B (or the second site) to see if traffic is even getting to the VPN endpoint and it is not seeing anything.  I also ran a packet capture on the border firewalls and did not see any traffic attempting to hit that location.  The client does not show any handshake activity, so I believe the issue is specifically in the client or maybe how it is trying route the second connection?

A temporary fix is to downgrade back to v1.0.12, but due to limitations of the Apple Appstore, it is only feasible if you happen to have a backup copy of the client.  I just happened to have a backup copy, so I was able to test with both clients.  

Has anyone else ran into this issue or has any suggestions?  I can provide more information on the topology if you think it would be helpful.

Thanks,

Dan

Dan West
Systems Administrator
Galois Inc.
http://corp.galois.com





From iiiiiiiiiiii at cock.li  Wed Jul 14 23:55:16 2021
From: iiiiiiiiiiii at cock.li (iiiiiiiiiiii)
Date: Wed, 14 Jul 2021 23:55:16 -0000
Subject: Android: more than one tunnel
In-Reply-To: <d994accd-d1a9-4f9a-bbf9-d6b99036b177@www.fastmail.com>
References: <mailman.0.1601114111.20018.wireguard@lists.zx2c4.com>
 <296f17d0-3f0b-6021-7bb3-4b4bf576f6c7@spam-free.eu>
 <e8e94195-b1f0-df62-4393-929f20f762a3@spam-free.eu>
 <d994accd-d1a9-4f9a-bbf9-d6b99036b177@www.fastmail.com>
Message-ID: <20210714235513.zmgf663ukvjb5dqy@cock.li>

I have experienced the same issue without overlapping AllowedIPs
subnets, so I believe it is Android-specific.

On 21/07/15 10:09AM, Eric Light wrote:
> Hi Chris!
> 
> I don't have any experience with the Android implementation specifically... but most likely, your two tunnels have overlapping AllowedIPs ranges.  When this happens, bringing up the second interface will override the routing created by the first interface.
> 
> Most commonly, this happens when someone configures both interfaces with AllowedIPs = 0.0.0.0/0.  To fix this problem, make sure there's no overlap of AllowedIPs in your tunnel definitions.
> 
> I hope this helps  :)
> 
> (my apologies if the problem is something Android-specific that I don't know about)
> 
> E
> 
> --------------------------------------------
> Q: Why is this email five sentences or less?
> A: http://five.sentenc.es
> 
> On Wed, 14 Jul 2021, at 17:49, Chris wrote:
> > I have two distinct tunnels defined on Android.
> > However, I can only toggle between the two. I cannot have both activated at the 
> > same time.
> > I need both connections.
> > Am I missing anything?
> > Why not several tunnels with several wg servers???
> > Why is this being prevented?
> > 
> > BR
> > Chris
> > 
> > 

From miguel.arroz at gmail.com  Tue Jul 13 04:18:19 2021
From: miguel.arroz at gmail.com (Miguel Arroz)
Date: Tue, 13 Jul 2021 04:18:19 -0000
Subject: Connect to more than one tunnel simultaneously on macOS
Message-ID: <AB949986-F736-458F-B3BD-4E70412939EC@gmail.com>

Hi,

First of all, thanks for all the work on Wireguard, including bringing a really nice integration to iOS and macOS. This works so smoothly that I completely forget I?m using VPN software, which I suppose is one of the best things that can be said about VPN software. :)

So, I was configuring a second tunnel on my Mac this weekend and realized the Mac app only allows connecting to one tunnel simultaneously. Even if it shows both tunnels connected, traffic only flows through one.

I?m tempted to take a look at the source and see if I can come up with a patch, but before that, I would like to ask: was this was done intentionally (due to some limitation on an API like Network Extension)? Or is it something desirable but not yet implemented?

Thank you,

Miguel Arroz


From remi.lapeyre at lenstra.fr  Fri Jul 16 12:53:32 2021
From: remi.lapeyre at lenstra.fr (=?utf-8?Q?R=C3=A9mi_Lapeyre?=)
Date: Fri, 16 Jul 2021 12:53:32 -0000
Subject: Missing route table configuration on "Quick Start" section
Message-ID: <E42173AE-ECFE-46E2-8F33-65F113F7A601@lenstra.fr>

Hi, in the ?Command-line Interface? paragraph of the ?Quick Start? section at https://www.wireguard.com/quickstart/#command-line-interface it seems to me like the list of commands given: 

# ip link add dev wg0 type wireguard
# ip address add dev wg0 192.168.2.1/24
# ip address add dev wg0 192.168.2.1 peer 192.168.2.2
# wg setconf wg0 myconfig.conf
# wg set wg0 listen-port 51820 private-key /path/to/private-key peer ABCDEF... allowed-ips 192.168.88.0/24 endpoint 209.202.254.14:8172
# ip link set up dev wg0


is missing the route table configuration with ?ip rule add ??. Those are given just a bit later in the GIF that shows 'wg-quick? working but I?m not sure whether configuring the wireguard interface with the command given will actually work.

Cheers,
R?mi

From simonmcnair at gmail.com  Tue Jul  6 13:25:56 2021
From: simonmcnair at gmail.com (Simon McNair)
Date: Tue, 06 Jul 2021 13:25:56 -0000
Subject: ERX wireguard assistance please
Message-ID: <7388816e-1f87-fa5e-f509-7276a6d3b80d@gmail.com>

Hi,
I've searched to try and find the solution to my issue but I'm no expert 
and I'm not entirely sure what to search for.? I would appreciate your 
help please.

In summary, If I connect to my LAN via local WiFi I successfully connect 
to wireguard:
interface: wg0
 ? public key: <removed>
 ? private key: (hidden)
 ? listening port: 12345

peer: <removed>
 ? endpoint: 192.168.100.102:50084
 ? allowed ips: 10.250.250.5/32
 ? latest handshake: Now
 ? transfer: 157.33 KiB received, 536.33 KiB sent
 ? persistent keepalive: every 25 seconds

However if I try and connect via a remote network it fails.? My DDNS is 
resolving correctly to the correct IP so I can only think that firewall 
rules or NAT is somehow causing the issue.? Has anyone come across this 
before please ?
I don't know how to monitor my WAN port 12345 for activity, I tried 
tcpdump -n -v -i wg0 to no real benefit.


The ERX is running v2.0.9-hotfix.2 and 
e50-v2-v1.0.20210606-v1.0.20210424.deb


High level the infrastructure is:

LAN: 192.168.100.0/24
wg ip : 10.250.250.1/24
listen port:12345
route-allowed-ips:true
peer allowed ip is : 10.250.250.5/32
Firewall policy WAN_LOCAL default action drop, rule 2 wireguard 
destination port 12345, protocol udp, action accept


I would appreciate any help you can provide. The aim is to be able to 
access resources in the 192.168.100.0/24 subnet using routed ip from the 
10.250.250.0/24 transit network.

My apologies if my syntax or understanding is flawed.

Regards

Simon



From simonmcnair at gmail.com  Wed Jul 14 09:06:22 2021
From: simonmcnair at gmail.com (Simon McNair)
Date: Wed, 14 Jul 2021 09:06:22 -0000
Subject: Android: more than one tunnel
In-Reply-To: <e8e94195-b1f0-df62-4393-929f20f762a3@spam-free.eu>
References: <mailman.0.1601114111.20018.wireguard@lists.zx2c4.com>
 <296f17d0-3f0b-6021-7bb3-4b4bf576f6c7@spam-free.eu>
 <e8e94195-b1f0-df62-4393-929f20f762a3@spam-free.eu>
Message-ID: <b87f5a75-40e6-a55e-ae5c-74040748a624@gmail.com>

I think this is a limitation of android, not wireguard.


https://developer.android.com/guide/topics/connectivity/vpn 
<https://developer.android.com/guide/topics/connectivity/vpn>

"
There?s only one active service per user or profile. Starting a new 
service, automatically stops an existing service"

Sent again due to being formatted as HTML rather than text and/or 
forgetting to cc the list.

Simon


On Wed, 14 Jul 2021, 06:54 Chris, <wireguard at spam-free.eu 
<mailto:wireguard at spam-free.eu>> wrote:

    I have two distinct tunnels defined on Android.
    However, I can only toggle between the two. I cannot have both
    activated at the
    same time.
    I need both connections.
    Am I missing anything?
    Why not several tunnels with several wg servers???
    Why is this being prevented?

    BR
    Chris


From sven.auhagen at voleatech.de  Sat Jul 17 10:09:13 2021
From: sven.auhagen at voleatech.de (Sven Auhagen)
Date: Sat, 17 Jul 2021 10:09:13 -0000
Subject: Mac APP DNS Search Domain
Message-ID: <20210717100909.lzi5mwsv5hb57w3w@svensmacbookair.sven.lan>

Hi,

the DNS search domain in the Wireguard Mac APP does not seem to work properly.
The search domain is added properly and I can see it in the DNS configuration
for the scoped query:

resolver #2
search domain[0] : mytestdomain.com
nameserver[0] : 192.168.6.1
if_index : 17 (utun4)
flags    : Scoped, Request A records
reach    : 0x00000002 (Reachable)

but the global resolver is using a mix of my LAN and Wireguard DNS resolver:

resolver #1
search domain[0] : sven.lan
nameserver[0] : 192.168.6.1
if_index : 17 (utun4)
flags    : Supplemental, Request A records
reach    : 0x00000002 (Reachable)
order    : 102200

I found a discussion on Github about the issue here:
https://github.com/tailscale/tailscale/issues/101#issuecomment-639286398

suggesting that the following will fix it:

--- a/Sources/WireGuardKit/PacketTunnelSettingsGenerator.swift
+++ b/Sources/WireGuardKit/PacketTunnelSettingsGenerator.swift
@@ -88,7 +88,7 @@ class PacketTunnelSettingsGenerator {
             let dnsSettings = NEDNSSettings(servers: dnsServerStrings)
             dnsSettings.searchDomains = tunnelConfiguration.interface.dnsSearch
             if !tunnelConfiguration.interface.dns.isEmpty {
-                dnsSettings.matchDomains = [""] // All DNS queries must first go through the tunnel's DNS
+                dnsSettings.matchDomains = [""] + dnsSettings.searchDomains // All DNS queries must first go through the tunnel's DNS
             }
             networkSettings.dnsSettings = dnsSettings
         }

I do not have an Apple Developer Account so I am not able to compile the code and test it.
Can anyone take a look as this would be a great help.

Best
Sven



Beste Gr??e/Best regards

Sven Auhagen
Dipl. Math. oec., M.Sc.
Voleatech GmbH
HRB: B 754643
USTID: DE303643180
Grathwohlstr. 5
72762 Reutlingen
Tel: +49 7121539550
Fax: +49 71215395599
E-Mail: sven.auhagen at voleatech.de
www.voleatech.de<https://www.voleatech.de>
[https://www.voleatech.de/wp-content/uploads/2021/03/vtair-emailbanner-enterprise.jpg] <https://www.voleatech.de/de/vtair/>
Diese Information ist ausschlie?lich f?r den Adressaten bestimmt und kann vertraulich oder gesetzlich gesch?tzte Informationen enthalten. Wenn Sie nicht der bestimmungsgem??e Adressat sind, unterrichten Sie bitte den Absender und vernichten Sie diese Mail. Anderen als dem bestimmungsgem??en Adressaten ist es untersagt, diese E-Mail zu lesen, zu speichern, weiterzuleiten oder ihren Inhalt auf welche Weise auch immer zu verwenden. F?r den Adressaten sind die Informationen in dieser Mail nur zum pers?nlichen Gebrauch. Eine Weiterleitung darf nur nach R?cksprache mit dem Absender erfolgen. Wir verwenden aktuelle Virenschutzprogramme. F?r Sch?den, die dem Empf?nger gleichwohl durch von uns zugesandte mit Viren befallene E-Mails entstehen, schlie?en wir jede Haftung aus.

From tom.ty89 at gmail.com  Fri Jul 30 03:27:45 2021
From: tom.ty89 at gmail.com (Tom Yan)
Date: Fri, 30 Jul 2021 11:27:45 +0800
Subject: wg-quick with default route fails on nfs root filesystem
In-Reply-To: <e4fef9da-ee7f-d1db-036e-b294785d9ee8@spam-free.eu>
References: <e4fef9da-ee7f-d1db-036e-b294785d9ee8@spam-free.eu>
Message-ID: <CAGnHSEmp5Dnk2y78Rt=9OdrkWjX+0kXgaCu=eQpzYzfyV-SwbA@mail.gmail.com>

Hi Chris,

So I did a test and took a look at the log:
...
wg-quick[2003]: [#] ip -4 route add 0.0.0.0/0 dev aliyun table 51820
wg-quick[2003]: [#] ip -4 rule add not fwmark 51820 table 51820
wg-quick[2003]: [#] ip -4 rule add table main suppress_prefixlength 0
...

What if wg-quick adds the route after adding both rules? AFAIK, a rule
will be a no-op if the table it looks up is empty.

Regards,
Tom

On Wed, 14 Jul 2021 at 18:00, Chris <wireguard at spam-free.eu> wrote:
>
> When wg-quick detects a default route through the tunnel it does this through a
> new routing table with a default route.
> However not to destroy the existing non-default routes these will looked up and
> used first. This results in the follwing policy rule entries:
> (The priority numers may be different from system to system)
>
> 32764:    from all lookup main suppress_prefixlength 0
> 32765:    not from all fwmark 0xca6c lookup 51820
>
> It is very important of course, that the suppress_prefix rule comes first in the
> list, before
> the second rule introduces the new default route (preventig the wireguard
> traffic through it's own tunnel).
>
> The way to archive this is done by the following command sequence:
>
> ip -4 rule add not fwmark 51820 table 51820
> ip -4 rule add table main suppress_prefixlength 0
>
> The sequence of the commands is important as the latter command gets the higher
> priority (lower numer).
>
> BUT:
> In case your root filesystem needs the local network, the second command will
> not be reached as the
> first command (setting the new default route) kills the root filesystem and the
> system stalls!!!!!!
>
> One possible solution:
> Instead of adding the suppress_prefixlength 0 command secondly it must be first.
> The you must find the priority of that rule and the add the default route with
> the same priority.
> A rule with same priority will be added AFTER the other rules.
>
> Example:
> ip -4 rule add table main suppress_prefixlength 0
> PRIO=$(ip rule list from all|grep suppress_prefixlength|sed -e
> '{s/^\(.*\)\:.*/\1/;q}')
> ip -4 rule add not fwmark 51820 table 51820 priority $PRIO
>
> This will lead to the correct sequence:
> 32765:    from all lookup main suppress_prefixlength 0
> 32765:    not from all fwmark 0xca6c lookup 51820
> (Note the same priority number)
>
> There are probably better ways to cirumvent cutting off the root filesystem.
>
> Chris
>