From harald.dunkel at aixigo.com  Wed Nov  3 09:15:14 2021
From: harald.dunkel at aixigo.com (Harald Dunkel)
Date: Wed, 3 Nov 2021 10:15:14 +0100
Subject: Split DNS for macOS
In-Reply-To: <E3607380-8E46-42CC-80B5-BB38F206E9BC@slarew.net>
References: <20211028071638.88001-1-stephen@slarew.net>
 <0e80daf1-b4c2-b177-28ca-79e6575a339c@baywinds.org>
 <ED3FB4B3-96AD-4B98-86EC-D9F7999A530F@slarew.net>
 <K25247-14.1635527002@spamhaus.org>
 <E3607380-8E46-42CC-80B5-BB38F206E9BC@slarew.net>
Message-ID: <54f3f2a6-7c58-dc42-56bd-fffe74bc3a18@aixigo.com>

Hi folks,

I really like this patch. Currently DNS on MacOS is unable to resolve
both my local DNS names and the domain in the office in parallel, if
Wireguard is enabled. I have to use somehost.local to fall back to
zeroconf for my LAN as a workaround, which is pretty annoying.

My suggestion would be to set SupplementalMatchDomains instead(!) of
SearchDomains, using the current config file syntax without '~'. Since
SupplementalMatchDomainsNoSearch is disabled by default, setting
SupplementalMatchDomains is sufficient to configure both lists. See

https://developer.apple.com/business/documentation/Configuration-Profile-Reference.pdf

This has to be verified, of course.


Regards
Harri



On 2021-10-29 23:07:38, Stephen Larew wrote:
>> On Oct 29, 2021, at 10:03, Andrew Fried <afried at spamteq.com> wrote:
>>
>>> On Oct 29, 2021, at 08:33, Stephen Larew <stephen at slarew.net> wrote:
>>>
>>>> On Oct 28, 2021, at 02:58, Bruce Ferrell <bferrell at baywinds.org> wrote:
>>>>
>>>> On 10/28/21 12:16 AM, Stephen Larew wrote:
>>>>> For many months now, I have been running a patched WireGuard macOS app
>>>>> that enables a split DNS configuration. I would like to try to upstream
>>>>> my patches for split DNS.
>>>>>
>>>>> There has been some interest in this patch:
>>>>> - "Mac APP DNS Search Domain" thread from July and August 2021 [1]
>>>>> - A commenter on my GitHub fork of wireguard-apple.
>>>>>
>>>>> What is split DNS? It allows sending DNS queries to a specific server
>>>>> based on the domain name. Systemd-resolved calls it a routing domain.
>>>>> Apple's Network Extension framework calls it a match domain.  Split DNS
>>>>> is especially useful for internal DNS servers.
>>>>>
>>>>> For example, if corp.example.com is a routing domain for the DNS server
>>>>> at 192.0.2.1 (only accessible over WireGuard), then
>>>>> server.corp.example.com is resolved using 192.0.2.1 while
>>>>> www.example.com is resolved using some other DNS resolver (depending on
>>>>> the other network settings in macOS).
>>>>>
>>>>> The proposed patch adds new syntax to the wg-quick DNS= line.
>>>>> Specifically, a tilde prefixed domain is treated as a routing domain.
>>>>> Multiple routing domains can be added.
>>>>>
>>>>> Limitations:
>>>>> - Needs modifications to iOS UI to work on iOS.
>>>>> - Only matching routing domains are sent to the DNS servers specified in
>>>>>   the DNS= config line.  No separate fallback catch-all DNS server can
>>>>>   be set.
>>>>> - Routing/match domains are also included in the list of search domains.
>>>>>   This could be changed with the matchDomainsNoSearch API, but lacking
>>>>>   more UI or config file changes to expose this option to the user, I
>>>>>   went with the default.
>>>>>
>>>>> [1] https://lore.kernel.org/wireguard/20210810074232.aah5ktq5yzysaaey at SvensMacBookAir-2.local/T/
>>>>> [2] https://github.com/slarew/wireguard-apple/commit/6ebc356d9e11ab91443e06de5e89f1af57fcdff8
>>>>
>>>> That seems to be a redefinition of the existing definition of split DNS.
>>>>
>>>> Most usually, split DNS is done at the DNS server and different zones are served to the resolver based on various criteria... Usually the origination IP of the query;  If the query comes from a client on your LAN, or a particular subnet, the contents of a particular, private, zone are returned.  If the query comes from, say the internet,  a public zone is returned.
>>>>
>>>> YOUR description, is how DNS works in general... Except  your patch also seems to either bypass the resolver libraries or wedge itself in front of them The system resolver libraries well tested and understood and they handle the following very nicely.
>>>>
>>>> There is the issue of what happens with large DNS responses.  Any DNS response over 512 bytes UDP fails and is required to be retried as a TCP query, which can handle the large response.  It's late and I'm too tired to look it up, but there IS an RFC for this.
>>>>
>>>> It's a little known issue, but I've seen it when working with other VPN products that ignored/didn't understand the behavior. The results weren't pretty and really embarrassing.
>>>>
>>>> Systemd has a bad habit of re-inventing the wheel... Badly. Eventually it get's sorted, but is that really progress? In the long haul, "move fast and break things" is a MAJOR pita for the vast majority of us.  But some like it.
>>>>
>>>> For some real fun, look into DHCPCD.  It faithfully implements a particular RFC.  In some networking environments using VPNs, it breaks routing horribly.... But it meets the RFC.  You'll find it in Raspbian by default, and most other Debian derived distros.  I automatically rip it out and replace it with the also available ISC DHCP client.  That one is fully compatible with Windows, OS X, iOS, and every android and smart device I could test it with.  A DHCP server configured to be compatible with DHCPCD broke all of the previously named.
>>>>
>>>> I'm giving this opinion away for free, so it's worth what you paid for it.
>>>
>>> Regardless of naming or definitions, I think the ability for a *local device* to route DNS queries to different DNS servers based on domain matching criteria is certainly useful. It?s not always possible or desirable to control and configure an upstream DNS server. Hence, this patch enables the local device to do split DNS.
>>>
>>> To be clear, this patch does not bypass or wedge around anything. In fact, it configures the native macOS DNS settings in the appropriate manner to effect a split DNS configuration.
>>>
>>> As a result of controlling the native macOS DNS resolution logic, any feature, absent or present, in the macOS DNS resolver libraries should be unaffected. This includes the large DNS response and TCP behavior. I do not expect the small/large UDP/TCP DNS features to change behavior when using a split DNS configuration as proposed in this patch.
>>>
>>> -Stephen
>>
>> Hi Stephen,
>>
>> A better solution to your problem would be to deploy DNSDIST:
>> 	https://dnsdist.org/
>>
>> I for one would hope that esoteric requests that address a solution for less than 1% of the users would be rejected with the overall goal of preventing feature creep and bloat.
>>
>> Andrew
> 
> DNSDIST may allow (I have not tried) one to create a split DNS scenario, but it is an extra piece of software that would need to be discovered, installed, configured, and maintained by a user or system administrator. I do not believe it would properly integrate with the macOS DNS resolution machinery. I do not believe it is a better solution to the problem.
> 
> As I understand it, under some circumstances, the Tailscale macOS app also implements split DNS in roughly the same manner as done in my patch. Namely, the tailscale app appears to use the Network Extension APIs to directly integrate with the macOS DNS resolution machinery. Perhaps the relevant difference is that tailscale approaches configuration differently (not based on wg-quick) than the WireGuard macOS app.
> 
> I do not have statistics or polling on who desires this split DNS feature. I have received private and public requests to upstream the feature. Tailscale also implements split DNS; presumable customers demand it. I suspect if the feature was available to users of the WireGuard app, then it would be used with precision to great effect. Users who do not need this split DNS feature do not lose any previous functionality in the macOS WireGuard app.
> 
> Personally, my one hesitation with this patch is that, as currently implemented, a new syntax is added to the wg-quick config file (tilde prefixed route/match domains). My patch does not address compatibility issues nor does it add documentation to wg-quick for the new syntax.
> 
> -Stephen
> 


From am2 at rcsnetwork.com  Tue Nov  2 18:06:55 2021
From: am2 at rcsnetwork.com (am2 at rcsnetwork.com)
Date: Tue, 2 Nov 2021 14:06:55 -0400
Subject: the format of the specified network name is invalid
In-Reply-To: <CAHFBhiah1NgozmGWtTgvZEfhyxDNA0EySiZao+fsK-Z9_1QLpQ@mail.gmail.com>
References: <CAHFBhiah1NgozmGWtTgvZEfhyxDNA0EySiZao+fsK-Z9_1QLpQ@mail.gmail.com>
Message-ID: <CAHFBhibg_LDeUZ=96d2u=7kh2py0zC5Abp_L0S7Krd2xptZOyQ@mail.gmail.com>

Ever since the last update to the windows client, I receive the
following error when trying to start a tunnel: "the format of the
specified network name is invalid" I can connect using the android
client and the windows client worked fine before the last update so
I'm pretty sure it has something to do with the windows client    Has
anyone else encountered this or know of a solution?  Wireguard is
basically unusable from this client at the moment.

From wireguard-mail at chil.at  Tue Nov  2 22:15:50 2021
From: wireguard-mail at chil.at (Christoph Loesch)
Date: Tue, 2 Nov 2021 23:15:50 +0100
Subject: WireGuard connection without interface-address / linknet
Message-ID: <2b2f3fe4-ca37-84ad-1418-c57f5d13e32b@chil.at>

Hi,

I am using WireGuard on an OpenWRT VM as server for clients basically to reach the server's internal LAN at 10.5.44.0/24

As clients I currently use different WireGuard implementations like:
- Ubiquiti EdgeRouter (EdgeOS v2 based on Debian/stretch) with package from: https://github.com/WireGuard/wireguard-vyatta-ubnt/releases
- Mikrotik RouterOS v7.1 with Mikrotik own (at the moment beta) implementation
- Windows 10 Client using https://download.wireguard.com/windows-client/wireguard-installer.exe

Server config looks like this:

config interface 'wg0'
 ??????? option proto 'wireguard'
 ??????? option private_key 'cNT...8Hc='
 ??????? option listen_port '51820'
 ??????? list addresses '172.27.0.1/16'
config wireguard_wg0
 ??????? option description 'router-test'
 ??????? option public_key 'qT5...YGo='
 ??????? option preshared_key 'Dle...ozI='
 ??????? option persistent_keepalive '25'
 ??????? option route_allowed_ips '1'
 ??????? list allowed_ips '172.27.34.28/32'
 ??????? list allowed_ips '10.34.28.0/24'

Client config looks like this:

[Interface]
PrivateKey = mDk...uVs=
Address = 172.27.34.28
[Peer]
PublicKey = 1sy...IkU=
PresharedKey = Dle...ozI=
AllowedIPs = 172.27.0.1/32,10.5.44.0/24
Endpoint = server.mydomain.at:51820
PersistentKeepalive = 25

Clients should just be able to reach the server's 10.5.44.0/24 subnet and this subnet should be able to reach clients at (in this one example) 10.34.28.0/24
Now this works all well as expected but I would like to omit using the 172.27.* addresses/linknet if possible because I dont really need/use this as it is only defined for the WireGuard tunnel itself.

On the EdgeRouter this also works perfectly fine if I remove the 172.27.* address on both sides. It is still possible to reach the other end repestively.
On the Mikrotik device and on the Windows client (using exact same configuration) it does not work as soon as I remove just the 172.27. address/linknet from configuration.
(I didn't test other clients yet)

I guess on the EdgeRouter this works because I set: set interfaces wireguard wg0 **route-allowed-ips true** - so the corresponding routes are added.
The Mikrotik device and the Windows client do not offer such an option, so those routes have to be added manually I guess?
But why does that work "out of the box" as soon as I add any linknet (172.27.* in my example) for the tunnel itself to the configuration?

Is this a fault in the implementation and I should file a bug report or is that expected that way?

(it's not a big issue using the linknet, I am just curious and would like avoid using it if it is not neccessarily required)

Thanks for any thoughts and kind regards,
Christoph


From mhp.driessen at gmail.com  Wed Nov  3 09:42:52 2021
From: mhp.driessen at gmail.com (Matty Driessen)
Date: Wed, 3 Nov 2021 10:42:52 +0100
Subject: Split DNS for macOS
In-Reply-To: <54f3f2a6-7c58-dc42-56bd-fffe74bc3a18@aixigo.com>
References: <20211028071638.88001-1-stephen@slarew.net>
 <0e80daf1-b4c2-b177-28ca-79e6575a339c@baywinds.org>
 <ED3FB4B3-96AD-4B98-86EC-D9F7999A530F@slarew.net>
 <K25247-14.1635527002@spamhaus.org>
 <E3607380-8E46-42CC-80B5-BB38F206E9BC@slarew.net>
 <54f3f2a6-7c58-dc42-56bd-fffe74bc3a18@aixigo.com>
Message-ID: <CAHsVW7N0EYtUegNd278fFigGv6oucE5+NbkuScKNGnHfaOau5Q@mail.gmail.com>

Hello,

I just want to chime in here and say that I think the current
implementation of search domains is simply not working the way it
should on the MacOS client.

My use case is pretty common, an internal DNS server that has entries
for internal servers. I defined a search domain in the WireGuard
configuration; DNS = 10.13.13.1 mydomain.internal. The search domain
is for convenience, so I can just use the servername instead of
servername.mydomain.internal. Now this works fine when I route all the
traffic through the VPN (AllowedIPs = 0.0.0.0/0) but the search domain
is completely ignored when I only route the traffic I need to
(AllowedIPs = 10.13.13.0/24 192.168.0.0/24).

I don't think this is a configuration error on my side. The DNS
responds fine when using servername.mydomain.internal. This problem is
even mentioned in the "WireGuard macOS & iOS TODO List"
9. matchDomains=[??] doesn?t do what the documentation says.
Specifically, DNS servers are not used if allowed IPs isn?t 0.0.0.0/0.

The description isn't 100% accurate (or outdated), the DNS server is
used but the search domain isn't being set on the primary resolver.
Some have solved this issue by adding the search domains to the list
of matchDomains; dnsSettings.matchDomains = [""] +
dnsSettings.searchDomains. But that way the DNS server specified in
WireGuard is still the primary resolver for all DNS queries.

Here is a link on how OpenVPN handles this and I think it's how it
should work when not using AllowedIPs 0.0.0.0/0.

https://openvpn.net/faq/how-does-ios-interpret-pushed-dns-servers-and-search-domains/
On a split-tunnel, where redirect-gateway is not pushed by the server,
and at least one pushed DNS server is present:
- route all DNS requests through pushed DNS server(s) if no added
search domains.
- route DNS requests for added search domains only, if at least one
added search domain.

Regards,
Matty

On Wed, Nov 3, 2021 at 10:20 AM Harald Dunkel <harald.dunkel at aixigo.com> wrote:
>
> Hi folks,
>
> I really like this patch. Currently DNS on MacOS is unable to resolve
> both my local DNS names and the domain in the office in parallel, if
> Wireguard is enabled. I have to use somehost.local to fall back to
> zeroconf for my LAN as a workaround, which is pretty annoying.
>
> My suggestion would be to set SupplementalMatchDomains instead(!) of
> SearchDomains, using the current config file syntax without '~'. Since
> SupplementalMatchDomainsNoSearch is disabled by default, setting
> SupplementalMatchDomains is sufficient to configure both lists. See
>
> https://developer.apple.com/business/documentation/Configuration-Profile-Reference.pdf
>
> This has to be verified, of course.
>
>
> Regards
> Harri
>
>
>
> On 2021-10-29 23:07:38, Stephen Larew wrote:
> >> On Oct 29, 2021, at 10:03, Andrew Fried <afried at spamteq.com> wrote:
> >>
> >>> On Oct 29, 2021, at 08:33, Stephen Larew <stephen at slarew.net> wrote:
> >>>
> >>>> On Oct 28, 2021, at 02:58, Bruce Ferrell <bferrell at baywinds.org> wrote:
> >>>>
> >>>> On 10/28/21 12:16 AM, Stephen Larew wrote:
> >>>>> For many months now, I have been running a patched WireGuard macOS app
> >>>>> that enables a split DNS configuration. I would like to try to upstream
> >>>>> my patches for split DNS.
> >>>>>
> >>>>> There has been some interest in this patch:
> >>>>> - "Mac APP DNS Search Domain" thread from July and August 2021 [1]
> >>>>> - A commenter on my GitHub fork of wireguard-apple.
> >>>>>
> >>>>> What is split DNS? It allows sending DNS queries to a specific server
> >>>>> based on the domain name. Systemd-resolved calls it a routing domain.
> >>>>> Apple's Network Extension framework calls it a match domain.  Split DNS
> >>>>> is especially useful for internal DNS servers.
> >>>>>
> >>>>> For example, if corp.example.com is a routing domain for the DNS server
> >>>>> at 192.0.2.1 (only accessible over WireGuard), then
> >>>>> server.corp.example.com is resolved using 192.0.2.1 while
> >>>>> www.example.com is resolved using some other DNS resolver (depending on
> >>>>> the other network settings in macOS).
> >>>>>
> >>>>> The proposed patch adds new syntax to the wg-quick DNS= line.
> >>>>> Specifically, a tilde prefixed domain is treated as a routing domain.
> >>>>> Multiple routing domains can be added.
> >>>>>
> >>>>> Limitations:
> >>>>> - Needs modifications to iOS UI to work on iOS.
> >>>>> - Only matching routing domains are sent to the DNS servers specified in
> >>>>>   the DNS= config line.  No separate fallback catch-all DNS server can
> >>>>>   be set.
> >>>>> - Routing/match domains are also included in the list of search domains.
> >>>>>   This could be changed with the matchDomainsNoSearch API, but lacking
> >>>>>   more UI or config file changes to expose this option to the user, I
> >>>>>   went with the default.
> >>>>>
> >>>>> [1] https://lore.kernel.org/wireguard/20210810074232.aah5ktq5yzysaaey at SvensMacBookAir-2.local/T/
> >>>>> [2] https://github.com/slarew/wireguard-apple/commit/6ebc356d9e11ab91443e06de5e89f1af57fcdff8
> >>>>
> >>>> That seems to be a redefinition of the existing definition of split DNS.
> >>>>
> >>>> Most usually, split DNS is done at the DNS server and different zones are served to the resolver based on various criteria... Usually the origination IP of the query;  If the query comes from a client on your LAN, or a particular subnet, the contents of a particular, private, zone are returned.  If the query comes from, say the internet,  a public zone is returned.
> >>>>
> >>>> YOUR description, is how DNS works in general... Except  your patch also seems to either bypass the resolver libraries or wedge itself in front of them The system resolver libraries well tested and understood and they handle the following very nicely.
> >>>>
> >>>> There is the issue of what happens with large DNS responses.  Any DNS response over 512 bytes UDP fails and is required to be retried as a TCP query, which can handle the large response.  It's late and I'm too tired to look it up, but there IS an RFC for this.
> >>>>
> >>>> It's a little known issue, but I've seen it when working with other VPN products that ignored/didn't understand the behavior. The results weren't pretty and really embarrassing.
> >>>>
> >>>> Systemd has a bad habit of re-inventing the wheel... Badly. Eventually it get's sorted, but is that really progress? In the long haul, "move fast and break things" is a MAJOR pita for the vast majority of us.  But some like it.
> >>>>
> >>>> For some real fun, look into DHCPCD.  It faithfully implements a particular RFC.  In some networking environments using VPNs, it breaks routing horribly.... But it meets the RFC.  You'll find it in Raspbian by default, and most other Debian derived distros.  I automatically rip it out and replace it with the also available ISC DHCP client.  That one is fully compatible with Windows, OS X, iOS, and every android and smart device I could test it with.  A DHCP server configured to be compatible with DHCPCD broke all of the previously named.
> >>>>
> >>>> I'm giving this opinion away for free, so it's worth what you paid for it.
> >>>
> >>> Regardless of naming or definitions, I think the ability for a *local device* to route DNS queries to different DNS servers based on domain matching criteria is certainly useful. It?s not always possible or desirable to control and configure an upstream DNS server. Hence, this patch enables the local device to do split DNS.
> >>>
> >>> To be clear, this patch does not bypass or wedge around anything. In fact, it configures the native macOS DNS settings in the appropriate manner to effect a split DNS configuration.
> >>>
> >>> As a result of controlling the native macOS DNS resolution logic, any feature, absent or present, in the macOS DNS resolver libraries should be unaffected. This includes the large DNS response and TCP behavior. I do not expect the small/large UDP/TCP DNS features to change behavior when using a split DNS configuration as proposed in this patch.
> >>>
> >>> -Stephen
> >>
> >> Hi Stephen,
> >>
> >> A better solution to your problem would be to deploy DNSDIST:
> >>      https://dnsdist.org/
> >>
> >> I for one would hope that esoteric requests that address a solution for less than 1% of the users would be rejected with the overall goal of preventing feature creep and bloat.
> >>
> >> Andrew
> >
> > DNSDIST may allow (I have not tried) one to create a split DNS scenario, but it is an extra piece of software that would need to be discovered, installed, configured, and maintained by a user or system administrator. I do not believe it would properly integrate with the macOS DNS resolution machinery. I do not believe it is a better solution to the problem.
> >
> > As I understand it, under some circumstances, the Tailscale macOS app also implements split DNS in roughly the same manner as done in my patch. Namely, the tailscale app appears to use the Network Extension APIs to directly integrate with the macOS DNS resolution machinery. Perhaps the relevant difference is that tailscale approaches configuration differently (not based on wg-quick) than the WireGuard macOS app.
> >
> > I do not have statistics or polling on who desires this split DNS feature. I have received private and public requests to upstream the feature. Tailscale also implements split DNS; presumable customers demand it. I suspect if the feature was available to users of the WireGuard app, then it would be used with precision to great effect. Users who do not need this split DNS feature do not lose any previous functionality in the macOS WireGuard app.
> >
> > Personally, my one hesitation with this patch is that, as currently implemented, a new syntax is added to the wg-quick config file (tilde prefixed route/match domains). My patch does not address compatibility issues nor does it add documentation to wg-quick for the new syntax.
> >
> > -Stephen
> >
>

From Jason at zx2c4.com  Wed Nov  3 10:56:28 2021
From: Jason at zx2c4.com (Jason A. Donenfeld)
Date: Wed, 3 Nov 2021 11:56:28 +0100
Subject: the format of the specified network name is invalid
In-Reply-To: <CAHFBhibg_LDeUZ=96d2u=7kh2py0zC5Abp_L0S7Krd2xptZOyQ@mail.gmail.com>
References: <CAHFBhiah1NgozmGWtTgvZEfhyxDNA0EySiZao+fsK-Z9_1QLpQ@mail.gmail.com>
 <CAHFBhibg_LDeUZ=96d2u=7kh2py0zC5Abp_L0S7Krd2xptZOyQ@mail.gmail.com>
Message-ID: <CAHmME9q0GHUHPrWuN-0s3X0kCN7QN9Ur0Y9naHko7o83DMbccg@mail.gmail.com>

Could you send:
- Your complete log file that shows the problem.
- The redacted config that causes the problem.
- The last version number that worked. (Don't tell me "the last one";
I need the actual version.)

If you send these, I can likely fix the issue. Without it, I'm in the dark.

Thanks,
Jason

From alex at alexburke.ca  Wed Nov  3 11:54:42 2021
From: alex at alexburke.ca (Alex Burke)
Date: Wed, 3 Nov 2021 12:54:42 +0100
Subject: Split DNS for macOS
In-Reply-To: <E3607380-8E46-42CC-80B5-BB38F206E9BC@slarew.net>
References: <E3607380-8E46-42CC-80B5-BB38F206E9BC@slarew.net>
Message-ID: <B2AFFB41-557E-41A0-94AB-BEDE8E4F3A06@alexburke.ca>

The functionality in [mac,i,iPad]OS that enables zone-based split DNS is quite powerful, and Stephen's contribution would fit my use case as well.

I use a specific DNS resolver of my choice, but having Wireguard able to automagically resolve "host.corp.internal"-style FQDNs when the relevant Wireguard connection is up would be a huge win, not only for me but for corporations with split-horizon DNS which want to implement Wireguard without having to route irrelevant traffic into that tunnel.


> On 29 October 2021 at 22:07, Stephen Larew <stephen at slarew.net> wrote:
> 
> ?
>> 
>>> On Oct 29, 2021, at 10:03, Andrew Fried <afried at spamteq.com> wrote:
>>> 
>>>> On Oct 29, 2021, at 08:33, Stephen Larew <stephen at slarew.net> wrote:
>>> 
>>>> On Oct 28, 2021, at 02:58, Bruce Ferrell <bferrell at baywinds.org> wrote:
>>>> 
>>>> On 10/28/21 12:16 AM, Stephen Larew wrote:
>>>>> For many months now, I have been running a patched WireGuard macOS app
>>>>> that enables a split DNS configuration. I would like to try to upstream
>>>>> my patches for split DNS.
>>>>> 
>>>>> There has been some interest in this patch:
>>>>> - "Mac APP DNS Search Domain" thread from July and August 2021 [1]
>>>>> - A commenter on my GitHub fork of wireguard-apple.
>>>>> 
>>>>> What is split DNS? It allows sending DNS queries to a specific server
>>>>> based on the domain name. Systemd-resolved calls it a routing domain.
>>>>> Apple's Network Extension framework calls it a match domain.  Split DNS
>>>>> is especially useful for internal DNS servers.
>>>>> 
>>>>> For example, if corp.example.com is a routing domain for the DNS server
>>>>> at 192.0.2.1 (only accessible over WireGuard), then
>>>>> server.corp.example.com is resolved using 192.0.2.1 while
>>>>> www.example.com is resolved using some other DNS resolver (depending on
>>>>> the other network settings in macOS).
>>>>> 
>>>>> The proposed patch adds new syntax to the wg-quick DNS= line.
>>>>> Specifically, a tilde prefixed domain is treated as a routing domain.
>>>>> Multiple routing domains can be added.
>>>>> 
>>>>> Limitations:
>>>>> - Needs modifications to iOS UI to work on iOS.
>>>>> - Only matching routing domains are sent to the DNS servers specified in
>>>>> the DNS= config line.  No separate fallback catch-all DNS server can
>>>>> be set.
>>>>> - Routing/match domains are also included in the list of search domains.
>>>>> This could be changed with the matchDomainsNoSearch API, but lacking
>>>>> more UI or config file changes to expose this option to the user, I
>>>>> went with the default.
>>>>> 
>>>>> [1] https://lore.kernel.org/wireguard/20210810074232.aah5ktq5yzysaaey at SvensMacBookAir-2.local/T/
>>>>> [2] https://github.com/slarew/wireguard-apple/commit/6ebc356d9e11ab91443e06de5e89f1af57fcdff8
>>>> 
>>>> That seems to be a redefinition of the existing definition of split DNS.
>>>> 
>>>> Most usually, split DNS is done at the DNS server and different zones are served to the resolver based on various criteria... Usually the origination IP of the query;  If the query comes from a client on your LAN, or a particular subnet, the contents of a particular, private, zone are returned.  If the query comes from, say the internet,  a public zone is returned.
>>>> 
>>>> YOUR description, is how DNS works in general... Except  your patch also seems to either bypass the resolver libraries or wedge itself in front of them The system resolver libraries well tested and understood and they handle the following very nicely.
>>>> 
>>>> There is the issue of what happens with large DNS responses.  Any DNS response over 512 bytes UDP fails and is required to be retried as a TCP query, which can handle the large response.  It's late and I'm too tired to look it up, but there IS an RFC for this.
>>>> 
>>>> It's a little known issue, but I've seen it when working with other VPN products that ignored/didn't understand the behavior. The results weren't pretty and really embarrassing.
>>>> 
>>>> Systemd has a bad habit of re-inventing the wheel... Badly. Eventually it get's sorted, but is that really progress? In the long haul, "move fast and break things" is a MAJOR pita for the vast majority of us.  But some like it.
>>>> 
>>>> For some real fun, look into DHCPCD.  It faithfully implements a particular RFC.  In some networking environments using VPNs, it breaks routing horribly.... But it meets the RFC.  You'll find it in Raspbian by default, and most other Debian derived distros.  I automatically rip it out and replace it with the also available ISC DHCP client.  That one is fully compatible with Windows, OS X, iOS, and every android and smart device I could test it with.  A DHCP server configured to be compatible with DHCPCD broke all of the previously named.
>>>> 
>>>> I'm giving this opinion away for free, so it's worth what you paid for it.
>>> 
>>> Regardless of naming or definitions, I think the ability for a *local device* to route DNS queries to different DNS servers based on domain matching criteria is certainly useful. It?s not always possible or desirable to control and configure an upstream DNS server. Hence, this patch enables the local device to do split DNS.
>>> 
>>> To be clear, this patch does not bypass or wedge around anything. In fact, it configures the native macOS DNS settings in the appropriate manner to effect a split DNS configuration.
>>> 
>>> As a result of controlling the native macOS DNS resolution logic, any feature, absent or present, in the macOS DNS resolver libraries should be unaffected. This includes the large DNS response and TCP behavior. I do not expect the small/large UDP/TCP DNS features to change behavior when using a split DNS configuration as proposed in this patch.
>>> 
>>> -Stephen
>> 
>> Hi Stephen,
>> 
>> A better solution to your problem would be to deploy DNSDIST:
>>    https://dnsdist.org/
>> 
>> I for one would hope that esoteric requests that address a solution for less than 1% of the users would be rejected with the overall goal of preventing feature creep and bloat.
>> 
>> Andrew
> 
> DNSDIST may allow (I have not tried) one to create a split DNS scenario, but it is an extra piece of software that would need to be discovered, installed, configured, and maintained by a user or system administrator. I do not believe it would properly integrate with the macOS DNS resolution machinery. I do not believe it is a better solution to the problem.
> 
> As I understand it, under some circumstances, the Tailscale macOS app also implements split DNS in roughly the same manner as done in my patch. Namely, the tailscale app appears to use the Network Extension APIs to directly integrate with the macOS DNS resolution machinery. Perhaps the relevant difference is that tailscale approaches configuration differently (not based on wg-quick) than the WireGuard macOS app.
> 
> I do not have statistics or polling on who desires this split DNS feature. I have received private and public requests to upstream the feature. Tailscale also implements split DNS; presumable customers demand it. I suspect if the feature was available to users of the WireGuard app, then it would be used with precision to great effect. Users who do not need this split DNS feature do not lose any previous functionality in the macOS WireGuard app.
> 
> Personally, my one hesitation with this patch is that, as currently implemented, a new syntax is added to the wg-quick config file (tilde prefixed route/match domains). My patch does not address compatibility issues nor does it add documentation to wg-quick for the new syntax.
> 
> -Stephen


From wireguard-mail at chil.at  Wed Nov  3 13:45:58 2021
From: wireguard-mail at chil.at (Christoph Loesch)
Date: Wed, 3 Nov 2021 14:45:58 +0100
Subject: WireGuard connection without interface-address / linknet
Message-ID: <1da52141-2cca-9ebb-e415-af471b14e74e@chil.at>

Hi,

(mail resent without URLs, because it got filtered by moderation)

I am using WireGuard on an OpenWRT VM as server for clients basically to reach the server's internal LAN at 10.5.44.0/24

As clients I currently use different WireGuard implementations like:
- Ubiquiti EdgeRouter (EdgeOS v2 based on Debian/stretch) with package from github/WireGuard/wireguard-vyatta-ubnt
- Mikrotik RouterOS v7.1 with Mikrotik own (at the moment beta) implementation
- Windows 10 Client from wireguard website

Server config looks like this:

config interface 'wg0'
 ??????? option proto 'wireguard'
 ??????? option private_key 'cNT...8Hc='
 ??????? option listen_port '51820'
 ??????? list addresses '172.27.0.1/16'
config wireguard_wg0
 ??????? option description 'router-test'
 ??????? option public_key 'qT5...YGo='
 ??????? option preshared_key 'Dle...ozI='
 ??????? option persistent_keepalive '25'
 ??????? option route_allowed_ips '1'
 ??????? list allowed_ips '172.27.34.28/32'
 ??????? list allowed_ips '10.34.28.0/24'

Client config looks like this:

[Interface]
PrivateKey = mDk...uVs=
Address = 172.27.34.28
[Peer]
PublicKey = 1sy...IkU=
PresharedKey = Dle...ozI=
AllowedIPs = 172.27.0.1/32,10.5.44.0/24
Endpoint = server.mydomain.at:51820
PersistentKeepalive = 25

Clients should just be able to reach the server's 10.5.44.0/24 subnet and this subnet should be able to reach clients at (in this one example) 10.34.28.0/24
Now this works all well as expected but I would like to omit using the 172.27.* addresses/linknet if possible because I dont really need/use this as it is only defined for the WireGuard tunnel itself.

On the EdgeRouter this also works perfectly fine if I remove the 172.27.* address on both sides. It is still possible to reach the other end repestively.
On the Mikrotik device and on the Windows client (using exact same configuration) it does not work as soon as I remove just the 172.27. address/linknet from configuration.
(I didn't test other clients yet)

I guess on the EdgeRouter this works because I set: set interfaces wireguard wg0 **route-allowed-ips true** - so the corresponding routes are added.
The Mikrotik device and the Windows client do not offer such an option, so those routes have to be added manually I guess?
But why does that work "out of the box" as soon as I add any linknet (172.27.* in my example) for the tunnel itself to the configuration?

Is this a fault in the implementation and I should file a bug report or is that expected that way?

(it's not a big issue using the linknet, I am just curious and would like avoid using it if it is not neccessarily required)

Thanks for any thoughts and kind regards,
Christoph

From wireguard-mail at chil.at  Wed Nov  3 16:03:25 2021
From: wireguard-mail at chil.at (Christoph Loesch)
Date: Wed, 3 Nov 2021 17:03:25 +0100
Subject: WireGuard connection without interface-address / linknet
In-Reply-To: <1da52141-2cca-9ebb-e415-af471b14e74e@chil.at>
References: <1da52141-2cca-9ebb-e415-af471b14e74e@chil.at>
Message-ID: <ddc62364-6764-3662-5d76-f720a4a375ff@chil.at>

Hi,

regarding the Windows Client I got a good hint in the chat from user another| who told me to look at the routing table.

There I figured out the differences:
- if an address is configured, the Windows Client sets the correct routes from defined AllowedIPs to reach the remote subnets.
- if *no address* is configured in [Interface] then not a single route is set from AllowedIPs but it sets routes for APIPA 169.254.* subnet.

Version information:
App version: 0.5.1
Driver version: 0.10.1
Go version: 1.17.2
Operating System: Windows 10.0.19043
Architecture: amd64

Kind regards,
Christoph

Am 03.11.2021 um 14:45 schrieb Christoph Loesch:
> Hi,
>
> (mail resent without URLs, because it got filtered by moderation)
>
> I am using WireGuard on an OpenWRT VM as server for clients basically to reach the server's internal LAN at 10.5.44.0/24
>
> As clients I currently use different WireGuard implementations like:
> - Ubiquiti EdgeRouter (EdgeOS v2 based on Debian/stretch) with package from github/WireGuard/wireguard-vyatta-ubnt
> - Mikrotik RouterOS v7.1 with Mikrotik own (at the moment beta) implementation
> - Windows 10 Client from wireguard website
>
> Server config looks like this:
>
> config interface 'wg0'
> ??????? option proto 'wireguard'
> ??????? option private_key 'cNT...8Hc='
> ??????? option listen_port '51820'
> ??????? list addresses '172.27.0.1/16'
> config wireguard_wg0
> ??????? option description 'router-test'
> ??????? option public_key 'qT5...YGo='
> ??????? option preshared_key 'Dle...ozI='
> ??????? option persistent_keepalive '25'
> ??????? option route_allowed_ips '1'
> ??????? list allowed_ips '172.27.34.28/32'
> ??????? list allowed_ips '10.34.28.0/24'
>
> Client config looks like this:
>
> [Interface]
> PrivateKey = mDk...uVs=
> Address = 172.27.34.28
> [Peer]
> PublicKey = 1sy...IkU=
> PresharedKey = Dle...ozI=
> AllowedIPs = 172.27.0.1/32,10.5.44.0/24
> Endpoint = server.mydomain.at:51820
> PersistentKeepalive = 25
>
> Clients should just be able to reach the server's 10.5.44.0/24 subnet and this subnet should be able to reach clients at (in this one example) 10.34.28.0/24
> Now this works all well as expected but I would like to omit using the 172.27.* addresses/linknet if possible because I dont really need/use this as it is only defined for the WireGuard tunnel itself.
>
> On the EdgeRouter this also works perfectly fine if I remove the 172.27.* address on both sides. It is still possible to reach the other end repestively.
> On the Mikrotik device and on the Windows client (using exact same configuration) it does not work as soon as I remove just the 172.27. address/linknet from configuration.
> (I didn't test other clients yet)
>
> I guess on the EdgeRouter this works because I set: set interfaces wireguard wg0 **route-allowed-ips true** - so the corresponding routes are added.
> The Mikrotik device and the Windows client do not offer such an option, so those routes have to be added manually I guess?
> But why does that work "out of the box" as soon as I add any linknet (172.27.* in my example) for the tunnel itself to the configuration?
>
> Is this a fault in the implementation and I should file a bug report or is that expected that way?
>
> (it's not a big issue using the linknet, I am just curious and would like avoid using it if it is not neccessarily required)
>
> Thanks for any thoughts and kind regards,
> Christoph

From Jason at zx2c4.com  Wed Nov  3 16:18:28 2021
From: Jason at zx2c4.com (Jason A. Donenfeld)
Date: Wed, 3 Nov 2021 17:18:28 +0100
Subject: WireGuard connection without interface-address / linknet
In-Reply-To: <ddc62364-6764-3662-5d76-f720a4a375ff@chil.at>
References: <1da52141-2cca-9ebb-e415-af471b14e74e@chil.at>
 <ddc62364-6764-3662-5d76-f720a4a375ff@chil.at>
Message-ID: <CAHmME9pDZ30wAYYgEe3TNp8pwiEW4TF0Lj=oxsaEsvvoQNGQwA@mail.gmail.com>

Fixed, thanks.

https://git.zx2c4.com/wireguard-windows/commit/?id=46da2d426aa79725abafdfb615f623ef1d2d9aa2

From wireguard-mail at chil.at  Wed Nov  3 16:39:12 2021
From: wireguard-mail at chil.at (Christoph Loesch)
Date: Wed, 3 Nov 2021 17:39:12 +0100
Subject: WireGuard connection without interface-address / linknet
In-Reply-To: <CAHmME9pDZ30wAYYgEe3TNp8pwiEW4TF0Lj=oxsaEsvvoQNGQwA@mail.gmail.com>
References: <1da52141-2cca-9ebb-e415-af471b14e74e@chil.at>
 <ddc62364-6764-3662-5d76-f720a4a375ff@chil.at>
 <CAHmME9pDZ30wAYYgEe3TNp8pwiEW4TF0Lj=oxsaEsvvoQNGQwA@mail.gmail.com>
Message-ID: <f2766c96-151d-ae9b-8e35-5e57f0c663d1@chil.at>

Thank you for the fix!

Am 03.11.2021 um 17:18 schrieb Jason A. Donenfeld:
> Fixed, thanks.
>
> https://git.zx2c4.com/wireguard-windows/commit/?id=46da2d426aa79725abafdfb615f623ef1d2d9aa2

From kevans at freebsd.org  Wed Nov  3 16:39:19 2021
From: kevans at freebsd.org (Kyle Evans)
Date: Wed, 3 Nov 2021 11:39:19 -0500
Subject: Wireguard on FreeBSD - a few questions
In-Reply-To: <cfddc41d-0768-af1a-d527-d2bccd5cfd56@deze.org>
References: <cfddc41d-0768-af1a-d527-d2bccd5cfd56@deze.org>
Message-ID: <CACNAnaE8DrntMR0RifxNzijLWfBXNHxm2poSY-DewOQszP-XqQ@mail.gmail.com>

On Wed, Nov 3, 2021 at 5:55 AM Frank Volf <frank at deze.org> wrote:
>
>
> Hi,
>
> This weekend I installed Wireguard on FreeBSD 13.0 and until now
> everything seems to work fine (I use the kernel module).
> Installation and configuration was easy and connecting with the Android
> app works great as well.
>

Excellent, that's good to hear! :-)

> I do have a few questions.
>
> 1) Is it possible on FreeBSD to enable some kind of logging? I did made
> a small configuration error with my first client and it was hard to find
> the error, because there does not seem to be any logging at all.  Some
> logging information would be appreciated and probably wold have pointed
> me faster to the fact that I needed to switch two keys in my config.
>

If you set 'debug' on the interface (`ifconfig wg0 debug`) then it'll
write some useful bits to syslog for your perusal.

> 2) I noticed that Wireguard uses a wildcard to listen to all IP
> addresses on my multi-homed machine on his dedicated UDP port. I would
> prefer if Wireguard would only bind to the specific IP address on the
> outside interface that is designated for that use. Is this possible?
>
> 3) Final question: is it possible on the server side to restrict the
> destinations that clients can connect to it? I know, that I can set the
> AllowedIPs on the client side to restrict that, but that setting can be
> changed at the client side. It would be nice if I could restrict
> destinations at the server side (so client X can only connect to an IP
> address of an internal server that it needs access to but nothing else).
> I can probably use a state full packet filtering firewall for this, but
> it would it be possible to configure this on the Wireguard server side
> as well?
>

For these last two, I'll defer to somebody else -- I'm not aware of
any such functionality on other platforms, but wireguard-freebsd will
follow suit if this is or will become an accepted concept elsewhere.

> That said, I'm pleased with the first test results of Wireguard on
> FreeBSD and hopefully it keeps on running fine. Great product!
>

Great, thanks for testing! =)

Kyle Evans

From rlifshay at gmail.com  Wed Nov  3 18:07:47 2021
From: rlifshay at gmail.com (Reuben Lifshay)
Date: Wed, 3 Nov 2021 11:07:47 -0700
Subject: Wireguard Android app stops working on Chrome OS after suspend
In-Reply-To: <CABip9K7f3AKBG9WuJ5_u0n7ExCXKjvdGiy-6YEe8TLzTu0Ezqg@mail.gmail.com>
References: <CABip9K7f3AKBG9WuJ5_u0n7ExCXKjvdGiy-6YEe8TLzTu0Ezqg@mail.gmail.com>
Message-ID: <CABip9K7zAJKQR__Hj-X9-NxrgG-fw4mxW2dPn1K-MMeAMnSHXA@mail.gmail.com>

I've had the Wireguard Android app on my chromebook for awhile and
it's seemed to work just fine. Recently I've started using it more
though, and I discovered that if I closed the lid of my chromebook for
a few minutes or if I let it go to sleep that I can no longer get any
of my tunnels to work. Toggling the tunnels and/or wifi does not fix
it, nor does force stopping the wireguard app. I have to do a full
reboot to get things working correctly again.

I did some debugging and discovered that the wireguard packets were
still being sent to and received by the remote server, but there was
something incorrect about them because they never appeared on the
corresponding wireguard interface on the server. I didn't see anything
that stood out in the app debug log either. After some searching I
found a reddit post where someone discovered that if they downgraded
to version 1.0.20210506 that the issue went away. I didn't try that
myself yet because I don't really want to enable dev mode to allow
sideloading apps right now.

I can enable debug mode for wireguard on one of my servers to get more
info if it would be helpful, but have yet to do so.

Here's a link to the reddit post:
https://www.reddit.com/r/chromeos/comments/q0lbnl/network_troubles_after_version_93_update/

Reuben

From frank at deze.org  Wed Nov  3 20:52:40 2021
From: frank at deze.org (Frank Volf)
Date: Wed, 3 Nov 2021 21:52:40 +0100
Subject: Wireguard on FreeBSD - a few questions
In-Reply-To: <CACNAnaE8DrntMR0RifxNzijLWfBXNHxm2poSY-DewOQszP-XqQ@mail.gmail.com>
References: <cfddc41d-0768-af1a-d527-d2bccd5cfd56@deze.org>
 <CACNAnaE8DrntMR0RifxNzijLWfBXNHxm2poSY-DewOQszP-XqQ@mail.gmail.com>
Message-ID: <3b364170-7a7e-d9aa-d365-fe0ce3390acd@deze.org>


Hi Kyle,
>> 1) Is it possible on FreeBSD to enable some kind of logging? I did made
>> a small configuration error with my first client and it was hard to find
>> the error, because there does not seem to be any logging at all.  Some
>> logging information would be appreciated and probably wold have pointed
>> me faster to the fact that I needed to switch two keys in my config.
>>
> If you set 'debug' on the interface (`ifconfig wg0 debug`) then it'll
> write some useful bits to syslog for your perusal.

O.k. good to know this. It would be even better if this was documented, 
I think a if_wg manual page for FreeBSD would be appropriate.

>> 2) I noticed that Wireguard uses a wildcard to listen to all IP
>> addresses on my multi-homed machine on his dedicated UDP port. I would
>> prefer if Wireguard would only bind to the specific IP address on the
>> outside interface that is designated for that use. Is this possible?

I think it is useful if you could bind Wireguard? to use/listen on a 
specific IP address, instead of the wildcard.
For example, for my tests I used a secondary (alias) IP address on a 
server as the entry point for Wireguard tunnels.
However, if the server starts a session to the client (or tries to check 
if the client is still alive), it uses the primary interface address 
instead.
Binding it to a specific IP address would solve this.

Kind regards,

Frank


From afried at spamteq.com  Wed Nov  3 21:34:27 2021
From: afried at spamteq.com (Andrew Fried)
Date: Wed, 3 Nov 2021 21:34:27 +0000
Subject: Split DNS for macOS
In-Reply-To: <CAHsVW7N2JA25KXDRM0ghNrn-9BZpuqY0OMHX4WTi7hmXjS9nGA@mail.gmail.com>
References: <CAHsVW7N2JA25KXDRM0ghNrn-9BZpuqY0OMHX4WTi7hmXjS9nGA@mail.gmail.com>
Message-ID: <K17663-81.1635975267@spamhaus.org>

Hi Matty,

I understand exactly what you're trying to accomplish and agree that 
split dns can be challenging, especially with multiple VPN gateways.

My point is that what you're describing is a DNS issue, not a 
firewall/vpn/routing issue.  As such, I think there's more eloquent way 
to solve DNS related issues.

The old fashioned way is to add exceptions to the equivalent of the 
/etc/host file.  Not ideal, doesn't scale well and pretty static, but if 
you're relying on just a few private host mappings it works pretty well.

The second and more palatable solution is to have the internal 
nameservers running software that supports views - such that queries for 
xxxx.example.com that originate from private address space return 
different answers than if the query originated from public space.

A third option would be set the internal recursives up as forwarders 
that only respond authoritatively for your private "mydomain.internet" 
and forward all other requests to nameservers capable of public recursion.

There's the dnsdist solution, which is an advanced dns proxy server 
capable of routing requests to different recursives based on the domain 
name.  DNSDIST does a lot of other stuff as well, but the heart of is 
intelligent proxying.  In our racks we use DNSDIST to distribute around 
a million DNS queries per minute and it works flawlessly.

Basically, what I'm suggesting is that DNS servers handle DNS and 
wireguard handle routing/transport.  Adding VPN functionality to a 
nameserver or dns capabilities to Wireguard adds complexities that can 
be better handled elsewhere.

What makes Wireguard so good is that it does one thing and does it 
really, really well.

Andrew


On 10/29/21 5:06 PM, Matty Driessen wrote:
> Hello Andrew,
> 
> I just want to chime in here and say that I think the current
> implementation of search domains is simply not working the way it
> should on the MacOS client.
> 
> My use case is pretty common, an internal DNS server that has entries
> for internal servers. I defined a search domain in the WireGuard
> configuration; DNS = 10.13.13.1 mydomain.internal. The search domain
> is for convenience, so I can just use the servername instead of
> servername.mydomain.internal. Now this works fine when I route all the
> traffic through the VPN (AllowedIPs = 0.0.0.0/0) but the search domain
> is completely ignored when I only route the traffic I need to
> (AllowedIPs = 10.13.13.0/24 192.168.0.0/24).
> 
> I don't think this is a configuration error on my side. The DNS
> responds fine when using servername.mydomain.internal. This problem is
> even mentioned in the "WireGuard macOS & iOS TODO List"
> 9. matchDomains=[??] doesn?t do what the documentation says.
> Specifically, DNS servers are not used if allowed IPs isn?t 0.0.0.0/0.
> 
> The description isn't 100% accurate (or outdated), the DNS server is
> used but the search domain isn't being set on the primary resolver.
> Some have solved this issue by adding the search domains to the list
> of matchDomains; dnsSettings.matchDomains = [""] +
> dnsSettings.searchDomains. But that way the DNS server specified in
> WireGuard is still the primary resolver for all DNS queries.
> 
> Here is a link on how OpenVPN handles this and I think it's how it
> should work when not using AllowedIPs 0.0.0.0/0.
> https://openvpn.net/faq/how-does-ios-interpret-pushed-dns-servers-and-search-domains/
> On a split-tunnel, where redirect-gateway is not pushed by the server,
> and at least one pushed DNS server is present:
> - route all DNS requests through pushed DNS server(s) if no added
> search domains.
> - route DNS requests for added search domains only, if at least one
> added search domain.
> 
> Yours sincerely,
> Matty
> 
> --
> 
> Hi Stephen,
> 
> A better solution to your problem would be to deploy DNSDIST:
>          https://dnsdist.org/
> 
> I for one would hope that esoteric requests that address a solution
> for less than 1% of the users would be rejected with the overall goal
> of preventing feature creep and bloat.
> 
> Andrew
> 

-- 
Andrew Fried
afried at spamteq.com
+1.703.667.4050 Office
+1.703.362.0067 Mobile

From alex at alexburke.ca  Wed Nov  3 21:46:11 2021
From: alex at alexburke.ca (Alex Burke)
Date: Wed, 3 Nov 2021 22:46:11 +0100
Subject: Split DNS for macOS
In-Reply-To: <K17663-81.1635975267@spamhaus.org>
References: <K17663-81.1635975267@spamhaus.org>
Message-ID: <0EAFBE4F-AC66-468E-B731-D6A7A4A5DBB0@alexburke.ca>

Hi Andrew,

> What makes Wireguard so good is that it does one thing and does it really, really well.


I am in complete agreement on this, but Wireguard wouldn't really be "doing" (implementing) what Stephen and Matty are trying to accomplish, but rather simply integrating with the mechanism the Apple platform has implemented for doing precisely that.

While I'm not in a position to weigh in on how much development work it would entail for Jason et al to implement to the high standard Wireguard is known for, or how much bloat it would add, it's undoubtedly a killer feature, and it's my professional opinion that a significant proportion of the Wireguard user base would benefit from supporting it ? especially for corporate use.


> On 3 Nov 2021 at 21:34, Andrew Fried <afried at spamteq.com> wrote:
> ?Hi Matty,
> 
> I understand exactly what you're trying to accomplish and agree that split dns can be challenging, especially with multiple VPN gateways.
> 
> My point is that what you're describing is a DNS issue, not a firewall/vpn/routing issue.  As such, I think there's more eloquent way to solve DNS related issues.
> 
> The old fashioned way is to add exceptions to the equivalent of the /etc/host file.  Not ideal, doesn't scale well and pretty static, but if you're relying on just a few private host mappings it works pretty well.
> 
> The second and more palatable solution is to have the internal nameservers running software that supports views - such that queries for xxxx.example.com that originate from private address space return different answers than if the query originated from public space.
> 
> A third option would be set the internal recursives up as forwarders that only respond authoritatively for your private "mydomain.internet" and forward all other requests to nameservers capable of public recursion.
> 
> There's the dnsdist solution, which is an advanced dns proxy server capable of routing requests to different recursives based on the domain name.  DNSDIST does a lot of other stuff as well, but the heart of is intelligent proxying.  In our racks we use DNSDIST to distribute around a million DNS queries per minute and it works flawlessly.
> 
> Basically, what I'm suggesting is that DNS servers handle DNS and wireguard handle routing/transport.  Adding VPN functionality to a nameserver or dns capabilities to Wireguard adds complexities that can be better handled elsewhere.
> 
> What makes Wireguard so good is that it does one thing and does it really, really well.
> 
> Andrew
> 
> 
> On 10/29/21 5:06 PM, Matty Driessen wrote:
>> Hello Andrew,
>> I just want to chime in here and say that I think the current
>> implementation of search domains is simply not working the way it
>> should on the MacOS client.
>> My use case is pretty common, an internal DNS server that has entries
>> for internal servers. I defined a search domain in the WireGuard
>> configuration; DNS = 10.13.13.1 mydomain.internal. The search domain
>> is for convenience, so I can just use the servername instead of
>> servername.mydomain.internal. Now this works fine when I route all the
>> traffic through the VPN (AllowedIPs = 0.0.0.0/0) but the search domain
>> is completely ignored when I only route the traffic I need to
>> (AllowedIPs = 10.13.13.0/24 192.168.0.0/24).
>> I don't think this is a configuration error on my side. The DNS
>> responds fine when using servername.mydomain.internal. This problem is
>> even mentioned in the "WireGuard macOS & iOS TODO List"
>> 9. matchDomains=[??] doesn?t do what the documentation says.
>> Specifically, DNS servers are not used if allowed IPs isn?t 0.0.0.0/0.
>> The description isn't 100% accurate (or outdated), the DNS server is
>> used but the search domain isn't being set on the primary resolver.
>> Some have solved this issue by adding the search domains to the list
>> of matchDomains; dnsSettings.matchDomains = [""] +
>> dnsSettings.searchDomains. But that way the DNS server specified in
>> WireGuard is still the primary resolver for all DNS queries.
>> Here is a link on how OpenVPN handles this and I think it's how it
>> should work when not using AllowedIPs 0.0.0.0/0.
>> https://openvpn.net/faq/how-does-ios-interpret-pushed-dns-servers-and-search-domains/
>> On a split-tunnel, where redirect-gateway is not pushed by the server,
>> and at least one pushed DNS server is present:
>> - route all DNS requests through pushed DNS server(s) if no added
>> search domains.
>> - route DNS requests for added search domains only, if at least one
>> added search domain.
>> Yours sincerely,
>> Matty
>> --
>> Hi Stephen,
>> A better solution to your problem would be to deploy DNSDIST:
>>         https://dnsdist.org/
>> I for one would hope that esoteric requests that address a solution
>> for less than 1% of the users would be rejected with the overall goal
>> of preventing feature creep and bloat.
>> Andrew
> 
> -- 
> Andrew Fried
> afried at spamteq.com
> +1.703.667.4050 Office
> +1.703.362.0067 Mobile


From igorbzin at gmail.com  Thu Nov  4 11:14:15 2021
From: igorbzin at gmail.com (Igor Bozin)
Date: Thu, 4 Nov 2021 12:14:15 +0100
Subject: Support for M1 MacBook Simulators
In-Reply-To: <re37dSyY_rXzrSL5DCMtppy1I37FlAT-pqCZjPdPpw8CHDxX3Nn3xeVECxTBMRFJBk-XEINrSbXY4KvWEtfii8GbwxZPNg7CZgCW-3asMus=@protonmail.ch>
References: <18206D0A-EA19-4230-AAEC-8D52BE66A1C7@gmail.com>
 <re37dSyY_rXzrSL5DCMtppy1I37FlAT-pqCZjPdPpw8CHDxX3Nn3xeVECxTBMRFJBk-XEINrSbXY4KvWEtfii8GbwxZPNg7CZgCW-3asMus=@protonmail.ch>
Message-ID: <5F19BC44-58DD-4391-AB17-239B4F980816@gmail.com>

Hello David and Neutron,

my setup is following: I have added the WG library as a SPM dependency to my project and followed all steps as described in the Github readme from wireguard:

https://github.com/WireGuard/wireguard-apple

What I am trying to do is to build a framework which has some other functionality on top and wrap WG within that framework. I can build just fine for iOS arm64 and iPhone simulator x86_64 architectures.

When I try to do the steps you described in the linked message, to build a framework for arm64 iPhone simulator, I still get the following error when I try to build the project with an arm64 simulator target:

/Users/igorbozin/Projects/cyan-internetprotectionsdk-ios/InternetProtectionApp.xcodeproj Building for iOS Simulator, but the linked library 'libwg-go.a' was built for iOS.


Is there a way to modify the makefile, so I can build a version of libwg-go.a for arm64 iPhone simulators?

Best regards,

Igor

> On 29.10.2021, at 17:24, Neutron <dotneutron at protonmail.ch> wrote:
> 
> Hey Igor,
> 
> I believe I encountered this in the past.
> https://lists.zx2c4.com/pipermail/wireguard/2021-September/007001.html
> 
> I'm not sure what your exact setup is since you mentioned it's a separate project presumably depending on WireGuardKit, but I can build libwg-go for arm64 just fine using the trick I mentioned in the linked thread. Just add this flag to Sources/WireGuardKitGo/Makefile.
> GOOS_iphonesimulator := ios
> 
> Hope it helps.
> 


From Jason at zx2c4.com  Fri Nov  5 14:42:26 2021
From: Jason at zx2c4.com (Jason A. Donenfeld)
Date: Fri, 05 Nov 2021 15:42:26 +0100
Subject: [ANNOUNCE] wireguard-freebsd snapshot v0.0.20211105 is available
Message-ID: <20211105144230.6137A6120A@mail.kernel.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

Hi,

An experimental snapshot, v0.0.20211105, of WireGuard for FreeBSD has been
been tagged in the git repository.

At this time this code is new, unvetted, possibly buggy, and should be
considered "experimental". It might contain security issues. We gladly
welcome your testing and bug reports, but do keep in mind that this code
is new, so some caution should be exercised at the moment for using it
in mission critical environments.

== Changes ==

  * compat: taskqueue draining was backported to stable/13
  
  Compilation should resume working with the latest stable/13.
  
  * if_wg: bump keepalive timers unconditionally on send
  
  Important fix to the timer state machine, which matches behavior on Linux and
  elsewhere.
  
  * if_wg: protect in6_mask2len with INET6
  
  Fix compilation on 12 with !INET6.

This snapshot contains commits from: Jason A. Donenfeld.

The source repository is available at the usual location:
  git clone https://git.zx2c4.com/wireguard-freebsd

This snapshot is available in compressed tarball form:
  https://git.zx2c4.com/wireguard-freebsd/snapshot/wireguard-freebsd-0.0.20211105.tar.xz
  SHA2-256: 454f2cf15329ccf40da15f8df61f8d70bb6d382efdfe4a5899ca16474f91903f

Thank you,
Jason Donenfeld


-----BEGIN PGP SIGNATURE-----

iQJEBAEBCAAuFiEEq5lC5tSkz8NBJiCnSfxwEqXeA64FAmGFQrYQHGphc29uQHp4
MmM0LmNvbQAKCRBJ/HASpd4DrucXD/9cnVv050l3zzSKriaIBnrZfVgUVfKtZWmg
KVmc/tKViFph1Mnx+n8EAHAf05n93v7J9b65l7syb7OUnv8IseoG16Iw38erbEsu
kkaF1BK9lJ4pLjBKtAcTnzwipm8aXf6Ge+HPwzRO30mOL11DnD1gbLqrFwIKQHUC
HI7K0Bv+6t54t17vnaVvO51JJME4NpRDQ0zqIMROROGIRYJLbnivZRHcIxN4lQuJ
IlY8Bzj3dyzYKJqdB6mvAm2XShGx4265Qo6uo31Qyuo8SJVXFnb5WEnc30Aw1tHc
Fi3cqWnia84TW/X+PyH3wHhRjC4JJNms7qpIvE4GN5uil6Z2sqfOF7WxkVAhiSX9
RltirnHuw8KELsomYmje2wDcDE//Yxucj9cV0/qOqCzhvgiuggNlIIRdQHkbVhsm
RAAh6N9fcFWxWQTfXbWqToFVOImzPBS7r1ewYZbT8VFvp41CtSArio+yc5feVbei
s78KMWPhJWX5s9cxXoyeJAL9YJAaeUvGvB1d0590+N8iC/jQwJrvViCd5fqlzXLe
7x6Ge02ekkYsWc+MubfEm1zFYGMrTxmCRfElIZz48BweuzMtOQRzRlGQQR88NyUx
vt71YDb8xSC1/zIeQdlfOgvqvtcPDC9ZkneHxWOV39Iwc7W9KfopXOTUVgfC4NEs
oDs2uLmsSQ==
=Q4TN
-----END PGP SIGNATURE-----

From dave at natulte.net  Sat Nov  6 04:54:43 2021
From: dave at natulte.net (David Anderson)
Date: Fri, 05 Nov 2021 21:54:43 -0700
Subject: Split DNS for macOS
In-Reply-To: <E3607380-8E46-42CC-80B5-BB38F206E9BC@slarew.net>
References: <20211028071638.88001-1-stephen@slarew.net>
 <0e80daf1-b4c2-b177-28ca-79e6575a339c@baywinds.org>
 <ED3FB4B3-96AD-4B98-86EC-D9F7999A530F@slarew.net>
 <K25247-14.1635527002@spamhaus.org>
 <E3607380-8E46-42CC-80B5-BB38F206E9BC@slarew.net>
Message-ID: <c700d141-5348-4ce5-9398-dfb281383607@www.fastmail.com>

On Fri, Oct 29, 2021, at 14:07, Stephen Larew wrote:
> As I understand it, under some circumstances, the Tailscale macOS app also implements split DNS in roughly the same manner as done in my patch. Namely, the tailscale app appears to use the Network Extension APIs to directly integrate with the macOS DNS resolution machinery. Perhaps the relevant difference is that tailscale approaches configuration differently (not based on wg-quick) than the WireGuard macOS app.

Hi, Tailscale person here.

Yes, the Tailscale macOS app supports configuring either split DNS. It's situationally popular for things like "direct anything under amazonaws.com to the AWS VPC internal resolver on the other side of my tunnel", which makes your AWS VMs and such magically resolve correctly without shoveling all your unrelated requests through AWS.

The semantics of NEDNSSettings I've worked out so far:
  * `searchDomains` sets interface-scoped search domains only, which appear to not be used at all for single-label DNS query expansion. IOW, it seems to do mostly nothing - or at least, I've not discovered anything it affects.
  * `matchDomains` specifies what DNS suffixes should be sent to your resolvers (specified in `servers`). Specifying a list of suffixes here implements split-DNS. Specifying "" (the empty suffix, which matches all names) captures all queries.
  * `matchDomains` _also_ installs any non-empty string you specify in the global search path.
  * `matchDomainsNoSearch` lets you make `matchDomains` be just match domains, without modifying the global search path. You don't get more granularity than that, either all `matchDomains` are search paths, or none.
  * If you want to fully capture all DNS traffic but also set some global search domains, you can list both the empty string and non-empty strings in `matchDomains`, and you'll get that behavior.
  * You only get a single set of resolver IPs. This means you can have many DNS suffixes with `matchDomains`, but all of them will go to the pool of `servers` you provided. You can't route foo.com to 1.2.3.4 and bar.com to 2.3.4.5 without having another intermediate proxy to break things out further.
  * Apple's DNS management service only installs the "default" resolver into the legacy /etc/resolv.conf, so a bunch of commandline tools inherited from BSD will be completely unaware of split DNS configurations, because they don't use whatever the "correct" resolution API is (I presume mach port something something).
  * Apple doesn't have a public API for reading back the OS DNS settings, because they don't want other applications poorly reimplementing the OS's algorithm for selecting among contributed configuration. There is presumably an undocumented API that scutil et al. can use to read things out, but I've not dug into that at all.

Regarding upstreaming: OS-level DNS capabilities vary wildly across linux, apple and windows, and across versions of same (e.g. Windows 8+ can do split DNS, Windows 7 cannot - but WSL/WSL2 can't on any version to date). I can ramble at length about each OS if desired, but the bottom line is "send all DNS to these servers" is the only configuration that can be implemented reliably on all of them.

So, the question would be: do you want upstream WireGuard applications to have consistent behavior on all platforms? If so, you have to either forego split DNS, or do a lot of work to polyfill missing features on each platform (Tailscale's attempt at this is https://github.com/tailscale/tailscale/tree/main/net/dns). Or expose the uneven feature surface to the user, and delegate the pain of non-portability to them.

- Dave

From mhp.driessen at gmail.com  Sat Nov  6 09:47:03 2021
From: mhp.driessen at gmail.com (Matty Driessen)
Date: Sat, 6 Nov 2021 10:47:03 +0100
Subject: Split DNS for macOS
In-Reply-To: <c700d141-5348-4ce5-9398-dfb281383607@www.fastmail.com>
References: <20211028071638.88001-1-stephen@slarew.net>
 <0e80daf1-b4c2-b177-28ca-79e6575a339c@baywinds.org>
 <ED3FB4B3-96AD-4B98-86EC-D9F7999A530F@slarew.net>
 <K25247-14.1635527002@spamhaus.org>
 <E3607380-8E46-42CC-80B5-BB38F206E9BC@slarew.net>
 <c700d141-5348-4ce5-9398-dfb281383607@www.fastmail.com>
Message-ID: <CAHsVW7PFws2zMQnAUDv0J+=_kxMqXD+V1SVExrZx9g1jaovCPg@mail.gmail.com>

Hello Dave,

Thank you for this explanation and the challenges that come with split-DNS.

The biggest gripe I have with the current implementation in WireGuard
is that the search domain is not set as a global search domain when
using a split-tunnel. As you said the reason for this is "If you want
to fully capture all DNS traffic but also set some global search
domains, you can list both the empty string and non-empty strings in
`matchDomains`, and you'll get that behavior.".

When I use the same configuration file with the wg-quick utility the
search domain from the configuration file is set as a global search
domain. I think this is the case for all OS'es and should at least be
the case when using the WireGuard UI tool as well?

Regards,
Matty

On Sat, Nov 6, 2021 at 5:59 AM David Anderson <dave at natulte.net> wrote:
>
> On Fri, Oct 29, 2021, at 14:07, Stephen Larew wrote:
> > As I understand it, under some circumstances, the Tailscale macOS app also implements split DNS in roughly the same manner as done in my patch. Namely, the tailscale app appears to use the Network Extension APIs to directly integrate with the macOS DNS resolution machinery. Perhaps the relevant difference is that tailscale approaches configuration differently (not based on wg-quick) than the WireGuard macOS app.
>
> Hi, Tailscale person here.
>
> Yes, the Tailscale macOS app supports configuring either split DNS. It's situationally popular for things like "direct anything under amazonaws.com to the AWS VPC internal resolver on the other side of my tunnel", which makes your AWS VMs and such magically resolve correctly without shoveling all your unrelated requests through AWS.
>
> The semantics of NEDNSSettings I've worked out so far:
>   * `searchDomains` sets interface-scoped search domains only, which appear to not be used at all for single-label DNS query expansion. IOW, it seems to do mostly nothing - or at least, I've not discovered anything it affects.
>   * `matchDomains` specifies what DNS suffixes should be sent to your resolvers (specified in `servers`). Specifying a list of suffixes here implements split-DNS. Specifying "" (the empty suffix, which matches all names) captures all queries.
>   * `matchDomains` _also_ installs any non-empty string you specify in the global search path.
>   * `matchDomainsNoSearch` lets you make `matchDomains` be just match domains, without modifying the global search path. You don't get more granularity than that, either all `matchDomains` are search paths, or none.
>   * If you want to fully capture all DNS traffic but also set some global search domains, you can list both the empty string and non-empty strings in `matchDomains`, and you'll get that behavior.
>   * You only get a single set of resolver IPs. This means you can have many DNS suffixes with `matchDomains`, but all of them will go to the pool of `servers` you provided. You can't route foo.com to 1.2.3.4 and bar.com to 2.3.4.5 without having another intermediate proxy to break things out further.
>   * Apple's DNS management service only installs the "default" resolver into the legacy /etc/resolv.conf, so a bunch of commandline tools inherited from BSD will be completely unaware of split DNS configurations, because they don't use whatever the "correct" resolution API is (I presume mach port something something).
>   * Apple doesn't have a public API for reading back the OS DNS settings, because they don't want other applications poorly reimplementing the OS's algorithm for selecting among contributed configuration. There is presumably an undocumented API that scutil et al. can use to read things out, but I've not dug into that at all.
>
> Regarding upstreaming: OS-level DNS capabilities vary wildly across linux, apple and windows, and across versions of same (e.g. Windows 8+ can do split DNS, Windows 7 cannot - but WSL/WSL2 can't on any version to date). I can ramble at length about each OS if desired, but the bottom line is "send all DNS to these servers" is the only configuration that can be implemented reliably on all of them.
>
> So, the question would be: do you want upstream WireGuard applications to have consistent behavior on all platforms? If so, you have to either forego split DNS, or do a lot of work to polyfill missing features on each platform (Tailscale's attempt at this is https://github.com/tailscale/tailscale/tree/main/net/dns). Or expose the uneven feature surface to the user, and delegate the pain of non-portability to them.
>
> - Dave

From wireguard-mail at chil.at  Sat Nov  6 13:22:37 2021
From: wireguard-mail at chil.at (Christoph Loesch)
Date: Sat, 6 Nov 2021 14:22:37 +0100
Subject: Android app: length of name
Message-ID: <6b07934f-f0ce-5909-0595-3d50bc2a7c3a@chil.at>

Hi,

in the android app the name is limited to 15 characters.

Does that have a specific reason?
Is it possible to allow a longer name like e.g. 32 characters?

On importing a file the error "Invalid name" could be more specific to that.
Because I had to struggle with the file name until I found out it is just a limitation of the length.

Kind regards,
Christoph


From me at aaronmdjones.net  Sat Nov  6 16:54:42 2021
From: me at aaronmdjones.net (Aaron Jones)
Date: Sat, 6 Nov 2021 16:54:42 +0000
Subject: Android app: length of name
In-Reply-To: <6b07934f-f0ce-5909-0595-3d50bc2a7c3a@chil.at>
References: <6b07934f-f0ce-5909-0595-3d50bc2a7c3a@chil.at>
Message-ID: <1c7c517e-f0ee-8d02-75e1-5db2d92268f0@aaronmdjones.net>

On 06/11/2021 13:22, Christoph Loesch wrote:
> Hi,
>
> in the android app the name is limited to 15 characters.
>
> Does that have a specific reason?
> Is it possible to allow a longer name like e.g. 32 characters?
>
> On importing a file the error "Invalid name" could be more specific
> to that. Because I had to struggle with the file name until I found
> out it is just a limitation of the length.
>
> Kind regards,
> Christoph

The network name determines the interface name, and on Linux, the
length of an interface name (e.g. "eth0") is limited to 15 characters.

Regards,
Aaron Jones


-------------- next part --------------
A non-text attachment was scrubbed...
Name: OpenPGP_signature
Type: application/pgp-signature
Size: 833 bytes
Desc: OpenPGP digital signature
URL: <http://lists.zx2c4.com/pipermail/wireguard/attachments/20211106/30552b5a/attachment.sig>

From wireguard-mail at chil.at  Sat Nov  6 17:21:00 2021
From: wireguard-mail at chil.at (Christoph Loesch)
Date: Sat, 6 Nov 2021 18:21:00 +0100
Subject: Android app: length of name
In-Reply-To: <1c7c517e-f0ee-8d02-75e1-5db2d92268f0@aaronmdjones.net>
References: <6b07934f-f0ce-5909-0595-3d50bc2a7c3a@chil.at>
 <1c7c517e-f0ee-8d02-75e1-5db2d92268f0@aaronmdjones.net>
Message-ID: <04d29715-a3d5-59b4-11bc-5b2b932937a9@chil.at>


Am 06.11.2021 um 17:54 schrieb Aaron Jones:
> On 06/11/2021 13:22, Christoph Loesch wrote:
>> Hi,
>>
>> in the android app the name is limited to 15 characters.
>>
>> Does that have a specific reason?
>> Is it possible to allow a longer name like e.g. 32 characters?
>>
>> On importing a file the error "Invalid name" could be more specific
>> to that. Because I had to struggle with the file name until I found
>> out it is just a limitation of the length.
>>
>> Kind regards,
>> Christoph
> The network name determines the interface name, and on Linux, the
> length of an interface name (e.g. "eth0") is limited to 15 characters.
>
> Regards,
> Aaron Jones
>
Oh, I didn't know that, thanks, that explains the limitation.

Then I would suggest to note the limitation of 15 characters in the error message on config import, if possible.

Kind regards,
Christoph


From ryanroosa at gmail.com  Tue Nov  9 17:19:14 2021
From: ryanroosa at gmail.com (Ryan Roosa)
Date: Tue, 9 Nov 2021 12:19:14 -0500
Subject: wireguard-freebsd handshaking issue upon underlying WAN
In-Reply-To: <CABOEyuJidG_u8SFGnhV6aw5y7PHVhaW_diDrpdMMHfOE4fHRWA@mail.gmail.com>
References: <CABOEyuLiu96_KxEH5sdGLaE5-s9e8F_c0a1PKHSnFcMKp5YHow@mail.gmail.com>
 <CAHmME9oPi=LvdcZC-FA+SvyxEmcRULtNB8hAQZoCAX=kQycTXA@mail.gmail.com>
 <CABOEyuJidG_u8SFGnhV6aw5y7PHVhaW_diDrpdMMHfOE4fHRWA@mail.gmail.com>
Message-ID: <CABOEyuKT7ebroX2yRU9xixDVCBmJV15Hy1vh-nUwrmdGjX6=QQ@mail.gmail.com>

Just wanted to provide some feedback that pfSense development
snapshots of 2.6.0 running WireGuard package v0.1.5_2 include the fix
and there I have validated that removing WAN connectivity at various
intervals up to 10 minutes no longer impacts subsequent handshaking
once the connection is restored. I have not yet tested on OPNSense but
I imagine the results will match once I do (if not I will reach out).
Thanks to everyone for their efforts on resolving this one, I really
appreciate it.

-Ryan

On Wed, Oct 27, 2021 at 7:45 PM Ryan Roosa <ryanroosa at gmail.com> wrote:
>
> Hi Jason,
> Thank you very much for this! I received word from the OPNSense team
> that the referenced snapshot should be made available in OPNSense
> 21.7.5. I will test and provide feedback just as soon as I can get on
> the aforementioned OPNSense release which includes the fix.
>
> Cheers,
> -Ryan
>
> On Tue, Oct 26, 2021 at 5:29 AM Jason A. Donenfeld <Jason at zx2c4.com> wrote:
> >
> > Hi Ryan,
> >
> > Thanks for the report. Kyle saw your reddit post earlier and tracked
> > this down, I think/hope, to a bug in the state machine cranking. I
> > committed the fix here -- https://w-g.pw/l/yQTw -- which will be part
> > of the next snapshot. Hopefully that will fix the issue, but if it
> > doesn't, please do update this thread so we can keep searching.
> >
> > Regards,
> > Jason

From ludwig.huber79 at icloud.com  Mon Nov  8 16:19:13 2021
From: ludwig.huber79 at icloud.com (Ludwig Huber)
Date: Mon, 8 Nov 2021 17:19:13 +0100
Subject: iOS on demand VPN
Message-ID: <84B7A7A3-9E7F-4040-9BA5-8BCAA00FF3AF@icloud.com>

Hello,

would it be possible that the Wireguard iOS App also gets support for VPN on demand via DNS domain match (https://developer.apple.com/documentation/networkextension/neondemandrule/1406150-dnssearchdomainmatch).
I would like to use Wireguard only when trying to access my smart home server. In the current version I have to leave VPN always on, which costs a lot of battery.

Thanks a lot,
Ludwig

From tim at tschumacher.net  Sat Nov  6 12:40:02 2021
From: tim at tschumacher.net (Tim Schumacher)
Date: Sat, 6 Nov 2021 13:40:02 +0100
Subject: wg-quick's Ethernet unplug problem
Message-ID: <20211106134002.68a3ade2@impa>

Hi,

there is this annoying problem with the way wg-quick sets up the routing, that when you unplug the Ethernet cable the routing rule (see ip rule) that checks the fwmark gets lost. So let's say you move to another location with your laptop and plug in an Ethernet cable again, it essentially disables your VPN. You're lucky if you have the popular kill switch in place blocking all your traffic instead of silently disabling your VPN.

PostUp = iptables -I OUTPUT ! -o %i -m mark ! --mark $(wg show %i fwmark) -m addrtype ! --dst-type LOCAL -j REJECT
PreDown = iptables -D OUTPUT ! -o %i -m mark ! --mark $(wg show %i fwmark) -m addrtype ! --dst-type LOCAL -j REJECT

I have no idea why the rule gets lost. It might be the kernel removing it for whatever reason. But it makes no sense to me because I don't see how the kernel would make a connection between that rule and the Ethernet link that goes down. Any ideas on the issue? Is this a known problem? Are there workarounds?

Greetings,
Tim

From noloader at gmail.com  Sat Nov  6 21:20:06 2021
From: noloader at gmail.com (Jeffrey Walton)
Date: Sat, 6 Nov 2021 17:20:06 -0400
Subject: Android app: length of name
In-Reply-To: <04d29715-a3d5-59b4-11bc-5b2b932937a9@chil.at>
References: <6b07934f-f0ce-5909-0595-3d50bc2a7c3a@chil.at>
 <1c7c517e-f0ee-8d02-75e1-5db2d92268f0@aaronmdjones.net>
 <04d29715-a3d5-59b4-11bc-5b2b932937a9@chil.at>
Message-ID: <CAH8yC8ngpm4R=owBvXSo6ZN34_BuZXG-odf_ybtmJHk8pBCCvg@mail.gmail.com>

On Sat, Nov 6, 2021 at 1:23 PM Christoph Loesch <wireguard-mail at chil.at> wrote:
> ...
> Oh, I didn't know that, thanks, that explains the limitation.
>
> Then I would suggest to note the limitation of 15 characters in the error message on config import, if possible.

I _think_ that may be this define:

$ grep -IR -E 'IFNAME|IF_NAME' /usr/include | grep -E 'LENGTH|SIZE'
/usr/include/net/if.h:#define IF_NAMESIZE       16
/usr/include/net/if.h:# define IFNAMSIZ IF_NAMESIZE

Jeff

From kevans at freebsd.org  Wed Nov 10 06:36:41 2021
From: kevans at freebsd.org (Kyle Evans)
Date: Wed, 10 Nov 2021 00:36:41 -0600
Subject: wireguard-freebsd handshaking issue upon underlying WAN
In-Reply-To: <CABOEyuKT7ebroX2yRU9xixDVCBmJV15Hy1vh-nUwrmdGjX6=QQ@mail.gmail.com>
References: <CABOEyuLiu96_KxEH5sdGLaE5-s9e8F_c0a1PKHSnFcMKp5YHow@mail.gmail.com>
 <CAHmME9oPi=LvdcZC-FA+SvyxEmcRULtNB8hAQZoCAX=kQycTXA@mail.gmail.com>
 <CABOEyuJidG_u8SFGnhV6aw5y7PHVhaW_diDrpdMMHfOE4fHRWA@mail.gmail.com>
 <CABOEyuKT7ebroX2yRU9xixDVCBmJV15Hy1vh-nUwrmdGjX6=QQ@mail.gmail.com>
Message-ID: <CACNAnaHYUxPvV53NditMvNfkn9YvYSv=ief=FDew2kmHjJ5hgg@mail.gmail.com>

On Tue, Nov 9, 2021 at 11:19 AM Ryan Roosa <ryanroosa at gmail.com> wrote:
>
> On Wed, Oct 27, 2021 at 7:45 PM Ryan Roosa <ryanroosa at gmail.com> wrote:
> >
> > On Tue, Oct 26, 2021 at 5:29 AM Jason A. Donenfeld <Jason at zx2c4.com> wrote:
> > >
> > > Hi Ryan,
> > >
> > > Thanks for the report. Kyle saw your reddit post earlier and tracked
> > > this down, I think/hope, to a bug in the state machine cranking. I
> > > committed the fix here -- https://w-g.pw/l/yQTw -- which will be part
> > > of the next snapshot. Hopefully that will fix the issue, but if it
> > > doesn't, please do update this thread so we can keep searching.
> > >
> > > Regards,
> > > Jason
> >
> > Hi Jason,
> > Thank you very much for this! I received word from the OPNSense team
> > that the referenced snapshot should be made available in OPNSense
> > 21.7.5. I will test and provide feedback just as soon as I can get on
> > the aforementioned OPNSense release which includes the fix.
> >
> > Cheers,
> > -Ryan
> >
>
> Just wanted to provide some feedback that pfSense development
> snapshots of 2.6.0 running WireGuard package v0.1.5_2 include the fix
> and there I have validated that removing WAN connectivity at various
> intervals up to 10 minutes no longer impacts subsequent handshaking
> once the connection is restored. I have not yet tested on OPNSense but
> I imagine the results will match once I do (if not I will reach out).
> Thanks to everyone for their efforts on resolving this one, I really
> appreciate it.
>
> -Ryan
>

That's good to hear, thanks for following up! :-)

Kyle Evans

From jorge-wg at at.anteris.net  Fri Nov 12 07:14:48 2021
From: jorge-wg at at.anteris.net (Georg Lehner)
Date: Fri, 12 Nov 2021 08:14:48 +0100
Subject: Wireguard setup tool
Message-ID: <37ef73aa-4244-9ee7-e2b8-d1738a8d3e78@at.anteris.net>

Hello,

Thank's for Wireguard, way cool!

I'd like to share a setup tool I'm using:

 ? https://at.magma-soft.at/darcs/wgsetup/

You'll find wgsetup, a shell script, and a README.md file there.

wgsetup fetches configuration information for a peer via https:// from a 
.well-known location.

A node using wgsetup can only "see" configuration information for peers 
in the same "realm". A realm shares a common secret which has to be 
distributed out-of-band.

When wgsetup sets up a connection to a peer it prints out shell 
executable peer configuration instructions for the case when the node is 
not reachable via https://, e.g. because they hide behind a NAT.

Connections are always configured with pre shared keys, which are never 
disclosed by wgsetup.

wgsetup requires a POSIX shell, gpg, some tools of the Gnu Coreutils and 
curl.? Current use is on Debian Linux, I'll be happy to receive feedback 
and adapt it to other environments.

Best Regards,

 ? Georg



From max at sdn.clinic  Mon Nov 15 15:57:58 2021
From: max at sdn.clinic (Maximilian Wilhelm)
Date: Mon, 15 Nov 2021 16:57:58 +0100
Subject: [PATCH 0/1] Add support to send/receive tunnel packets via Linux VRFs
Message-ID: <d1c0a3e8-c461-3d70-e23f-fceac21cbd43@sdn.clinic>

Hi,

this patch adds support to bind the listening/sending UDP sockets to a
specific network interface on Linux to allow sending/receiving encrypted
tunnel packets via a given VRF.

The motivation behind this is to put any external interface(s) of a host
into a VRF (e.g. "vrf_external") and have the rest of the interface in the
main VRF, so all tools and services can be used normally and can't directly
reach/leak to the Internet.  This way different island can be connected
easily and securely without the fear of leaking any communication into the
Internet.  See my blog article on VRF support for OpenVPN [0] for a more
detailed description.

Using VRFs is much simpler than setting up Network Namespaces, shoving
interfaces around and having to care about which processes should run
where.

This patch is very similar to the patch "Basic support for binding the
transport socket to a device" sent in June this year, but adds two more
settings in send4() and send6() without VRF support didn't work for me.

I'm unsure wether it would be better to use the name "bind_dev" within
the kernel or clearly denote that an ifindex is used, as it is now.

The user-space part of this change is available as a PR to the 
wireguard-linux
repo on GitHub [1], but can be sent as patches to the list as well.

Best
Max

[0] https://blog.sdn.clinic/2018/12/openvpn-and-vrfs/
[1] https://github.com/WireGuard/wireguard-tools/pull/12

Maximilian Wilhelm (1):
   wireguard: Add support to bind socket(s) to device

  drivers/net/wireguard/device.h  |  1 +
  drivers/net/wireguard/netlink.c |  7 ++++++-
  drivers/net/wireguard/socket.c  | 13 +++++++++++++
  include/uapi/linux/wireguard.h  |  2 ++
  4 files changed, 22 insertions(+), 1 deletion(-)

-- 
2.20.1

From max at sdn.clinic  Mon Nov 15 16:04:29 2021
From: max at sdn.clinic (Maximilian Wilhelm)
Date: Mon, 15 Nov 2021 17:04:29 +0100
Subject: [PATCH 1/1] wireguard: Add support to bind socket(s) to device
In-Reply-To: <d1c0a3e8-c461-3d70-e23f-fceac21cbd43@sdn.clinic>
References: <d1c0a3e8-c461-3d70-e23f-fceac21cbd43@sdn.clinic>
Message-ID: <80d36571-9345-5855-1710-ca5a796e2fe9@sdn.clinic>

From: Maximilian Wilhelm <max at sdn.clinic>

   This introduces support for binding Wireguards UDP socket(s) to a
   given interface allowing to send/receive encapsulated packets via
   a VRF.

Signed-off-by: Maximilian Wilhelm <max at sdn.clinic>
---
  drivers/net/wireguard/device.h  |  1 +
  drivers/net/wireguard/netlink.c |  7 ++++++-
  drivers/net/wireguard/socket.c  | 13 +++++++++++++
  include/uapi/linux/wireguard.h  |  2 ++
  4 files changed, 22 insertions(+), 1 deletion(-)

diff --git a/drivers/net/wireguard/device.h b/drivers/net/wireguard/device.h
index 854bc3d97150..85f9fe687cb8 100644
--- a/drivers/net/wireguard/device.h
+++ b/drivers/net/wireguard/device.h
@@ -56,6 +56,7 @@ struct wg_device {
  	struct list_head device_list, peer_list;
  	unsigned int num_peers, device_update_gen;
  	u32 fwmark;
+	u32 bind_ifindex;
  	u16 incoming_port;
  };

diff --git a/drivers/net/wireguard/netlink.c 
b/drivers/net/wireguard/netlink.c
index d0f3b6d7f408..064402a11eb3 100644
--- a/drivers/net/wireguard/netlink.c
+++ b/drivers/net/wireguard/netlink.c
@@ -27,7 +27,8 @@ static const struct nla_policy 
device_policy[WGDEVICE_A_MAX + 1] = {
  	[WGDEVICE_A_FLAGS]		= { .type = NLA_U32 },
  	[WGDEVICE_A_LISTEN_PORT]	= { .type = NLA_U16 },
  	[WGDEVICE_A_FWMARK]		= { .type = NLA_U32 },
-	[WGDEVICE_A_PEERS]		= { .type = NLA_NESTED }
+	[WGDEVICE_A_PEERS]		= { .type = NLA_NESTED },
+	[WGDEVICE_A_BIND_IFINDEX]	= { .type = NLA_U32 }
  };

  static const struct nla_policy peer_policy[WGPEER_A_MAX + 1] = {
@@ -232,6 +233,7 @@ static int wg_get_device_dump(struct sk_buff *skb, 
struct netlink_callback *cb)
  		if (nla_put_u16(skb, WGDEVICE_A_LISTEN_PORT,
  				wg->incoming_port) ||
  		    nla_put_u32(skb, WGDEVICE_A_FWMARK, wg->fwmark) ||
+		    nla_put_u32(skb, WGDEVICE_A_BIND_IFINDEX, wg->bind_ifindex) ||
  		    nla_put_u32(skb, WGDEVICE_A_IFINDEX, wg->dev->ifindex) ||
  		    nla_put_string(skb, WGDEVICE_A_IFNAME, wg->dev->name))
  			goto out;
@@ -531,6 +533,9 @@ static int wg_set_device(struct sk_buff *skb, struct 
genl_info *info)
  			wg_socket_clear_peer_endpoint_src(peer);
  	}

+	if (info->attrs[WGDEVICE_A_BIND_IFINDEX])
+		wg->bind_ifindex = nla_get_u32(info->attrs[WGDEVICE_A_BIND_IFINDEX]);
+
  	if (info->attrs[WGDEVICE_A_LISTEN_PORT]) {
  		ret = set_port(wg,
  			nla_get_u16(info->attrs[WGDEVICE_A_LISTEN_PORT]));
diff --git a/drivers/net/wireguard/socket.c b/drivers/net/wireguard/socket.c
index 8c496b747108..d319288a8f3b 100644
--- a/drivers/net/wireguard/socket.c
+++ b/drivers/net/wireguard/socket.c
@@ -35,6 +35,9 @@ static int send4(struct wg_device *wg, struct sk_buff 
*skb,
  	skb->dev = wg->dev;
  	skb->mark = wg->fwmark;

+	if (wg->bind_ifindex)
+		fl.flowi4_oif = wg->bind_ifindex;
+
  	rcu_read_lock_bh();
  	sock = rcu_dereference_bh(wg->sock4);

@@ -115,6 +118,9 @@ static int send6(struct wg_device *wg, struct 
sk_buff *skb,
  	skb->dev = wg->dev;
  	skb->mark = wg->fwmark;

+	if (wg->bind_ifindex)
+		fl.flowi6_oif = wg->bind_ifindex;
+
  	rcu_read_lock_bh();
  	sock = rcu_dereference_bh(wg->sock6);

@@ -379,6 +385,13 @@ int wg_socket_init(struct wg_device *wg, u16 port)
  	if (unlikely(!net))
  		return -ENONET;

+	if (wg->bind_ifindex) {
+		port4.bind_ifindex = wg->bind_ifindex;
+#if IS_ENABLED(CONFIG_IPV6)
+		port6.bind_ifindex = wg->bind_ifindex;
+#endif
+	}
+
  #if IS_ENABLED(CONFIG_IPV6)
  retry:
  #endif
diff --git a/include/uapi/linux/wireguard.h b/include/uapi/linux/wireguard.h
index ae88be14c947..5c49919596b8 100644
--- a/include/uapi/linux/wireguard.h
+++ b/include/uapi/linux/wireguard.h
@@ -114,6 +114,7 @@
   *        0: NLA_NESTED
   *            ...
   *        ...
+ *    WGDEVICE_A_BIND_IFINDEX: NLA_U32
   *
   * It is possible that the amount of configuration data exceeds that of
   * the maximum message length accepted by the kernel. In that case, 
several
@@ -157,6 +158,7 @@ enum wgdevice_attribute {
  	WGDEVICE_A_LISTEN_PORT,
  	WGDEVICE_A_FWMARK,
  	WGDEVICE_A_PEERS,
+	WGDEVICE_A_BIND_IFINDEX,
  	__WGDEVICE_A_LAST
  };
  #define WGDEVICE_A_MAX (__WGDEVICE_A_LAST - 1)
-- 
2.20.1


From liuhangbin at gmail.com  Tue Nov 16 08:13:59 2021
From: liuhangbin at gmail.com (Hangbin Liu)
Date: Tue, 16 Nov 2021 16:13:59 +0800
Subject: [PATCH wireguard] wireguard: selftests: refactor the test structure
Message-ID: <20211116081359.975655-1-liuhangbin@gmail.com>

The current wireguard selftest mixed all tests together, which makes it
very hard to read. Refactor the test and split it roughly to basic, nat,
saddr and regression tests to make it more clear.

Move the common functions to a new file lib.sh and rename
the main test to wg_test.sh

Also, fix some config issues during re-organize. e.g.
  - update basic topo in lib.sh
  - Fix the IPv6 prefix to 112 in all topology
  - add 127.0.0.2 on lo for saddr test as ncat need an IP address to
    bind

Signed-off-by: Hangbin Liu <liuhangbin at gmail.com>
---
 tools/testing/selftests/wireguard/lib.sh     | 149 +++++
 tools/testing/selftests/wireguard/netns.sh   | 662 -------------------
 tools/testing/selftests/wireguard/wg_test.sh | 621 +++++++++++++++++
 3 files changed, 770 insertions(+), 662 deletions(-)
 create mode 100644 tools/testing/selftests/wireguard/lib.sh
 delete mode 100755 tools/testing/selftests/wireguard/netns.sh
 create mode 100755 tools/testing/selftests/wireguard/wg_test.sh

diff --git a/tools/testing/selftests/wireguard/lib.sh b/tools/testing/selftests/wireguard/lib.sh
new file mode 100644
index 000000000000..2621a16c821b
--- /dev/null
+++ b/tools/testing/selftests/wireguard/lib.sh
@@ -0,0 +1,149 @@
+#!/bin/bash
+# SPDX-License-Identifier: GPL-2.0
+#
+
+exec 3>&1
+export LANG=C
+export WG_HIDE_KEYS=never
+has_basic_topo=false
+netns0="wg-test-$$-0"
+netns1="wg-test-$$-1"
+netns2="wg-test-$$-2"
+pretty() { echo -e "\x1b[32m\x1b[1m[+] ${1:+NS$1: }${2}\x1b[0m" >&3; }
+pp() { pretty "" "$*"; "$@"; }
+maybe_exec() { if [[ $BASHPID -eq $$ ]]; then "$@"; else exec "$@"; fi; }
+n0() { pretty 0 "$*"; maybe_exec ip netns exec $netns0 "$@"; }
+n1() { pretty 1 "$*"; maybe_exec ip netns exec $netns1 "$@"; }
+n2() { pretty 2 "$*"; maybe_exec ip netns exec $netns2 "$@"; }
+ip0() { pretty 0 "ip $*"; ip -n $netns0 "$@"; }
+ip1() { pretty 1 "ip $*"; ip -n $netns1 "$@"; }
+ip2() { pretty 2 "ip $*"; ip -n $netns2 "$@"; }
+sleep() { read -t "$1" -N 1 || true; }
+waitiperf() { pretty "${1//*-}" "wait for iperf:${3:-5201} pid $2"; while [[ $(ss -N "$1" -tlpH "sport = ${3:-5201}") != *\"iperf3\",pid=$2,fd=* ]]; do sleep 0.1; done; }
+waitncatudp() { pretty "${1//*-}" "wait for udp:1111 pid $2"; while [[ $(ss -N "$1" -ulpH 'sport = 1111') != *\"ncat\",pid=$2,fd=* ]]; do sleep 0.1; done; }
+waitiface() { pretty "${1//*-}" "wait for $2 to come up"; ip netns exec "$1" bash -c "while [[ \$(< \"/sys/class/net/$2/operstate\") != up ]]; do read -t .1 -N 0 || true; done;"; }
+orig_message_cost="$(< /proc/sys/net/core/message_cost)"
+
+cleanup() {
+	set +e
+	exec 2>/dev/null
+	printf "$orig_message_cost" > /proc/sys/net/core/message_cost
+	ip0 link del dev wg0
+	ip0 link del dev wg1
+	ip1 link del dev wg0
+	ip1 link del dev wg1
+	ip2 link del dev wg0
+	ip2 link del dev wg1
+	local to_kill="$(ip netns pids $netns0) $(ip netns pids $netns1) $(ip netns pids $netns2)"
+	[[ -n $to_kill ]] && kill $to_kill
+	pp ip netns del $netns1
+	pp ip netns del $netns2
+	pp ip netns del $netns0
+	exit
+}
+
+# The follow functions setup the below topology:
+#
+# ??????????????????????   ??????????????????????   ??????????????????????
+# ?   $ns1 namespace   ?   ?   $ns0 namespace   ?   ?   $ns2 namespace   ?
+# ?                    ?   ?                    ?   ?                    ?
+# ?     ??????????     ?   ?     ??????????     ?   ?     ??????????     ?
+# ?     ?  wg0   ?     ?   ?     ?   lo   ?     ?   ?     ?  wg0   ?     ?
+# ??????????????????????   ?     ??????????     ?   ??????????????????????
+# ??192.168.241.1/24  ??   ?                    ?   ??192.168.241.2/24  ??
+# ??fd00::1/112       ??   ?                    ?   ??fd00::2/112       ??
+# ??????????????????????   ?                    ?   ??????????????????????
+# ??????????????????????   ??????????????????????   ??????????????????????
+
+configure_peers() {
+	ip1 addr add 192.168.241.1/24 dev wg0
+	ip1 addr add fd00::1/112 dev wg0
+
+	ip2 addr add 192.168.241.2/24 dev wg0
+	ip2 addr add fd00::2/112 dev wg0
+
+	n1 wg set wg0 \
+		private-key <(echo "$key1") \
+		listen-port 1 \
+		peer "$pub2" \
+			preshared-key <(echo "$psk") \
+			allowed-ips 192.168.241.2/32,fd00::2/128
+	n2 wg set wg0 \
+		private-key <(echo "$key2") \
+		listen-port 2 \
+		peer "$pub1" \
+			preshared-key <(echo "$psk") \
+			allowed-ips 192.168.241.1/32,fd00::1/128
+
+	ip1 link set up dev wg0
+	ip2 link set up dev wg0
+}
+
+setup_basic_topo()
+{
+	printf 0 > /proc/sys/net/core/message_cost
+
+	ip netns del $netns0 2>/dev/null || true
+	ip netns del $netns1 2>/dev/null || true
+	ip netns del $netns2 2>/dev/null || true
+	pp ip netns add $netns0
+	pp ip netns add $netns1
+	pp ip netns add $netns2
+	ip0 link set up dev lo
+
+	ip1 link add dev wg0 type wireguard
+	ip2 link add dev wg0 type wireguard
+	key1="$(pp wg genkey)"
+	key2="$(pp wg genkey)"
+	key3="$(pp wg genkey)"
+	key4="$(pp wg genkey)"
+	pub1="$(pp wg pubkey <<<"$key1")"
+	pub2="$(pp wg pubkey <<<"$key2")"
+	pub3="$(pp wg pubkey <<<"$key3")"
+	pub4="$(pp wg pubkey <<<"$key4")"
+	psk="$(pp wg genpsk)"
+	[[ -n $key1 && -n $key2 && -n $psk ]]
+}
+
+do_tests() {
+	# Ping over IPv4
+	n2 ping -c 10 -f -W 1 192.168.241.1
+	n1 ping -c 10 -f -W 1 192.168.241.2
+
+	# Ping over IPv6
+	n2 ping6 -c 10 -f -W 1 fd00::1
+	n1 ping6 -c 10 -f -W 1 fd00::2
+
+	# TCP over IPv4
+	n2 iperf3 -s -1 -B 192.168.241.2 &
+	waitiperf $netns2 $!
+	n1 iperf3 -Z -t 3 -c 192.168.241.2
+
+	# TCP over IPv6
+	n1 iperf3 -s -1 -B fd00::1 &
+	waitiperf $netns1 $!
+	n2 iperf3 -Z -t 3 -c fd00::1
+
+	# UDP over IPv4
+	n1 iperf3 -s -1 -B 192.168.241.1 &
+	waitiperf $netns1 $!
+	n2 iperf3 -Z -t 3 -b 0 -u -c 192.168.241.1
+
+	# UDP over IPv6
+	n2 iperf3 -s -1 -B fd00::2 &
+	waitiperf $netns2 $!
+	n1 iperf3 -Z -t 3 -b 0 -u -c fd00::2
+
+	# TCP over IPv4, in parallel
+	for max in 4 5 50; do
+		local pids=( )
+		for ((i=0; i < max; ++i)) do
+			n2 iperf3 -p $(( 5200 + i )) -s -1 -B 192.168.241.2 &
+			pids+=( $! ); waitiperf $netns2 $! $(( 5200 + i ))
+		done
+		for ((i=0; i < max; ++i)) do
+			n1 iperf3 -Z -t 3 -p $(( 5200 + i )) -c 192.168.241.2 &
+		done
+		wait "${pids[@]}"
+	done
+}
diff --git a/tools/testing/selftests/wireguard/netns.sh b/tools/testing/selftests/wireguard/netns.sh
deleted file mode 100755
index 8a9461aa0878..000000000000
--- a/tools/testing/selftests/wireguard/netns.sh
+++ /dev/null
@@ -1,662 +0,0 @@
-#!/bin/bash
-# SPDX-License-Identifier: GPL-2.0
-#
-# Copyright (C) 2015-2019 Jason A. Donenfeld <Jason at zx2c4.com>. All Rights Reserved.
-#
-# This script tests the below topology:
-#
-# ???????????????????????   ????????????????????????????????????   ???????????????????????
-# ?   $ns1 namespace    ?   ?          $ns0 namespace          ?   ?   $ns2 namespace    ?
-# ?                     ?   ?                                  ?   ?                     ?
-# ???????????           ?   ?            ??????????            ?   ?           ???????????
-# ??  wg0   ??????????????????????????????   lo   ??????????????????????????????  wg0   ??
-# ???????????????????????   ?    ???????????????????????????   ?   ???????????????????????
-# ??192.168.241.1/24   ??   ?    ?(ns1)         (ns2)      ?   ?   ??192.168.241.2/24   ??
-# ??fd00::1/24         ??   ?    ?127.0.0.1:1   127.0.0.1:2?   ?   ??fd00::2/24         ??
-# ???????????????????????   ?    ?[::]:1        [::]:2     ?   ?   ???????????????????????
-# ???????????????????????   ?    ???????????????????????????   ?   ???????????????????????
-#                           ????????????????????????????????????
-#
-# After the topology is prepared we run a series of TCP/UDP iperf3 tests between the
-# wireguard peers in $ns1 and $ns2. Note that $ns0 is the endpoint for the wg0
-# interfaces in $ns1 and $ns2. See https://www.wireguard.com/netns/ for further
-# details on how this is accomplished.
-set -e
-
-exec 3>&1
-export LANG=C
-export WG_HIDE_KEYS=never
-netns0="wg-test-$$-0"
-netns1="wg-test-$$-1"
-netns2="wg-test-$$-2"
-pretty() { echo -e "\x1b[32m\x1b[1m[+] ${1:+NS$1: }${2}\x1b[0m" >&3; }
-pp() { pretty "" "$*"; "$@"; }
-maybe_exec() { if [[ $BASHPID -eq $$ ]]; then "$@"; else exec "$@"; fi; }
-n0() { pretty 0 "$*"; maybe_exec ip netns exec $netns0 "$@"; }
-n1() { pretty 1 "$*"; maybe_exec ip netns exec $netns1 "$@"; }
-n2() { pretty 2 "$*"; maybe_exec ip netns exec $netns2 "$@"; }
-ip0() { pretty 0 "ip $*"; ip -n $netns0 "$@"; }
-ip1() { pretty 1 "ip $*"; ip -n $netns1 "$@"; }
-ip2() { pretty 2 "ip $*"; ip -n $netns2 "$@"; }
-sleep() { read -t "$1" -N 1 || true; }
-waitiperf() { pretty "${1//*-}" "wait for iperf:${3:-5201} pid $2"; while [[ $(ss -N "$1" -tlpH "sport = ${3:-5201}") != *\"iperf3\",pid=$2,fd=* ]]; do sleep 0.1; done; }
-waitncatudp() { pretty "${1//*-}" "wait for udp:1111 pid $2"; while [[ $(ss -N "$1" -ulpH 'sport = 1111') != *\"ncat\",pid=$2,fd=* ]]; do sleep 0.1; done; }
-waitiface() { pretty "${1//*-}" "wait for $2 to come up"; ip netns exec "$1" bash -c "while [[ \$(< \"/sys/class/net/$2/operstate\") != up ]]; do read -t .1 -N 0 || true; done;"; }
-
-cleanup() {
-	set +e
-	exec 2>/dev/null
-	printf "$orig_message_cost" > /proc/sys/net/core/message_cost
-	ip0 link del dev wg0
-	ip0 link del dev wg1
-	ip1 link del dev wg0
-	ip1 link del dev wg1
-	ip2 link del dev wg0
-	ip2 link del dev wg1
-	local to_kill="$(ip netns pids $netns0) $(ip netns pids $netns1) $(ip netns pids $netns2)"
-	[[ -n $to_kill ]] && kill $to_kill
-	pp ip netns del $netns1
-	pp ip netns del $netns2
-	pp ip netns del $netns0
-	exit
-}
-
-orig_message_cost="$(< /proc/sys/net/core/message_cost)"
-trap cleanup EXIT
-printf 0 > /proc/sys/net/core/message_cost
-
-ip netns del $netns0 2>/dev/null || true
-ip netns del $netns1 2>/dev/null || true
-ip netns del $netns2 2>/dev/null || true
-pp ip netns add $netns0
-pp ip netns add $netns1
-pp ip netns add $netns2
-ip0 link set up dev lo
-
-ip0 link add dev wg0 type wireguard
-ip0 link set wg0 netns $netns1
-ip0 link add dev wg0 type wireguard
-ip0 link set wg0 netns $netns2
-key1="$(pp wg genkey)"
-key2="$(pp wg genkey)"
-key3="$(pp wg genkey)"
-key4="$(pp wg genkey)"
-pub1="$(pp wg pubkey <<<"$key1")"
-pub2="$(pp wg pubkey <<<"$key2")"
-pub3="$(pp wg pubkey <<<"$key3")"
-pub4="$(pp wg pubkey <<<"$key4")"
-psk="$(pp wg genpsk)"
-[[ -n $key1 && -n $key2 && -n $psk ]]
-
-configure_peers() {
-	ip1 addr add 192.168.241.1/24 dev wg0
-	ip1 addr add fd00::1/112 dev wg0
-
-	ip2 addr add 192.168.241.2/24 dev wg0
-	ip2 addr add fd00::2/112 dev wg0
-
-	n1 wg set wg0 \
-		private-key <(echo "$key1") \
-		listen-port 1 \
-		peer "$pub2" \
-			preshared-key <(echo "$psk") \
-			allowed-ips 192.168.241.2/32,fd00::2/128
-	n2 wg set wg0 \
-		private-key <(echo "$key2") \
-		listen-port 2 \
-		peer "$pub1" \
-			preshared-key <(echo "$psk") \
-			allowed-ips 192.168.241.1/32,fd00::1/128
-
-	ip1 link set up dev wg0
-	ip2 link set up dev wg0
-}
-configure_peers
-
-tests() {
-	# Ping over IPv4
-	n2 ping -c 10 -f -W 1 192.168.241.1
-	n1 ping -c 10 -f -W 1 192.168.241.2
-
-	# Ping over IPv6
-	n2 ping6 -c 10 -f -W 1 fd00::1
-	n1 ping6 -c 10 -f -W 1 fd00::2
-
-	# TCP over IPv4
-	n2 iperf3 -s -1 -B 192.168.241.2 &
-	waitiperf $netns2 $!
-	n1 iperf3 -Z -t 3 -c 192.168.241.2
-
-	# TCP over IPv6
-	n1 iperf3 -s -1 -B fd00::1 &
-	waitiperf $netns1 $!
-	n2 iperf3 -Z -t 3 -c fd00::1
-
-	# UDP over IPv4
-	n1 iperf3 -s -1 -B 192.168.241.1 &
-	waitiperf $netns1 $!
-	n2 iperf3 -Z -t 3 -b 0 -u -c 192.168.241.1
-
-	# UDP over IPv6
-	n2 iperf3 -s -1 -B fd00::2 &
-	waitiperf $netns2 $!
-	n1 iperf3 -Z -t 3 -b 0 -u -c fd00::2
-
-	# TCP over IPv4, in parallel
-	for max in 4 5 50; do
-		local pids=( )
-		for ((i=0; i < max; ++i)) do
-			n2 iperf3 -p $(( 5200 + i )) -s -1 -B 192.168.241.2 &
-			pids+=( $! ); waitiperf $netns2 $! $(( 5200 + i ))
-		done
-		for ((i=0; i < max; ++i)) do
-			n1 iperf3 -Z -t 3 -p $(( 5200 + i )) -c 192.168.241.2 &
-		done
-		wait "${pids[@]}"
-	done
-}
-
-[[ $(ip1 link show dev wg0) =~ mtu\ ([0-9]+) ]] && orig_mtu="${BASH_REMATCH[1]}"
-big_mtu=$(( 34816 - 1500 + $orig_mtu ))
-
-# Test using IPv4 as outer transport
-n1 wg set wg0 peer "$pub2" endpoint 127.0.0.1:2
-n2 wg set wg0 peer "$pub1" endpoint 127.0.0.1:1
-# Before calling tests, we first make sure that the stats counters and timestamper are working
-n2 ping -c 10 -f -W 1 192.168.241.1
-{ read _; read _; read _; read rx_bytes _; read _; read tx_bytes _; } < <(ip2 -stats link show dev wg0)
-(( rx_bytes == 1372 && (tx_bytes == 1428 || tx_bytes == 1460) ))
-{ read _; read _; read _; read rx_bytes _; read _; read tx_bytes _; } < <(ip1 -stats link show dev wg0)
-(( tx_bytes == 1372 && (rx_bytes == 1428 || rx_bytes == 1460) ))
-read _ rx_bytes tx_bytes < <(n2 wg show wg0 transfer)
-(( rx_bytes == 1372 && (tx_bytes == 1428 || tx_bytes == 1460) ))
-read _ rx_bytes tx_bytes < <(n1 wg show wg0 transfer)
-(( tx_bytes == 1372 && (rx_bytes == 1428 || rx_bytes == 1460) ))
-read _ timestamp < <(n1 wg show wg0 latest-handshakes)
-(( timestamp != 0 ))
-
-tests
-ip1 link set wg0 mtu $big_mtu
-ip2 link set wg0 mtu $big_mtu
-tests
-
-ip1 link set wg0 mtu $orig_mtu
-ip2 link set wg0 mtu $orig_mtu
-
-# Test using IPv6 as outer transport
-n1 wg set wg0 peer "$pub2" endpoint [::1]:2
-n2 wg set wg0 peer "$pub1" endpoint [::1]:1
-tests
-ip1 link set wg0 mtu $big_mtu
-ip2 link set wg0 mtu $big_mtu
-tests
-
-# Test that route MTUs work with the padding
-ip1 link set wg0 mtu 1300
-ip2 link set wg0 mtu 1300
-n1 wg set wg0 peer "$pub2" endpoint 127.0.0.1:2
-n2 wg set wg0 peer "$pub1" endpoint 127.0.0.1:1
-n0 iptables -A INPUT -m length --length 1360 -j DROP
-n1 ip route add 192.168.241.2/32 dev wg0 mtu 1299
-n2 ip route add 192.168.241.1/32 dev wg0 mtu 1299
-n2 ping -c 1 -W 1 -s 1269 192.168.241.1
-n2 ip route delete 192.168.241.1/32 dev wg0 mtu 1299
-n1 ip route delete 192.168.241.2/32 dev wg0 mtu 1299
-n0 iptables -F INPUT
-
-ip1 link set wg0 mtu $orig_mtu
-ip2 link set wg0 mtu $orig_mtu
-
-# Test using IPv4 that roaming works
-ip0 -4 addr del 127.0.0.1/8 dev lo
-ip0 -4 addr add 127.212.121.99/8 dev lo
-n1 wg set wg0 listen-port 9999
-n1 wg set wg0 peer "$pub2" endpoint 127.0.0.1:2
-n1 ping6 -W 1 -c 1 fd00::2
-[[ $(n2 wg show wg0 endpoints) == "$pub1	127.212.121.99:9999" ]]
-
-# Test using IPv6 that roaming works
-n1 wg set wg0 listen-port 9998
-n1 wg set wg0 peer "$pub2" endpoint [::1]:2
-n1 ping -W 1 -c 1 192.168.241.2
-[[ $(n2 wg show wg0 endpoints) == "$pub1	[::1]:9998" ]]
-
-# Test that crypto-RP filter works
-n1 wg set wg0 peer "$pub2" allowed-ips 192.168.241.0/24
-exec 4< <(n1 ncat -l -u -p 1111)
-ncat_pid=$!
-waitncatudp $netns1 $ncat_pid
-n2 ncat -u 192.168.241.1 1111 <<<"X"
-read -r -N 1 -t 1 out <&4 && [[ $out == "X" ]]
-kill $ncat_pid
-more_specific_key="$(pp wg genkey | pp wg pubkey)"
-n1 wg set wg0 peer "$more_specific_key" allowed-ips 192.168.241.2/32
-n2 wg set wg0 listen-port 9997
-exec 4< <(n1 ncat -l -u -p 1111)
-ncat_pid=$!
-waitncatudp $netns1 $ncat_pid
-n2 ncat -u 192.168.241.1 1111 <<<"X"
-! read -r -N 1 -t 1 out <&4 || false
-kill $ncat_pid
-n1 wg set wg0 peer "$more_specific_key" remove
-[[ $(n1 wg show wg0 endpoints) == "$pub2	[::1]:9997" ]]
-
-# Test that we can change private keys keys and immediately handshake
-n1 wg set wg0 private-key <(echo "$key1") peer "$pub2" preshared-key <(echo "$psk") allowed-ips 192.168.241.2/32 endpoint 127.0.0.1:2
-n2 wg set wg0 private-key <(echo "$key2") listen-port 2 peer "$pub1" preshared-key <(echo "$psk") allowed-ips 192.168.241.1/32
-n1 ping -W 1 -c 1 192.168.241.2
-n1 wg set wg0 private-key <(echo "$key3")
-n2 wg set wg0 peer "$pub3" preshared-key <(echo "$psk") allowed-ips 192.168.241.1/32 peer "$pub1" remove
-n1 ping -W 1 -c 1 192.168.241.2
-n2 wg set wg0 peer "$pub3" remove
-
-# Test that we can route wg through wg
-ip1 addr flush dev wg0
-ip2 addr flush dev wg0
-ip1 addr add fd00::5:1/112 dev wg0
-ip2 addr add fd00::5:2/112 dev wg0
-n1 wg set wg0 private-key <(echo "$key1") peer "$pub2" preshared-key <(echo "$psk") allowed-ips fd00::5:2/128 endpoint 127.0.0.1:2
-n2 wg set wg0 private-key <(echo "$key2") listen-port 2 peer "$pub1" preshared-key <(echo "$psk") allowed-ips fd00::5:1/128 endpoint 127.212.121.99:9998
-ip1 link add wg1 type wireguard
-ip2 link add wg1 type wireguard
-ip1 addr add 192.168.241.1/24 dev wg1
-ip1 addr add fd00::1/112 dev wg1
-ip2 addr add 192.168.241.2/24 dev wg1
-ip2 addr add fd00::2/112 dev wg1
-ip1 link set mtu 1340 up dev wg1
-ip2 link set mtu 1340 up dev wg1
-n1 wg set wg1 listen-port 5 private-key <(echo "$key3") peer "$pub4" allowed-ips 192.168.241.2/32,fd00::2/128 endpoint [fd00::5:2]:5
-n2 wg set wg1 listen-port 5 private-key <(echo "$key4") peer "$pub3" allowed-ips 192.168.241.1/32,fd00::1/128 endpoint [fd00::5:1]:5
-tests
-# Try to set up a routing loop between the two namespaces
-ip1 link set netns $netns0 dev wg1
-ip0 addr add 192.168.241.1/24 dev wg1
-ip0 link set up dev wg1
-n0 ping -W 1 -c 1 192.168.241.2
-n1 wg set wg0 peer "$pub2" endpoint 192.168.241.2:7
-ip2 link del wg0
-ip2 link del wg1
-read _ _ tx_bytes_before < <(n0 wg show wg1 transfer)
-! n0 ping -W 1 -c 10 -f 192.168.241.2 || false
-sleep 1
-read _ _ tx_bytes_after < <(n0 wg show wg1 transfer)
-(( tx_bytes_after - tx_bytes_before < 70000 ))
-
-ip0 link del wg1
-ip1 link del wg0
-
-# Test using NAT. We now change the topology to this:
-# ??????????????????????????????????????????    ??????????????????????????????????????????????????     ??????????????????????????????????????????
-# ?             $ns1 namespace             ?    ?                 $ns0 namespace                 ?     ?             $ns2 namespace             ?
-# ?                                        ?    ?                                                ?     ?                                        ?
-# ?  ???????             ???????           ?    ?    ????????              ????????              ?     ?  ???????            ???????            ?
-# ?  ? wg0 ???????????????vethc???????????????????????vethrc?              ?vethrs?????????????????????????veths?????????????? wg0 ?            ?
-# ?  ??????????????????  ???????????????????    ?    ??????????????????    ????????????????????? ?     ?  ?????????????????? ?????????????????? ?
-# ?  ?192.168.241.1/24?  ?192.168.1.100/24??    ?    ?192.168.1.1/24  ?    ?10.0.0.1/24        ? ?     ?  ?10.0.0.100/24   ? ?192.168.241.2/24? ?
-# ?  ?fd00::1/24      ?  ?                ??    ?    ?                ?    ?SNAT:192.168.1.0/24? ?     ?  ?                ? ?fd00::2/24      ? ?
-# ?  ??????????????????  ???????????????????    ?    ??????????????????    ????????????????????? ?     ?  ?????????????????? ?????????????????? ?
-# ??????????????????????????????????????????    ??????????????????????????????????????????????????     ??????????????????????????????????????????
-
-ip1 link add dev wg0 type wireguard
-ip2 link add dev wg0 type wireguard
-configure_peers
-
-ip0 link add vethrc type veth peer name vethc
-ip0 link add vethrs type veth peer name veths
-ip0 link set vethc netns $netns1
-ip0 link set veths netns $netns2
-ip0 link set vethrc up
-ip0 link set vethrs up
-ip0 addr add 192.168.1.1/24 dev vethrc
-ip0 addr add 10.0.0.1/24 dev vethrs
-ip1 addr add 192.168.1.100/24 dev vethc
-ip1 link set vethc up
-ip1 route add default via 192.168.1.1
-ip2 addr add 10.0.0.100/24 dev veths
-ip2 link set veths up
-waitiface $netns0 vethrc
-waitiface $netns0 vethrs
-waitiface $netns1 vethc
-waitiface $netns2 veths
-
-n0 bash -c 'printf 1 > /proc/sys/net/ipv4/ip_forward'
-n0 bash -c 'printf 2 > /proc/sys/net/netfilter/nf_conntrack_udp_timeout'
-n0 bash -c 'printf 2 > /proc/sys/net/netfilter/nf_conntrack_udp_timeout_stream'
-n0 iptables -t nat -A POSTROUTING -s 192.168.1.0/24 -d 10.0.0.0/24 -j SNAT --to 10.0.0.1
-
-n1 wg set wg0 peer "$pub2" endpoint 10.0.0.100:2 persistent-keepalive 1
-n1 ping -W 1 -c 1 192.168.241.2
-n2 ping -W 1 -c 1 192.168.241.1
-[[ $(n2 wg show wg0 endpoints) == "$pub1	10.0.0.1:1" ]]
-# Demonstrate n2 can still send packets to n1, since persistent-keepalive will prevent connection tracking entry from expiring (to see entries: `n0 conntrack -L`).
-pp sleep 3
-n2 ping -W 1 -c 1 192.168.241.1
-n1 wg set wg0 peer "$pub2" persistent-keepalive 0
-
-# Test that sk_bound_dev_if works
-n1 ping -I wg0 -c 1 -W 1 192.168.241.2
-# What about when the mark changes and the packet must be rerouted?
-n1 iptables -t mangle -I OUTPUT -j MARK --set-xmark 1
-n1 ping -c 1 -W 1 192.168.241.2 # First the boring case
-n1 ping -I wg0 -c 1 -W 1 192.168.241.2 # Then the sk_bound_dev_if case
-n1 iptables -t mangle -D OUTPUT -j MARK --set-xmark 1
-
-# Test that onion routing works, even when it loops
-n1 wg set wg0 peer "$pub3" allowed-ips 192.168.242.2/32 endpoint 192.168.241.2:5
-ip1 addr add 192.168.242.1/24 dev wg0
-ip2 link add wg1 type wireguard
-ip2 addr add 192.168.242.2/24 dev wg1
-n2 wg set wg1 private-key <(echo "$key3") listen-port 5 peer "$pub1" allowed-ips 192.168.242.1/32
-ip2 link set wg1 up
-n1 ping -W 1 -c 1 192.168.242.2
-ip2 link del wg1
-n1 wg set wg0 peer "$pub3" endpoint 192.168.242.2:5
-! n1 ping -W 1 -c 1 192.168.242.2 || false # Should not crash kernel
-n1 wg set wg0 peer "$pub3" remove
-ip1 addr del 192.168.242.1/24 dev wg0
-
-# Do a wg-quick(8)-style policy routing for the default route, making sure vethc has a v6 address to tease out bugs.
-ip1 -6 addr add fc00::9/96 dev vethc
-ip1 -6 route add default via fc00::1
-ip2 -4 addr add 192.168.99.7/32 dev wg0
-ip2 -6 addr add abab::1111/128 dev wg0
-n1 wg set wg0 fwmark 51820 peer "$pub2" allowed-ips 192.168.99.7,abab::1111
-ip1 -6 route add default dev wg0 table 51820
-ip1 -6 rule add not fwmark 51820 table 51820
-ip1 -6 rule add table main suppress_prefixlength 0
-ip1 -4 route add default dev wg0 table 51820
-ip1 -4 rule add not fwmark 51820 table 51820
-ip1 -4 rule add table main suppress_prefixlength 0
-n1 bash -c 'printf 0 > /proc/sys/net/ipv4/conf/vethc/rp_filter'
-# Flood the pings instead of sending just one, to trigger routing table reference counting bugs.
-n1 ping -W 1 -c 100 -f 192.168.99.7
-n1 ping -W 1 -c 100 -f abab::1111
-
-# Have ns2 NAT into wg0 packets from ns0, but return an icmp error along the right route.
-n2 iptables -t nat -A POSTROUTING -s 10.0.0.0/24 -d 192.168.241.0/24 -j SNAT --to 192.168.241.2
-n0 iptables -t filter -A INPUT \! -s 10.0.0.0/24 -i vethrs -j DROP # Manual rpfilter just to be explicit.
-n2 bash -c 'printf 1 > /proc/sys/net/ipv4/ip_forward'
-ip0 -4 route add 192.168.241.1 via 10.0.0.100
-n2 wg set wg0 peer "$pub1" remove
-[[ $(! n0 ping -W 1 -c 1 192.168.241.1 || false) == *"From 10.0.0.100 icmp_seq=1 Destination Host Unreachable"* ]]
-
-n0 iptables -t nat -F
-n0 iptables -t filter -F
-n2 iptables -t nat -F
-ip0 link del vethrc
-ip0 link del vethrs
-ip1 link del wg0
-ip2 link del wg0
-
-# Test that saddr routing is sticky but not too sticky, changing to this topology:
-# ??????????????????????????????????????????    ??????????????????????????????????????????
-# ?             $ns1 namespace             ?    ?             $ns2 namespace             ?
-# ?                                        ?    ?                                        ?
-# ?  ???????             ???????           ?    ?  ???????            ???????            ?
-# ?  ? wg0 ???????????????veth1?????????????????????veth2?????????????? wg0 ?            ?
-# ?  ??????????????????  ???????????????????    ?  ?????????????????? ?????????????????? ?
-# ?  ?192.168.241.1/24?  ?10.0.0.1/24     ??    ?  ?10.0.0.2/24     ? ?192.168.241.2/24? ?
-# ?  ?fd00::1/24      ?  ?fd00:aa::1/96   ??    ?  ?fd00:aa::2/96   ? ?fd00::2/24      ? ?
-# ?  ??????????????????  ???????????????????    ?  ?????????????????? ?????????????????? ?
-# ??????????????????????????????????????????    ??????????????????????????????????????????
-
-ip1 link add dev wg0 type wireguard
-ip2 link add dev wg0 type wireguard
-configure_peers
-ip1 link add veth1 type veth peer name veth2
-ip1 link set veth2 netns $netns2
-n1 bash -c 'printf 0 > /proc/sys/net/ipv6/conf/all/accept_dad'
-n2 bash -c 'printf 0 > /proc/sys/net/ipv6/conf/all/accept_dad'
-n1 bash -c 'printf 0 > /proc/sys/net/ipv6/conf/veth1/accept_dad'
-n2 bash -c 'printf 0 > /proc/sys/net/ipv6/conf/veth2/accept_dad'
-n1 bash -c 'printf 1 > /proc/sys/net/ipv4/conf/veth1/promote_secondaries'
-
-# First we check that we aren't overly sticky and can fall over to new IPs when old ones are removed
-ip1 addr add 10.0.0.1/24 dev veth1
-ip1 addr add fd00:aa::1/96 dev veth1
-ip2 addr add 10.0.0.2/24 dev veth2
-ip2 addr add fd00:aa::2/96 dev veth2
-ip1 link set veth1 up
-ip2 link set veth2 up
-waitiface $netns1 veth1
-waitiface $netns2 veth2
-n1 wg set wg0 peer "$pub2" endpoint 10.0.0.2:2
-n1 ping -W 1 -c 1 192.168.241.2
-ip1 addr add 10.0.0.10/24 dev veth1
-ip1 addr del 10.0.0.1/24 dev veth1
-n1 ping -W 1 -c 1 192.168.241.2
-n1 wg set wg0 peer "$pub2" endpoint [fd00:aa::2]:2
-n1 ping -W 1 -c 1 192.168.241.2
-ip1 addr add fd00:aa::10/96 dev veth1
-ip1 addr del fd00:aa::1/96 dev veth1
-n1 ping -W 1 -c 1 192.168.241.2
-
-# Now we show that we can successfully do reply to sender routing
-ip1 link set veth1 down
-ip2 link set veth2 down
-ip1 addr flush dev veth1
-ip2 addr flush dev veth2
-ip1 addr add 10.0.0.1/24 dev veth1
-ip1 addr add 10.0.0.2/24 dev veth1
-ip1 addr add fd00:aa::1/96 dev veth1
-ip1 addr add fd00:aa::2/96 dev veth1
-ip2 addr add 10.0.0.3/24 dev veth2
-ip2 addr add fd00:aa::3/96 dev veth2
-ip1 link set veth1 up
-ip2 link set veth2 up
-waitiface $netns1 veth1
-waitiface $netns2 veth2
-n2 wg set wg0 peer "$pub1" endpoint 10.0.0.1:1
-n2 ping -W 1 -c 1 192.168.241.1
-[[ $(n2 wg show wg0 endpoints) == "$pub1	10.0.0.1:1" ]]
-n2 wg set wg0 peer "$pub1" endpoint [fd00:aa::1]:1
-n2 ping -W 1 -c 1 192.168.241.1
-[[ $(n2 wg show wg0 endpoints) == "$pub1	[fd00:aa::1]:1" ]]
-n2 wg set wg0 peer "$pub1" endpoint 10.0.0.2:1
-n2 ping -W 1 -c 1 192.168.241.1
-[[ $(n2 wg show wg0 endpoints) == "$pub1	10.0.0.2:1" ]]
-n2 wg set wg0 peer "$pub1" endpoint [fd00:aa::2]:1
-n2 ping -W 1 -c 1 192.168.241.1
-[[ $(n2 wg show wg0 endpoints) == "$pub1	[fd00:aa::2]:1" ]]
-
-# What happens if the inbound destination address belongs to a different interface as the default route?
-ip1 link add dummy0 type dummy
-ip1 addr add 10.50.0.1/24 dev dummy0
-ip1 link set dummy0 up
-ip2 route add 10.50.0.0/24 dev veth2
-n2 wg set wg0 peer "$pub1" endpoint 10.50.0.1:1
-n2 ping -W 1 -c 1 192.168.241.1
-[[ $(n2 wg show wg0 endpoints) == "$pub1	10.50.0.1:1" ]]
-
-ip1 link del dummy0
-ip1 addr flush dev veth1
-ip2 addr flush dev veth2
-ip1 route flush dev veth1
-ip2 route flush dev veth2
-
-# Now we see what happens if another interface route takes precedence over an ongoing one
-ip1 link add veth3 type veth peer name veth4
-ip1 link set veth4 netns $netns2
-ip1 addr add 10.0.0.1/24 dev veth1
-ip2 addr add 10.0.0.2/24 dev veth2
-ip1 addr add 10.0.0.3/24 dev veth3
-ip1 link set veth1 up
-ip2 link set veth2 up
-ip1 link set veth3 up
-ip2 link set veth4 up
-waitiface $netns1 veth1
-waitiface $netns2 veth2
-waitiface $netns1 veth3
-waitiface $netns2 veth4
-ip1 route flush dev veth1
-ip1 route flush dev veth3
-ip1 route add 10.0.0.0/24 dev veth1 src 10.0.0.1 metric 2
-n1 wg set wg0 peer "$pub2" endpoint 10.0.0.2:2
-n1 ping -W 1 -c 1 192.168.241.2
-[[ $(n2 wg show wg0 endpoints) == "$pub1	10.0.0.1:1" ]]
-ip1 route add 10.0.0.0/24 dev veth3 src 10.0.0.3 metric 1
-n1 bash -c 'printf 0 > /proc/sys/net/ipv4/conf/veth1/rp_filter'
-n2 bash -c 'printf 0 > /proc/sys/net/ipv4/conf/veth4/rp_filter'
-n1 bash -c 'printf 0 > /proc/sys/net/ipv4/conf/all/rp_filter'
-n2 bash -c 'printf 0 > /proc/sys/net/ipv4/conf/all/rp_filter'
-n1 ping -W 1 -c 1 192.168.241.2
-[[ $(n2 wg show wg0 endpoints) == "$pub1	10.0.0.3:1" ]]
-
-ip1 link del veth1
-ip1 link del veth3
-ip1 link del wg0
-ip2 link del wg0
-
-# We test that Netlink/IPC is working properly by doing things that usually cause split responses
-ip0 link add dev wg0 type wireguard
-config=( "[Interface]" "PrivateKey=$(wg genkey)" "[Peer]" "PublicKey=$(wg genkey)" )
-for a in {1..255}; do
-	for b in {0..255}; do
-		config+=( "AllowedIPs=$a.$b.0.0/16,$a::$b/128" )
-	done
-done
-n0 wg setconf wg0 <(printf '%s\n' "${config[@]}")
-i=0
-for ip in $(n0 wg show wg0 allowed-ips); do
-	((++i))
-done
-((i == 255*256*2+1))
-ip0 link del wg0
-ip0 link add dev wg0 type wireguard
-config=( "[Interface]" "PrivateKey=$(wg genkey)" )
-for a in {1..40}; do
-	config+=( "[Peer]" "PublicKey=$(wg genkey)" )
-	for b in {1..52}; do
-		config+=( "AllowedIPs=$a.$b.0.0/16" )
-	done
-done
-n0 wg setconf wg0 <(printf '%s\n' "${config[@]}")
-i=0
-while read -r line; do
-	j=0
-	for ip in $line; do
-		((++j))
-	done
-	((j == 53))
-	((++i))
-done < <(n0 wg show wg0 allowed-ips)
-((i == 40))
-ip0 link del wg0
-ip0 link add wg0 type wireguard
-config=( )
-for i in {1..29}; do
-	config+=( "[Peer]" "PublicKey=$(wg genkey)" )
-done
-config+=( "[Peer]" "PublicKey=$(wg genkey)" "AllowedIPs=255.2.3.4/32,abcd::255/128" )
-n0 wg setconf wg0 <(printf '%s\n' "${config[@]}")
-n0 wg showconf wg0 > /dev/null
-ip0 link del wg0
-
-allowedips=( )
-for i in {1..197}; do
-        allowedips+=( abcd::$i )
-done
-saved_ifs="$IFS"
-IFS=,
-allowedips="${allowedips[*]}"
-IFS="$saved_ifs"
-ip0 link add wg0 type wireguard
-n0 wg set wg0 peer "$pub1"
-n0 wg set wg0 peer "$pub2" allowed-ips "$allowedips"
-{
-	read -r pub allowedips
-	[[ $pub == "$pub1" && $allowedips == "(none)" ]]
-	read -r pub allowedips
-	[[ $pub == "$pub2" ]]
-	i=0
-	for _ in $allowedips; do
-		((++i))
-	done
-	((i == 197))
-} < <(n0 wg show wg0 allowed-ips)
-ip0 link del wg0
-
-! n0 wg show doesnotexist || false
-
-ip0 link add wg0 type wireguard
-n0 wg set wg0 private-key <(echo "$key1") peer "$pub2" preshared-key <(echo "$psk")
-[[ $(n0 wg show wg0 private-key) == "$key1" ]]
-[[ $(n0 wg show wg0 preshared-keys) == "$pub2	$psk" ]]
-n0 wg set wg0 private-key /dev/null peer "$pub2" preshared-key /dev/null
-[[ $(n0 wg show wg0 private-key) == "(none)" ]]
-[[ $(n0 wg show wg0 preshared-keys) == "$pub2	(none)" ]]
-n0 wg set wg0 peer "$pub2"
-n0 wg set wg0 private-key <(echo "$key2")
-[[ $(n0 wg show wg0 public-key) == "$pub2" ]]
-[[ -z $(n0 wg show wg0 peers) ]]
-n0 wg set wg0 peer "$pub2"
-[[ -z $(n0 wg show wg0 peers) ]]
-n0 wg set wg0 private-key <(echo "$key1")
-n0 wg set wg0 peer "$pub2"
-[[ $(n0 wg show wg0 peers) == "$pub2" ]]
-n0 wg set wg0 private-key <(echo "/${key1:1}")
-[[ $(n0 wg show wg0 private-key) == "+${key1:1}" ]]
-n0 wg set wg0 peer "$pub2" allowed-ips 0.0.0.0/0,10.0.0.0/8,100.0.0.0/10,172.16.0.0/12,192.168.0.0/16
-n0 wg set wg0 peer "$pub2" allowed-ips 0.0.0.0/0
-n0 wg set wg0 peer "$pub2" allowed-ips ::/0,1700::/111,5000::/4,e000::/37,9000::/75
-n0 wg set wg0 peer "$pub2" allowed-ips ::/0
-n0 wg set wg0 peer "$pub2" remove
-for low_order_point in AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA= AQAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA= 4Ot6fDtBuK4WVuP68Z/EatoJjeucMrH9hmIFFl9JuAA= X5yVvKNQjCSx0LFVnIPvWwREXMRYHI6G2CJO3dCfEVc= 7P///////////////////////////////////////38= 7f///////////////////////////////////////38= 7v///////////////////////////////////////38=; do
-	n0 wg set wg0 peer "$low_order_point" persistent-keepalive 1 endpoint 127.0.0.1:1111
-done
-[[ -n $(n0 wg show wg0 peers) ]]
-exec 4< <(n0 ncat -l -u -p 1111)
-ncat_pid=$!
-waitncatudp $netns0 $ncat_pid
-ip0 link set wg0 up
-! read -r -n 1 -t 2 <&4 || false
-kill $ncat_pid
-ip0 link del wg0
-
-# Ensure that dst_cache references don't outlive netns lifetime
-ip1 link add dev wg0 type wireguard
-ip2 link add dev wg0 type wireguard
-configure_peers
-ip1 link add veth1 type veth peer name veth2
-ip1 link set veth2 netns $netns2
-ip1 addr add fd00:aa::1/64 dev veth1
-ip2 addr add fd00:aa::2/64 dev veth2
-ip1 link set veth1 up
-ip2 link set veth2 up
-waitiface $netns1 veth1
-waitiface $netns2 veth2
-ip1 -6 route add default dev veth1 via fd00:aa::2
-ip2 -6 route add default dev veth2 via fd00:aa::1
-n1 wg set wg0 peer "$pub2" endpoint [fd00:aa::2]:2
-n2 wg set wg0 peer "$pub1" endpoint [fd00:aa::1]:1
-n1 ping6 -c 1 fd00::2
-pp ip netns delete $netns1
-pp ip netns delete $netns2
-pp ip netns add $netns1
-pp ip netns add $netns2
-
-# Ensure there aren't circular reference loops
-ip1 link add wg1 type wireguard
-ip2 link add wg2 type wireguard
-ip1 link set wg1 netns $netns2
-ip2 link set wg2 netns $netns1
-pp ip netns delete $netns1
-pp ip netns delete $netns2
-pp ip netns add $netns1
-pp ip netns add $netns2
-
-sleep 2 # Wait for cleanup and grace periods
-declare -A objects
-while read -t 0.1 -r line 2>/dev/null || [[ $? -ne 142 ]]; do
-	[[ $line =~ .*(wg[0-9]+:\ [A-Z][a-z]+\ ?[0-9]*)\ .*(created|destroyed).* ]] || continue
-	objects["${BASH_REMATCH[1]}"]+="${BASH_REMATCH[2]}"
-done < /dev/kmsg
-alldeleted=1
-for object in "${!objects[@]}"; do
-	if [[ ${objects["$object"]} != *createddestroyed && ${objects["$object"]} != *createdcreateddestroyeddestroyed ]]; then
-		echo "Error: $object: merely ${objects["$object"]}" >&3
-		alldeleted=0
-	fi
-done
-[[ $alldeleted -eq 1 ]]
-pretty "" "Objects that were created were also destroyed."
diff --git a/tools/testing/selftests/wireguard/wg_test.sh b/tools/testing/selftests/wireguard/wg_test.sh
new file mode 100755
index 000000000000..faea481f19f8
--- /dev/null
+++ b/tools/testing/selftests/wireguard/wg_test.sh
@@ -0,0 +1,621 @@
+#!/bin/bash
+# SPDX-License-Identifier: GPL-2.0
+#
+# Copyright (C) 2015-2019 Jason A. Donenfeld <Jason at zx2c4.com>. All Rights Reserved.
+#
+set -e
+
+source lib.sh
+
+TESTS="basic nat saddr regression"
+
+################################################################################
+# basic test
+# This script tests the below topology:
+#
+# ???????????????????????   ????????????????????????????????????   ???????????????????????
+# ?   $ns1 namespace    ?   ?          $ns0 namespace          ?   ?   $ns2 namespace    ?
+# ?                     ?   ?                                  ?   ?                     ?
+# ???????????           ?   ?            ??????????            ?   ?           ???????????
+# ??  wg0   ??????????????????????????????   lo   ??????????????????????????????  wg0   ??
+# ???????????????????????   ?    ???????????????????????????   ?   ???????????????????????
+# ??192.168.241.1/24   ??   ?    ?(ns1)         (ns2)      ?   ?   ??192.168.241.2/24   ??
+# ??fd00::1/112        ??   ?    ?127.0.0.1:1   127.0.0.1:2?   ?   ??fd00::2/112        ??
+# ???????????????????????   ?    ?[::]:1        [::]:2     ?   ?   ???????????????????????
+# ???????????????????????   ?    ???????????????????????????   ?   ???????????????????????
+#                           ????????????????????????????????????
+#
+# After the topology is prepared we run a series of TCP/UDP iperf3 tests between the
+# wireguard peers in $ns1 and $ns2. Note that $ns0 is the endpoint for the wg0
+# interfaces in $ns1 and $ns2. See https://www.wireguard.com/netns/ for further
+# details on how this is accomplished.
+
+basic_test()
+{
+	$has_basic_topo || { setup_basic_topo && has_basic_topo=true; }
+
+	# add wg0 form ns0 to use lo end points
+	ip1 link del wg0
+	ip2 link del wg0
+	ip0 link add dev wg0 type wireguard
+	ip0 link set wg0 netns $netns1
+	ip0 link add dev wg0 type wireguard
+	ip0 link set wg0 netns $netns2
+
+	configure_peers
+
+	[[ $(ip1 link show dev wg0) =~ mtu\ ([0-9]+) ]] && orig_mtu="${BASH_REMATCH[1]}"
+	big_mtu=$(( 34816 - 1500 + $orig_mtu ))
+
+	# Test using IPv4 as outer transport
+	n1 wg set wg0 peer "$pub2" endpoint 127.0.0.1:2
+	n2 wg set wg0 peer "$pub1" endpoint 127.0.0.1:1
+	# Before calling tests, we first make sure that the stats counters and timestamper are working
+	n2 ping -c 10 -f -W 1 192.168.241.1
+	{ read _; read _; read _; read rx_bytes _; read _; read tx_bytes _; } < <(ip2 -stats link show dev wg0)
+	(( rx_bytes == 1372 && (tx_bytes == 1428 || tx_bytes == 1460) ))
+	{ read _; read _; read _; read rx_bytes _; read _; read tx_bytes _; } < <(ip1 -stats link show dev wg0)
+	(( tx_bytes == 1372 && (rx_bytes == 1428 || rx_bytes == 1460) ))
+	read _ rx_bytes tx_bytes < <(n2 wg show wg0 transfer)
+	(( rx_bytes == 1372 && (tx_bytes == 1428 || tx_bytes == 1460) ))
+	read _ rx_bytes tx_bytes < <(n1 wg show wg0 transfer)
+	(( tx_bytes == 1372 && (rx_bytes == 1428 || rx_bytes == 1460) ))
+	read _ timestamp < <(n1 wg show wg0 latest-handshakes)
+	(( timestamp != 0 ))
+
+	do_tests
+
+	ip1 link set wg0 mtu $big_mtu
+	ip2 link set wg0 mtu $big_mtu
+	do_tests
+
+	ip1 link set wg0 mtu $orig_mtu
+	ip2 link set wg0 mtu $orig_mtu
+
+	# Test using IPv6 as outer transport
+	n1 wg set wg0 peer "$pub2" endpoint [::1]:2
+	n2 wg set wg0 peer "$pub1" endpoint [::1]:1
+	do_tests
+
+	ip1 link set wg0 mtu $big_mtu
+	ip2 link set wg0 mtu $big_mtu
+	do_tests
+
+	# Test that route MTUs work with the padding
+	ip1 link set wg0 mtu 1300
+	ip2 link set wg0 mtu 1300
+	n1 wg set wg0 peer "$pub2" endpoint 127.0.0.1:2
+	n2 wg set wg0 peer "$pub1" endpoint 127.0.0.1:1
+	n0 iptables -A INPUT -m length --length 1360 -j DROP
+	n1 ip route add 192.168.241.2/32 dev wg0 mtu 1299
+	n2 ip route add 192.168.241.1/32 dev wg0 mtu 1299
+	n2 ping -c 1 -W 1 -s 1269 192.168.241.1
+	n2 ip route delete 192.168.241.1/32 dev wg0 mtu 1299
+	n1 ip route delete 192.168.241.2/32 dev wg0 mtu 1299
+	n0 iptables -F INPUT
+
+	ip1 link set wg0 mtu $orig_mtu
+	ip2 link set wg0 mtu $orig_mtu
+
+	# Test using IPv4 that roaming works
+	ip0 -4 addr del 127.0.0.1/8 dev lo
+	ip0 -4 addr add 127.212.121.99/8 dev lo
+	n1 wg set wg0 listen-port 9999
+	n1 wg set wg0 peer "$pub2" endpoint 127.0.0.1:2
+	n1 ping6 -W 1 -c 1 fd00::2
+	[[ $(n2 wg show wg0 endpoints) == "$pub1	127.212.121.99:9999" ]]
+
+	# Test using IPv6 that roaming works
+	n1 wg set wg0 listen-port 9998
+	n1 wg set wg0 peer "$pub2" endpoint [::1]:2
+	n1 ping -W 1 -c 1 192.168.241.2
+	[[ $(n2 wg show wg0 endpoints) == "$pub1	[::1]:9998" ]]
+
+	# Test that crypto-RP filter works
+	n1 wg set wg0 peer "$pub2" allowed-ips 192.168.241.0/24
+	exec 4< <(n1 ncat -l -u -p 1111)
+	ncat_pid=$!
+	waitncatudp $netns1 $ncat_pid
+	n2 ncat -u 192.168.241.1 1111 <<<"X"
+	read -r -N 1 -t 1 out <&4 && [[ $out == "X" ]]
+	kill $ncat_pid
+	more_specific_key="$(pp wg genkey | pp wg pubkey)"
+	n1 wg set wg0 peer "$more_specific_key" allowed-ips 192.168.241.2/32
+	n2 wg set wg0 listen-port 9997
+	exec 4< <(n1 ncat -l -u -p 1111)
+	ncat_pid=$!
+	waitncatudp $netns1 $ncat_pid
+	n2 ncat -u 192.168.241.1 1111 <<<"X"
+	! read -r -N 1 -t 1 out <&4 || false
+	kill $ncat_pid
+	n1 wg set wg0 peer "$more_specific_key" remove
+	[[ $(n1 wg show wg0 endpoints) == "$pub2	[::1]:9997" ]]
+
+	# Test that we can change private keys keys and immediately handshake
+	n1 wg set wg0 private-key <(echo "$key1") peer "$pub2" preshared-key <(echo "$psk") allowed-ips 192.168.241.2/32 endpoint 127.0.0.1:2
+	n2 wg set wg0 private-key <(echo "$key2") listen-port 2 peer "$pub1" preshared-key <(echo "$psk") allowed-ips 192.168.241.1/32
+	n1 ping -W 1 -c 1 192.168.241.2
+	n1 wg set wg0 private-key <(echo "$key3")
+	n2 wg set wg0 peer "$pub3" preshared-key <(echo "$psk") allowed-ips 192.168.241.1/32 peer "$pub1" remove
+	n1 ping -W 1 -c 1 192.168.241.2
+	n2 wg set wg0 peer "$pub3" remove
+
+	# Test that we can route wg through wg
+	ip1 addr flush dev wg0
+	ip2 addr flush dev wg0
+	ip1 addr add fd00::5:1/112 dev wg0
+	ip2 addr add fd00::5:2/112 dev wg0
+	n1 wg set wg0 private-key <(echo "$key1") peer "$pub2" preshared-key <(echo "$psk") allowed-ips fd00::5:2/128 endpoint 127.0.0.1:2
+	n2 wg set wg0 private-key <(echo "$key2") listen-port 2 peer "$pub1" preshared-key <(echo "$psk") allowed-ips fd00::5:1/128 endpoint 127.212.121.99:9998
+	ip1 link add wg1 type wireguard
+	ip2 link add wg1 type wireguard
+	ip1 addr add 192.168.241.1/24 dev wg1
+	ip1 addr add fd00::1/112 dev wg1
+	ip2 addr add 192.168.241.2/24 dev wg1
+	ip2 addr add fd00::2/112 dev wg1
+	ip1 link set mtu 1340 up dev wg1
+	ip2 link set mtu 1340 up dev wg1
+	n1 wg set wg1 listen-port 5 private-key <(echo "$key3") peer "$pub4" allowed-ips 192.168.241.2/32,fd00::2/128 endpoint [fd00::5:2]:5
+	n2 wg set wg1 listen-port 5 private-key <(echo "$key4") peer "$pub3" allowed-ips 192.168.241.1/32,fd00::1/128 endpoint [fd00::5:1]:5
+	do_tests
+	# Try to set up a routing loop between the two namespaces
+	ip1 link set netns $netns0 dev wg1
+	ip0 addr add 192.168.241.1/24 dev wg1
+	ip0 link set up dev wg1
+	n0 ping -W 1 -c 1 192.168.241.2
+	n1 wg set wg0 peer "$pub2" endpoint 192.168.241.2:7
+	ip2 link del wg0
+	ip2 link del wg1
+	read _ _ tx_bytes_before < <(n0 wg show wg1 transfer)
+	! n0 ping -W 1 -c 10 -f 192.168.241.2 || false
+	sleep 1
+	read _ _ tx_bytes_after < <(n0 wg show wg1 transfer)
+	(( tx_bytes_after - tx_bytes_before < 70000 ))
+
+	ip0 link del wg1
+	ip1 link del wg0
+
+	# re-add wg0 for later tests
+	ip1 link add dev wg0 type wireguard
+	ip2 link add dev wg0 type wireguard
+}
+
+################################################################################
+# NAT test
+# Test using NAT. We now change the topology to this:
+# ??????????????????????????????????????????    ??????????????????????????????????????????????????     ??????????????????????????????????????????
+# ?             $ns1 namespace             ?    ?                 $ns0 namespace                 ?     ?             $ns2 namespace             ?
+# ?                                        ?    ?                                                ?     ?                                        ?
+# ?  ???????             ???????           ?    ?    ????????              ????????              ?     ?  ???????            ???????            ?
+# ?  ? wg0 ???????????????vethc???????????????????????vethrc?              ?vethrs?????????????????????????veths?????????????? wg0 ?            ?
+# ?  ??????????????????  ???????????????????    ?    ??????????????????    ????????????????????? ?     ?  ?????????????????? ?????????????????? ?
+# ?  ?192.168.241.1/24?  ?192.168.1.100/24??    ?    ?192.168.1.1/24  ?    ?10.0.0.1/24        ? ?     ?  ?10.0.0.100/24   ? ?192.168.241.2/24? ?
+# ?  ?fd00::1/112     ?  ?                ??    ?    ?                ?    ?SNAT:192.168.1.0/24? ?     ?  ?                ? ?fd00::2/112     ? ?
+# ?  ??????????????????  ???????????????????    ?    ??????????????????    ????????????????????? ?     ?  ?????????????????? ?????????????????? ?
+# ??????????????????????????????????????????    ??????????????????????????????????????????????????     ??????????????????????????????????????????
+
+nat_test()
+{
+	$has_basic_topo || { setup_basic_topo && has_basic_topo=true; }
+	configure_peers
+
+	ip0 link add vethrc type veth peer name vethc
+	ip0 link add vethrs type veth peer name veths
+	ip0 link set vethc netns $netns1
+	ip0 link set veths netns $netns2
+	ip0 link set vethrc up
+	ip0 link set vethrs up
+	ip0 addr add 192.168.1.1/24 dev vethrc
+	ip0 addr add 10.0.0.1/24 dev vethrs
+	ip1 addr add 192.168.1.100/24 dev vethc
+	ip1 link set vethc up
+	ip1 route add default via 192.168.1.1
+	ip2 addr add 10.0.0.100/24 dev veths
+	ip2 link set veths up
+	waitiface $netns0 vethrc
+	waitiface $netns0 vethrs
+	waitiface $netns1 vethc
+	waitiface $netns2 veths
+
+	n0 bash -c 'printf 1 > /proc/sys/net/ipv4/ip_forward'
+	n0 bash -c 'printf 2 > /proc/sys/net/netfilter/nf_conntrack_udp_timeout'
+	n0 bash -c 'printf 2 > /proc/sys/net/netfilter/nf_conntrack_udp_timeout_stream'
+	n0 iptables -t nat -A POSTROUTING -s 192.168.1.0/24 -d 10.0.0.0/24 -j SNAT --to 10.0.0.1
+
+	n1 wg set wg0 peer "$pub2" endpoint 10.0.0.100:2 persistent-keepalive 1
+	n1 ping -W 1 -c 1 192.168.241.2
+	n2 ping -W 1 -c 1 192.168.241.1
+	[[ $(n2 wg show wg0 endpoints) == "$pub1	10.0.0.1:1" ]]
+	# Demonstrate n2 can still send packets to n1, since persistent-keepalive will prevent connection tracking entry from expiring (to see entries: `n0 conntrack -L`).
+	pp sleep 3
+	n2 ping -W 1 -c 1 192.168.241.1
+	n1 wg set wg0 peer "$pub2" persistent-keepalive 0
+
+	# Test that sk_bound_dev_if works
+	n1 ping -I wg0 -c 1 -W 1 192.168.241.2
+	# What about when the mark changes and the packet must be rerouted?
+	n1 iptables -t mangle -I OUTPUT -j MARK --set-xmark 1
+	n1 ping -c 1 -W 1 192.168.241.2 # First the boring case
+	n1 ping -I wg0 -c 1 -W 1 192.168.241.2 # Then the sk_bound_dev_if case
+	n1 iptables -t mangle -D OUTPUT -j MARK --set-xmark 1
+
+	# Test that onion routing works, even when it loops
+	n1 wg set wg0 peer "$pub3" allowed-ips 192.168.242.2/32 endpoint 192.168.241.2:5
+	ip1 addr add 192.168.242.1/24 dev wg0
+	ip2 link add wg1 type wireguard
+	ip2 addr add 192.168.242.2/24 dev wg1
+	n2 wg set wg1 private-key <(echo "$key3") listen-port 5 peer "$pub1" allowed-ips 192.168.242.1/32
+	ip2 link set wg1 up
+	n1 ping -W 1 -c 1 192.168.242.2
+	ip2 link del wg1
+	n1 wg set wg0 peer "$pub3" endpoint 192.168.242.2:5
+	! n1 ping -W 1 -c 1 192.168.242.2 || false # Should not crash kernel
+	n1 wg set wg0 peer "$pub3" remove
+	ip1 addr del 192.168.242.1/24 dev wg0
+
+	# Do a wg-quick(8)-style policy routing for the default route, making sure vethc has a v6 address to tease out bugs.
+	ip1 -6 addr add fc00::9/96 dev vethc
+	ip1 -6 route add default via fc00::1
+	ip2 -4 addr add 192.168.99.7/32 dev wg0
+	ip2 -6 addr add abab::1111/128 dev wg0
+	n1 wg set wg0 fwmark 51820 peer "$pub2" allowed-ips 192.168.99.7,abab::1111
+	ip1 -6 route add default dev wg0 table 51820
+	ip1 -6 rule add not fwmark 51820 table 51820
+	ip1 -6 rule add table main suppress_prefixlength 0
+	ip1 -4 route add default dev wg0 table 51820
+	ip1 -4 rule add not fwmark 51820 table 51820
+	ip1 -4 rule add table main suppress_prefixlength 0
+	n1 bash -c 'printf 0 > /proc/sys/net/ipv4/conf/vethc/rp_filter'
+	# Flood the pings instead of sending just one, to trigger routing table reference counting bugs.
+	n1 ping -W 1 -c 100 -f 192.168.99.7
+	n1 ping -W 1 -c 100 -f abab::1111
+
+	# Have ns2 NAT into wg0 packets from ns0, but return an icmp error along the right route.
+	n2 iptables -t nat -A POSTROUTING -s 10.0.0.0/24 -d 192.168.241.0/24 -j SNAT --to 192.168.241.2
+	n0 iptables -t filter -A INPUT \! -s 10.0.0.0/24 -i vethrs -j DROP # Manual rpfilter just to be explicit.
+	n2 bash -c 'printf 1 > /proc/sys/net/ipv4/ip_forward'
+	ip0 -4 route add 192.168.241.1 via 10.0.0.100
+	n2 wg set wg0 peer "$pub1" remove
+	[[ $(! n0 ping -W 1 -c 1 192.168.241.1 || false) == *"From 10.0.0.100 icmp_seq=1 Destination Host Unreachable"* ]]
+
+	n0 iptables -t nat -F
+	n0 iptables -t filter -F
+	n2 iptables -t nat -F
+	ip0 link del vethrc
+	ip0 link del vethrs
+	ip1 link del wg0
+	ip2 link del wg0
+
+	# re-add wg0 for later tests
+	ip1 link add dev wg0 type wireguard
+	ip2 link add dev wg0 type wireguard
+}
+
+################################################################################
+# saddr test
+# Test that saddr routing is sticky but not too sticky, changing to this topology:
+# ??????????????????????????????????????????    ??????????????????????????????????????????
+# ?             $ns1 namespace             ?    ?             $ns2 namespace             ?
+# ?                                        ?    ?                                        ?
+# ?  ???????             ???????           ?    ?  ???????            ???????            ?
+# ?  ? wg0 ???????????????veth1?????????????????????veth2?????????????? wg0 ?            ?
+# ?  ??????????????????  ???????????????????    ?  ?????????????????? ?????????????????? ?
+# ?  ?192.168.241.1/24?  ?10.0.0.1/24     ??    ?  ?10.0.0.2/24     ? ?192.168.241.2/24? ?
+# ?  ?fd00::1/112     ?  ?fd00:aa::1/96   ??    ?  ?fd00:aa::2/96   ? ?fd00::2/112     ? ?
+# ?  ??????????????????  ???????????????????    ?  ?????????????????? ?????????????????? ?
+# ??????????????????????????????????????????    ??????????????????????????????????????????
+
+saddr_test()
+{
+	$has_basic_topo || { setup_basic_topo && has_basic_topo=true; }
+	configure_peers
+
+	ip1 link add veth1 type veth peer name veth2
+	ip1 link set veth2 netns $netns2
+	n1 bash -c 'printf 0 > /proc/sys/net/ipv6/conf/all/accept_dad'
+	n2 bash -c 'printf 0 > /proc/sys/net/ipv6/conf/all/accept_dad'
+	n1 bash -c 'printf 0 > /proc/sys/net/ipv6/conf/veth1/accept_dad'
+	n2 bash -c 'printf 0 > /proc/sys/net/ipv6/conf/veth2/accept_dad'
+	n1 bash -c 'printf 1 > /proc/sys/net/ipv4/conf/veth1/promote_secondaries'
+
+	# First we check that we aren't overly sticky and can fall over to new IPs when old ones are removed
+	ip1 addr add 10.0.0.1/24 dev veth1
+	ip1 addr add fd00:aa::1/96 dev veth1
+	ip2 addr add 10.0.0.2/24 dev veth2
+	ip2 addr add fd00:aa::2/96 dev veth2
+	ip1 link set veth1 up
+	ip2 link set veth2 up
+	waitiface $netns1 veth1
+	waitiface $netns2 veth2
+	n1 wg set wg0 peer "$pub2" endpoint 10.0.0.2:2
+	n1 ping -W 1 -c 1 192.168.241.2
+	ip1 addr add 10.0.0.10/24 dev veth1
+	ip1 addr del 10.0.0.1/24 dev veth1
+	n1 ping -W 1 -c 1 192.168.241.2
+	n1 wg set wg0 peer "$pub2" endpoint [fd00:aa::2]:2
+	n1 ping -W 1 -c 1 192.168.241.2
+	ip1 addr add fd00:aa::10/96 dev veth1
+	ip1 addr del fd00:aa::1/96 dev veth1
+	n1 ping -W 1 -c 1 192.168.241.2
+
+	# Now we show that we can successfully do reply to sender routing
+	ip1 link set veth1 down
+	ip2 link set veth2 down
+	ip1 addr flush dev veth1
+	ip2 addr flush dev veth2
+	ip1 addr add 10.0.0.1/24 dev veth1
+	ip1 addr add 10.0.0.2/24 dev veth1
+	ip1 addr add fd00:aa::1/96 dev veth1
+	ip1 addr add fd00:aa::2/96 dev veth1
+	ip2 addr add 10.0.0.3/24 dev veth2
+	ip2 addr add fd00:aa::3/96 dev veth2
+	ip1 link set veth1 up
+	ip2 link set veth2 up
+	waitiface $netns1 veth1
+	waitiface $netns2 veth2
+	n2 wg set wg0 peer "$pub1" endpoint 10.0.0.1:1
+	n2 ping -W 1 -c 1 192.168.241.1
+	[[ $(n2 wg show wg0 endpoints) == "$pub1	10.0.0.1:1" ]]
+	n2 wg set wg0 peer "$pub1" endpoint [fd00:aa::1]:1
+	n2 ping -W 1 -c 1 192.168.241.1
+	[[ $(n2 wg show wg0 endpoints) == "$pub1	[fd00:aa::1]:1" ]]
+	n2 wg set wg0 peer "$pub1" endpoint 10.0.0.2:1
+	n2 ping -W 1 -c 1 192.168.241.1
+	[[ $(n2 wg show wg0 endpoints) == "$pub1	10.0.0.2:1" ]]
+	n2 wg set wg0 peer "$pub1" endpoint [fd00:aa::2]:1
+	n2 ping -W 1 -c 1 192.168.241.1
+	[[ $(n2 wg show wg0 endpoints) == "$pub1	[fd00:aa::2]:1" ]]
+
+	# What happens if the inbound destination address belongs to a different interface as the default route?
+	ip1 link add dummy0 type dummy
+	ip1 addr add 10.50.0.1/24 dev dummy0
+	ip1 link set dummy0 up
+	ip2 route add 10.50.0.0/24 dev veth2
+	n2 wg set wg0 peer "$pub1" endpoint 10.50.0.1:1
+	n2 ping -W 1 -c 1 192.168.241.1
+	[[ $(n2 wg show wg0 endpoints) == "$pub1	10.50.0.1:1" ]]
+
+	ip1 link del dummy0
+	ip1 addr flush dev veth1
+	ip2 addr flush dev veth2
+	ip1 route flush dev veth1
+	ip2 route flush dev veth2
+
+	# Now we see what happens if another interface route takes precedence over an ongoing one
+	ip1 link add veth3 type veth peer name veth4
+	ip1 link set veth4 netns $netns2
+	ip1 addr add 10.0.0.1/24 dev veth1
+	ip2 addr add 10.0.0.2/24 dev veth2
+	ip1 addr add 10.0.0.3/24 dev veth3
+	ip1 link set veth1 up
+	ip2 link set veth2 up
+	ip1 link set veth3 up
+	ip2 link set veth4 up
+	waitiface $netns1 veth1
+	waitiface $netns2 veth2
+	waitiface $netns1 veth3
+	waitiface $netns2 veth4
+	ip1 route flush dev veth1
+	ip1 route flush dev veth3
+	ip1 route add 10.0.0.0/24 dev veth1 src 10.0.0.1 metric 2
+	n1 wg set wg0 peer "$pub2" endpoint 10.0.0.2:2
+	n1 ping -W 1 -c 1 192.168.241.2
+	[[ $(n2 wg show wg0 endpoints) == "$pub1	10.0.0.1:1" ]]
+	ip1 route add 10.0.0.0/24 dev veth3 src 10.0.0.3 metric 1
+	n1 bash -c 'printf 0 > /proc/sys/net/ipv4/conf/veth1/rp_filter'
+	n2 bash -c 'printf 0 > /proc/sys/net/ipv4/conf/veth4/rp_filter'
+	n1 bash -c 'printf 0 > /proc/sys/net/ipv4/conf/all/rp_filter'
+	n2 bash -c 'printf 0 > /proc/sys/net/ipv4/conf/all/rp_filter'
+	n1 ping -W 1 -c 1 192.168.241.2
+	[[ $(n2 wg show wg0 endpoints) == "$pub1	10.0.0.3:1" ]]
+
+	ip1 link del veth1
+	ip1 link del veth3
+	ip1 link del wg0
+	ip2 link del wg0
+
+	# We test that Netlink/IPC is working properly by doing things that usually cause split responses
+	ip0 link add dev wg0 type wireguard
+	config=( "[Interface]" "PrivateKey=$(wg genkey)" "[Peer]" "PublicKey=$(wg genkey)" )
+	for a in {1..255}; do
+		for b in {0..255}; do
+			config+=( "AllowedIPs=$a.$b.0.0/16,$a::$b/128" )
+		done
+	done
+	n0 wg setconf wg0 <(printf '%s\n' "${config[@]}")
+	i=0
+	for ip in $(n0 wg show wg0 allowed-ips); do
+		((++i))
+	done
+	((i == 255*256*2+1))
+	ip0 link del wg0
+	ip0 link add dev wg0 type wireguard
+	config=( "[Interface]" "PrivateKey=$(wg genkey)" )
+	for a in {1..40}; do
+		config+=( "[Peer]" "PublicKey=$(wg genkey)" )
+		for b in {1..52}; do
+			config+=( "AllowedIPs=$a.$b.0.0/16" )
+		done
+	done
+	n0 wg setconf wg0 <(printf '%s\n' "${config[@]}")
+	i=0
+	while read -r line; do
+		j=0
+		for ip in $line; do
+			((++j))
+		done
+		((j == 53))
+		((++i))
+	done < <(n0 wg show wg0 allowed-ips)
+	((i == 40))
+	ip0 link del wg0
+	ip0 link add wg0 type wireguard
+	config=( )
+	for i in {1..29}; do
+		config+=( "[Peer]" "PublicKey=$(wg genkey)" )
+	done
+	config+=( "[Peer]" "PublicKey=$(wg genkey)" "AllowedIPs=255.2.3.4/32,abcd::255/128" )
+	n0 wg setconf wg0 <(printf '%s\n' "${config[@]}")
+	n0 wg showconf wg0 > /dev/null
+	ip0 link del wg0
+
+	allowedips=( )
+	for i in {1..197}; do
+	        allowedips+=( abcd::$i )
+	done
+	saved_ifs="$IFS"
+	IFS=,
+	allowedips="${allowedips[*]}"
+	IFS="$saved_ifs"
+	ip0 link add wg0 type wireguard
+	n0 wg set wg0 peer "$pub1"
+	n0 wg set wg0 peer "$pub2" allowed-ips "$allowedips"
+	{
+		read -r pub allowedips
+		[[ $pub == "$pub1" && $allowedips == "(none)" ]]
+		read -r pub allowedips
+		[[ $pub == "$pub2" ]]
+		i=0
+		for _ in $allowedips; do
+			((++i))
+		done
+		((i == 197))
+	} < <(n0 wg show wg0 allowed-ips)
+	ip0 link del wg0
+
+	! n0 wg show doesnotexist || false
+
+	# add addr on lo for ncat
+	ip0 -4 addr del 127.0.0.2/8 dev lo
+
+	ip0 link add wg0 type wireguard
+	n0 wg set wg0 private-key <(echo "$key1") peer "$pub2" preshared-key <(echo "$psk")
+	[[ $(n0 wg show wg0 private-key) == "$key1" ]]
+	[[ $(n0 wg show wg0 preshared-keys) == "$pub2	$psk" ]]
+	n0 wg set wg0 private-key /dev/null peer "$pub2" preshared-key /dev/null
+	[[ $(n0 wg show wg0 private-key) == "(none)" ]]
+	[[ $(n0 wg show wg0 preshared-keys) == "$pub2	(none)" ]]
+	n0 wg set wg0 peer "$pub2"
+	n0 wg set wg0 private-key <(echo "$key2")
+	[[ $(n0 wg show wg0 public-key) == "$pub2" ]]
+	[[ -z $(n0 wg show wg0 peers) ]]
+	n0 wg set wg0 peer "$pub2"
+	[[ -z $(n0 wg show wg0 peers) ]]
+	n0 wg set wg0 private-key <(echo "$key1")
+	n0 wg set wg0 peer "$pub2"
+	[[ $(n0 wg show wg0 peers) == "$pub2" ]]
+	n0 wg set wg0 private-key <(echo "/${key1:1}")
+	[[ $(n0 wg show wg0 private-key) == "+${key1:1}" ]]
+	n0 wg set wg0 peer "$pub2" allowed-ips 0.0.0.0/0,10.0.0.0/8,100.0.0.0/10,172.16.0.0/12,192.168.0.0/16
+	n0 wg set wg0 peer "$pub2" allowed-ips 0.0.0.0/0
+	n0 wg set wg0 peer "$pub2" allowed-ips ::/0,1700::/111,5000::/4,e000::/37,9000::/75
+	n0 wg set wg0 peer "$pub2" allowed-ips ::/0
+	n0 wg set wg0 peer "$pub2" remove
+	for low_order_point in AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA= AQAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA= 4Ot6fDtBuK4WVuP68Z/EatoJjeucMrH9hmIFFl9JuAA= X5yVvKNQjCSx0LFVnIPvWwREXMRYHI6G2CJO3dCfEVc= 7P///////////////////////////////////////38= 7f///////////////////////////////////////38= 7v///////////////////////////////////////38=; do
+		n0 wg set wg0 peer "$low_order_point" persistent-keepalive 1 endpoint 127.0.0.1:1111
+	done
+	[[ -n $(n0 wg show wg0 peers) ]]
+	exec 4< <(n0 ncat -l -u -p 1111)
+	ncat_pid=$!
+	waitncatudp $netns0 $ncat_pid
+	ip0 link set wg0 up
+	! read -r -n 1 -t 2 <&4 || false
+	kill $ncat_pid
+	ip0 link del wg0
+
+	# re-add wg0 for later tests
+	ip1 link add dev wg0 type wireguard
+	ip2 link add dev wg0 type wireguard
+}
+
+regression_test()
+{
+	$has_basic_topo || { setup_basic_topo && has_basic_topo=true; }
+	configure_peers
+
+	# Ensure that dst_cache references don't outlive netns lifetime
+	ip1 link add veth1 type veth peer name veth2
+	ip1 link set veth2 netns $netns2
+	ip1 addr add fd00:aa::1/64 dev veth1
+	ip2 addr add fd00:aa::2/64 dev veth2
+	ip1 link set veth1 up
+	ip2 link set veth2 up
+	waitiface $netns1 veth1
+	waitiface $netns2 veth2
+	ip1 -6 route add default dev veth1 via fd00:aa::2
+	ip2 -6 route add default dev veth2 via fd00:aa::1
+	n1 wg set wg0 peer "$pub2" endpoint [fd00:aa::2]:2
+	n2 wg set wg0 peer "$pub1" endpoint [fd00:aa::1]:1
+	n1 ping6 -c 1 fd00::2
+	pp ip netns delete $netns1
+	pp ip netns delete $netns2
+	pp ip netns add $netns1
+	pp ip netns add $netns2
+
+	# Ensure there aren't circular reference loops
+	ip1 link add wg1 type wireguard
+	ip2 link add wg2 type wireguard
+	ip1 link set wg1 netns $netns2
+	ip2 link set wg2 netns $netns1
+	pp ip netns delete $netns1
+	pp ip netns delete $netns2
+	pp ip netns add $netns1
+	pp ip netns add $netns2
+
+	sleep 2 # Wait for cleanup and grace periods
+	declare -A objects
+	while read -t 0.1 -r line 2>/dev/null || [[ $? -ne 142 ]]; do
+		[[ $line =~ .*(wg[0-9]+:\ [A-Z][a-z]+\ ?[0-9]*)\ .*(created|destroyed).* ]] || continue
+		objects["${BASH_REMATCH[1]}"]+="${BASH_REMATCH[2]}"
+	done < /dev/kmsg
+	alldeleted=1
+	for object in "${!objects[@]}"; do
+		if [[ ${objects["$object"]} != *createddestroyed && ${objects["$object"]} != *createdcreateddestroyeddestroyed ]]; then
+			echo "Error: $object: merely ${objects["$object"]}" >&3
+			alldeleted=0
+		fi
+	done
+	[[ $alldeleted -eq 1 ]]
+	pretty "" "Objects that were created were also destroyed."
+}
+
+
+################################################################################
+# usage
+
+usage()
+{
+	cat <<EOF
+usage: ${0##*/} OPTS
+
+        -t <test>   Test(s) to run (default: all)
+                    (options: $TESTS)
+        -h          This help message
+EOF
+}
+
+################################################################################
+# main
+
+while getopts t:h o
+do
+	case $o in
+		t) TESTS=$OPTARG;;
+		h) usage; exit 0;;
+		*) usage; exit 1;;
+	esac
+done
+
+trap cleanup EXIT
+
+for t in $TESTS
+do
+	case $t in
+	basic_test|basic)		basic_test;;
+	nat_test|nat)			nat_test;;
+	saddr_test|saddr)		saddr_test;;
+	regression_test|regression)	regression_test;;
+
+	help) echo "Test names: $TESTS"; exit 0;;
+	esac
+done
-- 
2.31.1


From Jason at zx2c4.com  Tue Nov 16 14:35:40 2021
From: Jason at zx2c4.com (Jason A. Donenfeld)
Date: Tue, 16 Nov 2021 15:35:40 +0100
Subject: [PATCH wireguard] wireguard: selftests: refactor the test
 structure
In-Reply-To: <20211116081359.975655-1-liuhangbin@gmail.com>
References: <20211116081359.975655-1-liuhangbin@gmail.com>
Message-ID: <CAHmME9pNFe7grqhW7=YQgRq10g4K5bqVuJrq0HonEVNbQSRuYg@mail.gmail.com>

Hi Hangbin,

I don't know how interested in this I am. Splitting things into two
files means more confusing maintenance, and categorizing sections
strictly into functions means there's more overhead when adding tests
(e.g. "where do they fit?"), because the categories you've chosen are
fairly broad, rather than being functions for each specific test. I'd
be more amenable to something _entirely_ granular, because that'd be
consistent, or what we have now, which is just linear. Full
granularity, though, has its own downsides, of increased clutter.
Alternatively, if you'd like to add some comments around the different
areas to better document what's happening, perhaps that'd accomplish
the same thing as this patch.

Jason

From peter.georg at physik.uni-regensburg.de  Tue Nov 16 16:52:22 2021
From: peter.georg at physik.uni-regensburg.de (Peter Georg)
Date: Tue, 16 Nov 2021 17:52:22 +0100
Subject: [PATCH compat] Update for RHEL 8.5
Message-ID: <20211116165222.3236464-1-peter.georg@physik.uni-regensburg.de>

RHEL 8.5 has been released. Replace all ISCENTOS8S checks with ISRHEL8.
Increase RHEL_MINOR for CentOS 8 Stream detection to 6.

Signed-off-by: Peter Georg <peter.georg at physik.uni-regensburg.de>
---
 src/compat/compat-asm.h | 4 ++--
 src/compat/compat.h     | 4 ++--
 2 files changed, 4 insertions(+), 4 deletions(-)

diff --git a/src/compat/compat-asm.h b/src/compat/compat-asm.h
index 5bfdb94..951fc10 100644
--- a/src/compat/compat-asm.h
+++ b/src/compat/compat-asm.h
@@ -15,7 +15,7 @@
 #define ISRHEL7
 #elif RHEL_MAJOR == 8
 #define ISRHEL8
-#if RHEL_MINOR >= 4
+#if RHEL_MINOR >= 6
 #define ISCENTOS8S
 #endif
 #endif
@@ -51,7 +51,7 @@
 #undef pull
 #endif
 
-#if LINUX_VERSION_CODE < KERNEL_VERSION(5, 4, 76) && !defined(ISCENTOS8S) && !defined(SYM_FUNC_START)
+#if LINUX_VERSION_CODE < KERNEL_VERSION(5, 4, 76) && !defined(ISRHEL8) && !defined(SYM_FUNC_START)
 #define SYM_FUNC_START ENTRY
 #define SYM_FUNC_END ENDPROC
 #endif
diff --git a/src/compat/compat.h b/src/compat/compat.h
index ee45a3c..9aa767a 100644
--- a/src/compat/compat.h
+++ b/src/compat/compat.h
@@ -16,7 +16,7 @@
 #define ISRHEL7
 #elif RHEL_MAJOR == 8
 #define ISRHEL8
-#if RHEL_MINOR >= 5
+#if RHEL_MINOR >= 6
 #define ISCENTOS8S
 #endif
 #endif
@@ -855,7 +855,7 @@ static inline void skb_mark_not_on_list(struct sk_buff *skb)
 #endif
 #endif
 
-#if LINUX_VERSION_CODE < KERNEL_VERSION(5, 5, 0) && !defined(ISCENTOS8S)
+#if LINUX_VERSION_CODE < KERNEL_VERSION(5, 5, 0) && !defined(ISRHEL8)
 #define genl_dumpit_info(cb) ({ \
 	struct { struct nlattr **attrs; } *a = (void *)((u8 *)cb->args + offsetofend(struct dump_ctx, next_allowedip)); \
 	BUILD_BUG_ON(sizeof(cb->args) < offsetofend(struct dump_ctx, next_allowedip) + sizeof(*a)); \
-- 
2.31.1


From Jason at zx2c4.com  Tue Nov 16 17:14:47 2021
From: Jason at zx2c4.com (Jason A. Donenfeld)
Date: Tue, 16 Nov 2021 18:14:47 +0100
Subject: [PATCH compat] Update for RHEL 8.5
In-Reply-To: <20211116165222.3236464-1-peter.georg@physik.uni-regensburg.de>
References: <20211116165222.3236464-1-peter.georg@physik.uni-regensburg.de>
Message-ID: <CAHmME9py4aEC_jJSZs1uQb=9xsS+1C85rMjGAYvGG3A8cTmp0g@mail.gmail.com>

Applied! Thank you.

https://git.zx2c4.com/wireguard-linux-compat/commit/?id=77256108c0edcaee78dc16352e7339fb13ddeb05

From tim at tschumacher.net  Wed Nov 17 20:49:52 2021
From: tim at tschumacher.net (Tim Schumacher)
Date: Wed, 17 Nov 2021 21:49:52 +0100
Subject: wg-quick's Ethernet unplug problem
Message-ID: <fbb34b62-8227-6106-2101-f7f1398b2143@tschumacher.net>

Hi,

there is this annoying problem with the way wg-quick sets up the 
routing, that when you unplug the Ethernet cable the routing rule (see 
ip rule) that checks the fwmark gets lost. So let's say you move to 
another location with your laptop and plug in an Ethernet cable again, 
it essentially disables your VPN. You're lucky if you have the popular 
kill switch in place blocking all your traffic instead of silently 
disabling your VPN.

PostUp = iptables -I OUTPUT ! -o %i -m mark ! --mark $(wg show %i 
fwmark) -m addrtype ! --dst-type LOCAL -j REJECT
PreDown = iptables -D OUTPUT ! -o %i -m mark ! --mark $(wg show %i 
fwmark) -m addrtype ! --dst-type LOCAL -j REJECT

I have no idea why the rule gets lost. It might be the kernel removing 
it for whatever reason. But it makes no sense to me because I don't see 
how the kernel would make a connection between that rule and the 
Ethernet link that goes down. Any ideas on the issue? Is this a known 
problem? Are there workarounds?

Greetings,
Tim


From keescook at chromium.org  Thu Nov 18 18:36:15 2021
From: keescook at chromium.org (Kees Cook)
Date: Thu, 18 Nov 2021 10:36:15 -0800
Subject: [PATCH] skbuff: Switch structure bounds to struct_group()
Message-ID: <20211118183615.1281978-1-keescook@chromium.org>

In preparation for FORTIFY_SOURCE performing compile-time and run-time
field bounds checking for memcpy(), memmove(), and memset(), avoid
intentionally writing across neighboring fields.

Replace the existing empty member position markers "headers_start" and
"headers_end" with a struct_group(). This will allow memcpy() and sizeof()
to more easily reason about sizes, and improve readability.

"pahole" shows no size nor member offset changes to struct sk_buff.
"objdump -d" shows no object code changes (outside of WARNs affected by
source line number changes).

Signed-off-by: Kees Cook <keescook at chromium.org>
Reviewed-by: Gustavo A. R. Silva <gustavoars at kernel.org>
Reviewed-by: Jason A. Donenfeld <Jason at zx2c4.com> # drivers/net/wireguard/*
---
 drivers/net/wireguard/queueing.h |  4 +---
 include/linux/skbuff.h           | 10 +++-------
 net/core/skbuff.c                | 14 +++++---------
 3 files changed, 9 insertions(+), 19 deletions(-)

diff --git a/drivers/net/wireguard/queueing.h b/drivers/net/wireguard/queueing.h
index 4ef2944a68bc..52da5e963003 100644
--- a/drivers/net/wireguard/queueing.h
+++ b/drivers/net/wireguard/queueing.h
@@ -79,9 +79,7 @@ static inline void wg_reset_packet(struct sk_buff *skb, bool encapsulating)
 	u8 sw_hash = skb->sw_hash;
 	u32 hash = skb->hash;
 	skb_scrub_packet(skb, true);
-	memset(&skb->headers_start, 0,
-	       offsetof(struct sk_buff, headers_end) -
-		       offsetof(struct sk_buff, headers_start));
+	memset(&skb->headers, 0, sizeof(skb->headers));
 	if (encapsulating) {
 		skb->l4_hash = l4_hash;
 		skb->sw_hash = sw_hash;
diff --git a/include/linux/skbuff.h b/include/linux/skbuff.h
index 686a666d073d..875adfd056b6 100644
--- a/include/linux/skbuff.h
+++ b/include/linux/skbuff.h
@@ -808,12 +808,10 @@ struct sk_buff {
 	__u8			active_extensions;
 #endif
 
-	/* fields enclosed in headers_start/headers_end are copied
+	/* Fields enclosed in headers group are copied
 	 * using a single memcpy() in __copy_skb_header()
 	 */
-	/* private: */
-	__u32			headers_start[0];
-	/* public: */
+	struct_group(headers,
 
 /* if you move pkt_type around you also must adapt those constants */
 #ifdef __BIG_ENDIAN_BITFIELD
@@ -932,9 +930,7 @@ struct sk_buff {
 	u64			kcov_handle;
 #endif
 
-	/* private: */
-	__u32			headers_end[0];
-	/* public: */
+	); /* end headers group */
 
 	/* These elements must be at the end, see alloc_skb() for details.  */
 	sk_buff_data_t		tail;
diff --git a/net/core/skbuff.c b/net/core/skbuff.c
index ba2f38246f07..3a42b2a3a571 100644
--- a/net/core/skbuff.c
+++ b/net/core/skbuff.c
@@ -992,12 +992,10 @@ void napi_consume_skb(struct sk_buff *skb, int budget)
 }
 EXPORT_SYMBOL(napi_consume_skb);
 
-/* Make sure a field is enclosed inside headers_start/headers_end section */
+/* Make sure a field is contained by headers group */
 #define CHECK_SKB_FIELD(field) \
-	BUILD_BUG_ON(offsetof(struct sk_buff, field) <		\
-		     offsetof(struct sk_buff, headers_start));	\
-	BUILD_BUG_ON(offsetof(struct sk_buff, field) >		\
-		     offsetof(struct sk_buff, headers_end));	\
+	BUILD_BUG_ON(offsetof(struct sk_buff, field) !=		\
+		     offsetof(struct sk_buff, headers.field));	\
 
 static void __copy_skb_header(struct sk_buff *new, const struct sk_buff *old)
 {
@@ -1009,14 +1007,12 @@ static void __copy_skb_header(struct sk_buff *new, const struct sk_buff *old)
 	__skb_ext_copy(new, old);
 	__nf_copy(new, old, false);
 
-	/* Note : this field could be in headers_start/headers_end section
+	/* Note : this field could be in the headers group.
 	 * It is not yet because we do not want to have a 16 bit hole
 	 */
 	new->queue_mapping = old->queue_mapping;
 
-	memcpy(&new->headers_start, &old->headers_start,
-	       offsetof(struct sk_buff, headers_end) -
-	       offsetof(struct sk_buff, headers_start));
+	memcpy(&new->headers, &old->headers, sizeof(new->headers));
 	CHECK_SKB_FIELD(protocol);
 	CHECK_SKB_FIELD(csum);
 	CHECK_SKB_FIELD(hash);
-- 
2.30.2


From wireguard-mail at chil.at  Thu Nov 18 23:40:10 2021
From: wireguard-mail at chil.at (Christoph Loesch)
Date: Fri, 19 Nov 2021 00:40:10 +0100
Subject: client uses wrong source ip for outgoing connections
Message-ID: <d60ce240-5bb3-eed7-85a0-31d05cb0133e@chil.at>

Hi,

I am using wireguard on about 20 EdgeRouters (based on Debian stretch).
Each router has exact same configuration (apart from router ip addresses and wireguard keys/passphrases).
Works very well on most of them but on five routers wireguard uses the wrong ip address for outgoing connections over the tunnel.
All routers use kernel 4.14.54-UBNT and wireguard-tools v1.0.20210914
Wireguard debian package is from github/WireGuard/wireguard-vyatta-ubnt

On the problematic routers the public ip address is used for the tunnel instead the private ip address.
Interestingly even in the bad example the wg tunnel is running and the server can reach the routers(=wg clients), but not the other way round.

In the following examples 172.27.0.1 is the wireguard server internal ip address.
Routers use ip addresses in the 10.0.0.0/8 range for the wg tunnel which are allowed on the server.
I already even debugged this with tcpdump where I found out it uses the wrong ip.
But looking at a simple ping you also notice the wrong ip after the word "from".

Good example:
eng196-router:~$ \ping -I wg0 -c1 172.27.0.1
ping: Warning: source address might be selected on device other than wg0.
PING 172.27.0.1 (172.27.0.1) from 10.29.85.100 wg0: 56(84) bytes of data.
64 bytes from 172.27.0.1: icmp_seq=1 ttl=64 time=6.82 ms
--- 172.27.0.1 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 6.826/6.826/6.826/0.000 ms

Bad example:
zi1-router:~$ \ping -I wg0 -c1 172.27.0.1
ping: Warning: source address might be selected on device other than wg0.
PING 172.27.0.1 (172.27.0.1) from 78.41.x.y wg0: 56(84) bytes of data.
--- 172.27.0.1 ping statistics ---
1 packets transmitted, 0 received, 100% packet loss, time 0ms

Configurations:
eng196-router:~# wg
interface: wg0
 ? public key: SoV2obcH0qWfCRY3gZbkLNeMa1QRcnhNDCeiI9weszA=
 ? private key: (hidden)
 ? listening port: 58205
peer: 1syRMYD1jIVFMUMm5hF/j0MzjMQmuC5mlcT1VVugIkU=
 ? preshared key: (hidden)
 ? endpoint: 86.59.x.y:1024
 ? allowed ips: 172.27.0.0/24, 10.5.44.0/24
 ? latest handshake: 53 seconds ago
 ? transfer: 24.57 MiB received, 26.48 MiB sent
 ? persistent keepalive: every 25 seconds

zi1-router:~# wg
interface: wg0
 ? public key: aYtVhblpR0XSsAb/dXF3zM9Hu+LxlvrR5RWFU2psF3M=
 ? private key: (hidden)
 ? listening port: 45514
peer: 1syRMYD1jIVFMUMm5hF/j0MzjMQmuC5mlcT1VVugIkU=
 ? preshared key: (hidden)
 ? endpoint: 86.59.x.y:51820
 ? allowed ips: 172.27.0.0/24, 10.5.44.0/24
 ? latest handshake: 13 seconds ago
 ? transfer: 1.79 MiB received, 6.26 MiB sent
 ? persistent keepalive: every 25 seconds

What could cause the wrong selection?
Why does that work for most routers but for some not? There must be some difference or something gets confused up by specific ip addresses I guess?
How could I debug this further to find the difference and/or cause for this problem?

Thanks for any hints and kind regards,
Christoph


From wireguard-mail at chil.at  Fri Nov 19 00:11:47 2021
From: wireguard-mail at chil.at (Christoph Loesch)
Date: Fri, 19 Nov 2021 01:11:47 +0100
Subject: client uses wrong source ip for outgoing connections
In-Reply-To: <d60ce240-5bb3-eed7-85a0-31d05cb0133e@chil.at>
References: <d60ce240-5bb3-eed7-85a0-31d05cb0133e@chil.at>
Message-ID: <744d7291-e43b-4e8c-76be-c78c11204e17@chil.at>

if relevant, some more details about interface and routes from good and bad example to compare:

root at eng196-router:~# ip a sh wg0
46: wg0: <POINTOPOINT,NOARP,UP,LOWER_UP> mtu 1420 qdisc noqueue state UNKNOWN group default qlen 1000
 ??? link/none
root at eng196-router:~# ip r sh dev wg0
10.5.44.0/24 scope link
172.27.0.0/24 scope link
root at eng196-router:~# ip a sh br1
11: br1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
 ??? link/ether 44:d9:e7:x:y:z brd ff:ff:ff:ff:ff:ff
 ??? inet 10.29.85.100/24 brd 10.29.85.255 scope global br1
 ?????? valid_lft forever preferred_lft forever
 ??? inet6 fe80::7c4c:1dff:fe84:fece/64 scope link
 ?????? valid_lft forever preferred_lft forever

root at zi1-router:~# ip a sh wg0
18: wg0: <POINTOPOINT,NOARP,UP,LOWER_UP> mtu 1420 qdisc noqueue state UNKNOWN group default qlen 1000
 ??? link/none
root at zi1-router:~# ip r sh dev wg0
10.5.44.0/24 scope link
172.27.0.0/24 scope link
root at zi1-router:~# ip a sh br1
12: br1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
 ??? link/ether 74:83:c2:x:y:z brd ff:ff:ff:ff:ff:ff
 ??? inet 10.34.0.100/24 brd 10.34.0.255 scope global br1
 ?????? valid_lft forever preferred_lft forever
 ??? inet6 fe80::2c2e:76ff:fedc:d8e/64 scope link
 ?????? valid_lft forever preferred_lft forever

Am 19.11.2021 um 00:40 schrieb Christoph Loesch:
> Hi,
>
> I am using wireguard on about 20 EdgeRouters (based on Debian stretch).
> Each router has exact same configuration (apart from router ip addresses and wireguard keys/passphrases).
> Works very well on most of them but on five routers wireguard uses the wrong ip address for outgoing connections over the tunnel.
> All routers use kernel 4.14.54-UBNT and wireguard-tools v1.0.20210914
> Wireguard debian package is from github/WireGuard/wireguard-vyatta-ubnt
>
> On the problematic routers the public ip address is used for the tunnel instead the private ip address.
> Interestingly even in the bad example the wg tunnel is running and the server can reach the routers(=wg clients), but not the other way round.
>
> In the following examples 172.27.0.1 is the wireguard server internal ip address.
> Routers use ip addresses in the 10.0.0.0/8 range for the wg tunnel which are allowed on the server.
> I already even debugged this with tcpdump where I found out it uses the wrong ip.
> But looking at a simple ping you also notice the wrong ip after the word "from".
>
> Good example:
> eng196-router:~$ \ping -I wg0 -c1 172.27.0.1
> ping: Warning: source address might be selected on device other than wg0.
> PING 172.27.0.1 (172.27.0.1) from 10.29.85.100 wg0: 56(84) bytes of data.
> 64 bytes from 172.27.0.1: icmp_seq=1 ttl=64 time=6.82 ms
> --- 172.27.0.1 ping statistics ---
> 1 packets transmitted, 1 received, 0% packet loss, time 0ms
> rtt min/avg/max/mdev = 6.826/6.826/6.826/0.000 ms
>
> Bad example:
> zi1-router:~$ \ping -I wg0 -c1 172.27.0.1
> ping: Warning: source address might be selected on device other than wg0.
> PING 172.27.0.1 (172.27.0.1) from 78.41.x.y wg0: 56(84) bytes of data.
> --- 172.27.0.1 ping statistics ---
> 1 packets transmitted, 0 received, 100% packet loss, time 0ms
>
> Configurations:
> eng196-router:~# wg
> interface: wg0
> ? public key: SoV2obcH0qWfCRY3gZbkLNeMa1QRcnhNDCeiI9weszA=
> ? private key: (hidden)
> ? listening port: 58205
> peer: 1syRMYD1jIVFMUMm5hF/j0MzjMQmuC5mlcT1VVugIkU=
> ? preshared key: (hidden)
> ? endpoint: 86.59.x.y:1024
> ? allowed ips: 172.27.0.0/24, 10.5.44.0/24
> ? latest handshake: 53 seconds ago
> ? transfer: 24.57 MiB received, 26.48 MiB sent
> ? persistent keepalive: every 25 seconds
>
> zi1-router:~# wg
> interface: wg0
> ? public key: aYtVhblpR0XSsAb/dXF3zM9Hu+LxlvrR5RWFU2psF3M=
> ? private key: (hidden)
> ? listening port: 45514
> peer: 1syRMYD1jIVFMUMm5hF/j0MzjMQmuC5mlcT1VVugIkU=
> ? preshared key: (hidden)
> ? endpoint: 86.59.x.y:51820
> ? allowed ips: 172.27.0.0/24, 10.5.44.0/24
> ? latest handshake: 13 seconds ago
> ? transfer: 1.79 MiB received, 6.26 MiB sent
> ? persistent keepalive: every 25 seconds
>
> What could cause the wrong selection?
> Why does that work for most routers but for some not? There must be some difference or something gets confused up by specific ip addresses I guess?
> How could I debug this further to find the difference and/or cause for this problem?
>
> Thanks for any hints and kind regards,
> Christoph
>

From kuba at kernel.org  Fri Nov 19 07:13:55 2021
From: kuba at kernel.org (Jakub Kicinski)
Date: Thu, 18 Nov 2021 23:13:55 -0800
Subject: [PATCH] skbuff: Switch structure bounds to struct_group()
In-Reply-To: <20211118183615.1281978-1-keescook@chromium.org>
References: <20211118183615.1281978-1-keescook@chromium.org>
Message-ID: <20211118231355.7a39d22f@kicinski-fedora-pc1c0hjn.dhcp.thefacebook.com>

On Thu, 18 Nov 2021 10:36:15 -0800 Kees Cook wrote:
> In preparation for FORTIFY_SOURCE performing compile-time and run-time
> field bounds checking for memcpy(), memmove(), and memset(), avoid
> intentionally writing across neighboring fields.
> 
> Replace the existing empty member position markers "headers_start" and
> "headers_end" with a struct_group(). This will allow memcpy() and sizeof()
> to more easily reason about sizes, and improve readability.
> 
> "pahole" shows no size nor member offset changes to struct sk_buff.
> "objdump -d" shows no object code changes (outside of WARNs affected by
> source line number changes).

This adds ~27k of these warnings to W=1 gcc builds:

include/linux/skbuff.h:851:1: warning: directive in macro's argument list

From cao88yu at gmail.com  Fri Nov 19 13:10:42 2021
From: cao88yu at gmail.com (=?UTF-8?B?5pu554Wc?=)
Date: Fri, 19 Nov 2021 21:10:42 +0800
Subject: Wireguard source ip selection issue with multi interfaces
Message-ID: <CACu-5+0Z1JPCttbZpM13870N9Wce8K3V2ZRdhV6XFmVk4FJKxQ@mail.gmail.com>

Hi,
As Jason said in mailing list
(https://lists.zx2c4.com/pipermail/wireguard/2017-November/002018.html)
years ago, if wireguard reply the client with wrong src ip, then it is
a bug. And now I'm trying to set up a minimal configuration.

Server info:

Ubuntu 20.04 in VMware with two Host-only network adapters:

wacke at Ubuntu:/etc$ lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description: Ubuntu 20.04.3 LTS
Release: 20.04
Codename: focal

wacke at Ubuntu:/etc$ uname -a
Linux Ubuntu 5.14.8-051408-generic #202109280455 SMP Tue Sep 28
04:57:31 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux

wacke at Ubuntu:/etc$ sudo ip route show
169.254.0.0/16 dev ens33 scope link metric 1000
172.17.0.0/16 dev docker0 proto kernel scope link src 172.17.0.1 linkdown
192.168.187.0/24 dev ens33 proto kernel scope link src 192.168.187.129
metric 100
192.168.187.0/24 dev ens35 proto kernel scope link src 192.168.187.128
metric 101
192.168.199.0/24 dev wg0 proto kernel scope link src 192.168.199.1

wacke at Ubuntu:/etc$ sudo cat /etc/wireguard/wg0.conf
[Interface]
PrivateKey = oLLeTlvwWsfGoyQpDwZo0k7AsMf0LkkfycGPwFjamEA=
ListenPort = 5999

[Peer]
PublicKey = JYvK2/FBbbSrDbkH29ejWejkBJS3ch/SFtyZwNf5O1c=
AllowedIPs = 192.168.199.2/32
PersistentKeepalive = 25

[Peer]
PublicKey = keIPXZFgvB79biV74kGc5R9vAHpzyVyQHci8KBSDIUw=
AllowedIPs = 192.168.199.3/32
PersistentKeepalive = 25

wacke at Ubuntu:/etc$ sudo wg setconf wg0 /etc/wireguard/wg0.conf
wacke at Ubuntu:/etc$ sudo ip address add dev wg0 192.168.199.1/24
wacke at Ubuntu:/etc$ sudo ip link set up dev wg0
wacke at Ubuntu:/etc$ sudo wg show
interface: wg0
  public key: Vkdx0MhQkc9anLaUehR/Df1zuKjxceflxMCAeAaCWCo=
  private key: (hidden)
  listening port: 5999

peer: JYvK2/FBbbSrDbkH29ejWejkBJS3ch/SFtyZwNf5O1c=
  endpoint: 192.168.187.1:35560
  allowed ips: 192.168.199.2/32
  transfer: 2.89 KiB received, 1.80 KiB sent
  persistent keepalive: every 25 seconds

peer: keIPXZFgvB79biV74kGc5R9vAHpzyVyQHci8KBSDIUw=
  allowed ips: 192.168.199.3/32
  persistent keepalive: every 25 seconds

Client info:

HOME-Server:/NAS/Software/Software # uname -a
Linux HOME-Server 5.14.14-2-default #1 SMP Thu Oct 21 05:05:03 UTC
2021 (2b5383f) x86_64 x86_64 x86_64 GNU/Linux

HOME-Server:/NAS/Software/Software # cat wg0.conf
[Interface]
PrivateKey = 4Pw1cetxd9TdfH3TSab+9UcRBlHwZ1o/vmrUgkerZls=

[Peer]
PublicKey = Vkdx0MhQkc9anLaUehR/Df1zuKjxceflxMCAeAaCWCo=
Endpoint = 192.168.187.128:5999
AllowedIPs = 192.168.199.0/24
PersistentKeepalive = 25

HOME-Server:/NAS/Software/Software # ip link add wg0 type wireguard
HOME-Server:/NAS/Software/Software # ip address add dev wg0 192.168.199.2/24
HOME-Server:/NAS/Software/Software # wg setconf wg0 wg0.conf
HOME-Server:/NAS/Software/Software # ip link set wg0 up
HOME-Server:/NAS/Software/Software # wg show
interface: wg0
  public key: JYvK2/FBbbSrDbkH29ejWejkBJS3ch/SFtyZwNf5O1c=
  private key: (hidden)
  listening port: 53058

peer: Vkdx0MhQkc9anLaUehR/Df1zuKjxceflxMCAeAaCWCo=
  endpoint: 192.168.187.128:5999
  allowed ips: 192.168.199.0/24
  transfer: 0 B received, 59.26 KiB sent
  persistent keepalive: every 25 seconds


tcpdump of client side:

HOME-Server:/NAS/Software/Software # tcpdump -i any -vn "(dst port 5999)"
tcpdump: data link type LINUX_SLL2
tcpdump: listening on any, link-type LINUX_SLL2 (Linux cooked v2),
snapshot length 262144 bytes
22:10:37.794301 vmnet1 Out IP (tos 0x88, ttl 64, id 54724, offset 0,
flags [none], proto UDP (17), length 176)
    192.168.187.1.53058 > 192.168.187.128.5999: UDP, length 148
22:10:43.170306 vmnet1 Out IP (tos 0x88, ttl 64, id 55327, offset 0,
flags [none], proto UDP (17), length 176)
    192.168.187.1.53058 > 192.168.187.128.5999: UDP, length 148
22:10:48.546281 vmnet1 Out IP (tos 0x88, ttl 64, id 56142, offset 0,
flags [none], proto UDP (17), length 176)
    192.168.187.1.53058 > 192.168.187.128.5999: UDP, length 148
22:10:53.670343 vmnet1 Out IP (tos 0x88, ttl 64, id 57053, offset 0,
flags [none], proto UDP (17), length 176)
    192.168.187.1.53058 > 192.168.187.128.5999: UDP, length 148
22:11:18.754308 vmnet1 Out IP (tos 0x88, ttl 64, id 58505, offset 0,
flags [none], proto UDP (17), length 176)
    192.168.187.1.53058 > 192.168.187.128.5999: UDP, length 148
22:11:24.130286 vmnet1 Out IP (tos 0x88, ttl 64, id 58621, offset 0,
flags [none], proto UDP (17), length 176)
    192.168.187.1.53058 > 192.168.187.128.5999: UDP, length 148
22:11:29.506305 vmnet1 Out IP (tos 0x88, ttl 64, id 58810, offset 0,
flags [none], proto UDP (17), length 176)
    192.168.187.1.53058 > 192.168.187.128.5999: UDP, length 148
22:11:34.882326 vmnet1 Out IP (tos 0x88, ttl 64, id 59444, offset 0,
flags [none], proto UDP (17), length 176)
    192.168.187.1.53058 > 192.168.187.128.5999: UDP, length 148
^C
8 packets captured
8 packets received by filter
0 packets dropped by kernel

tcpdump of server side:

wacke at Ubuntu:~$ sudo tcpdump -i any -vn "(src port 5999)"
tcpdump: listening on any, link-type LINUX_SLL (Linux cooked v1),
capture size 262144 bytes
22:10:11.754862 IP (tos 0x88, ttl 64, id 22024, offset 0, flags
[none], proto UDP (17), length 120)
    192.168.187.129.5999 > 192.168.187.1.53058: UDP, length 92
22:10:16.874400 IP (tos 0x88, ttl 64, id 22399, offset 0, flags
[none], proto UDP (17), length 120)
    192.168.187.129.5999 > 192.168.187.1.53058: UDP, length 92
22:10:21.994433 IP (tos 0x88, ttl 64, id 23057, offset 0, flags
[none], proto UDP (17), length 120)
    192.168.187.129.5999 > 192.168.187.1.53058: UDP, length 92
22:10:27.370558 IP (tos 0x88, ttl 64, id 23912, offset 0, flags
[none], proto UDP (17), length 120)
    192.168.187.129.5999 > 192.168.187.1.53058: UDP, length 92
22:10:32.494605 IP (tos 0x88, ttl 64, id 24944, offset 0, flags
[none], proto UDP (17), length 120)
    192.168.187.129.5999 > 192.168.187.1.53058: UDP, length 92
22:10:37.610779 IP (tos 0x88, ttl 64, id 25327, offset 0, flags
[none], proto UDP (17), length 120)
    192.168.187.129.5999 > 192.168.187.1.53058: UDP, length 92
22:10:42.986834 IP (tos 0x88, ttl 64, id 25498, offset 0, flags
[none], proto UDP (17), length 120)
    192.168.187.129.5999 > 192.168.187.1.53058: UDP, length 92
22:10:48.362908 IP (tos 0x88, ttl 64, id 26521, offset 0, flags
[none], proto UDP (17), length 120)
    192.168.187.129.5999 > 192.168.187.1.53058: UDP, length 92
22:10:53.486942 IP (tos 0x88, ttl 64, id 27214, offset 0, flags
[none], proto UDP (17), length 120)
    192.168.187.129.5999 > 192.168.187.1.53058: UDP, length 92
22:11:18.571437 IP (tos 0x88, ttl 64, id 31112, offset 0, flags
[none], proto UDP (17), length 120)
    192.168.187.129.5999 > 192.168.187.1.53058: UDP, length 92
22:11:23.947460 IP (tos 0x88, ttl 64, id 31897, offset 0, flags
[none], proto UDP (17), length 120)
    192.168.187.129.5999 > 192.168.187.1.53058: UDP, length 92
22:11:29.323608 IP (tos 0x88, ttl 64, id 32715, offset 0, flags
[none], proto UDP (17), length 120)
    192.168.187.129.5999 > 192.168.187.1.53058: UDP, length 92
22:11:34.699676 IP (tos 0x88, ttl 64, id 33840, offset 0, flags
[none], proto UDP (17), length 120)
    192.168.187.129.5999 > 192.168.187.1.53058: UDP, length 92
^C
13 packets captured
13 packets received by filter
0 packets dropped by kernel

As the tcpdump log showed, the wireguard server always chose the ip
with lowest metric as source ip (192.168.187.129) to reply to the
client (while the client tried to connect 192.168.187.128).

Hope the info above can help to debug. Many thanks.

From keescook at chromium.org  Fri Nov 19 16:24:20 2021
From: keescook at chromium.org (Kees Cook)
Date: Fri, 19 Nov 2021 08:24:20 -0800
Subject: [PATCH] skbuff: Switch structure bounds to struct_group()
In-Reply-To: <20211118231355.7a39d22f@kicinski-fedora-pc1c0hjn.dhcp.thefacebook.com>
References: <20211118183615.1281978-1-keescook@chromium.org>
 <20211118231355.7a39d22f@kicinski-fedora-pc1c0hjn.dhcp.thefacebook.com>
Message-ID: <202111190821.8147EF796B@keescook>

On Thu, Nov 18, 2021 at 11:13:55PM -0800, Jakub Kicinski wrote:
> On Thu, 18 Nov 2021 10:36:15 -0800 Kees Cook wrote:
> > In preparation for FORTIFY_SOURCE performing compile-time and run-time
> > field bounds checking for memcpy(), memmove(), and memset(), avoid
> > intentionally writing across neighboring fields.
> > 
> > Replace the existing empty member position markers "headers_start" and
> > "headers_end" with a struct_group(). This will allow memcpy() and sizeof()
> > to more easily reason about sizes, and improve readability.
> > 
> > "pahole" shows no size nor member offset changes to struct sk_buff.
> > "objdump -d" shows no object code changes (outside of WARNs affected by
> > source line number changes).
> 
> This adds ~27k of these warnings to W=1 gcc builds:
> 
> include/linux/skbuff.h:851:1: warning: directive in macro's argument list

Oh my, I see it[1]. I will get that fixed. This smells like a missing
header or something weird. I have a dim memory of fixing this warning
long ago when evolving this series.

Thanks!

-Kees

[1] https://patchwork.kernel.org/project/netdevbpf/patch/20211118183615.1281978-1-keescook at chromium.org/

-- 
Kees Cook

From keescook at chromium.org  Fri Nov 19 18:26:19 2021
From: keescook at chromium.org (Kees Cook)
Date: Fri, 19 Nov 2021 10:26:19 -0800
Subject: [PATCH] skbuff: Switch structure bounds to struct_group()
In-Reply-To: <20211118231355.7a39d22f@kicinski-fedora-pc1c0hjn.dhcp.thefacebook.com>
References: <20211118183615.1281978-1-keescook@chromium.org>
 <20211118231355.7a39d22f@kicinski-fedora-pc1c0hjn.dhcp.thefacebook.com>
Message-ID: <202111191015.509A0BD@keescook>

On Thu, Nov 18, 2021 at 11:13:55PM -0800, Jakub Kicinski wrote:
> On Thu, 18 Nov 2021 10:36:15 -0800 Kees Cook wrote:
> > In preparation for FORTIFY_SOURCE performing compile-time and run-time
> > field bounds checking for memcpy(), memmove(), and memset(), avoid
> > intentionally writing across neighboring fields.
> > 
> > Replace the existing empty member position markers "headers_start" and
> > "headers_end" with a struct_group(). This will allow memcpy() and sizeof()
> > to more easily reason about sizes, and improve readability.
> > 
> > "pahole" shows no size nor member offset changes to struct sk_buff.
> > "objdump -d" shows no object code changes (outside of WARNs affected by
> > source line number changes).
> 
> This adds ~27k of these warnings to W=1 gcc builds:
> 
> include/linux/skbuff.h:851:1: warning: directive in macro's argument list

Hrm, I can't reproduce this, using several GCC versions and net-next. What
compiler version[1] and base commit[2] were used here?

-Kees

[1] https://github.com/kuba-moo/nipa/pull/10
[2] https://github.com/kuba-moo/nipa/pull/11

-- 
Kees Cook

From kuba at kernel.org  Fri Nov 19 18:41:44 2021
From: kuba at kernel.org (Jakub Kicinski)
Date: Fri, 19 Nov 2021 10:41:44 -0800
Subject: [PATCH] skbuff: Switch structure bounds to struct_group()
In-Reply-To: <202111191015.509A0BD@keescook>
References: <20211118183615.1281978-1-keescook@chromium.org>
 <20211118231355.7a39d22f@kicinski-fedora-pc1c0hjn.dhcp.thefacebook.com>
 <202111191015.509A0BD@keescook>
Message-ID: <20211119104144.7cb1eac6@kicinski-fedora-pc1c0hjn.dhcp.thefacebook.com>

On Fri, 19 Nov 2021 10:26:19 -0800 Kees Cook wrote:
> On Thu, Nov 18, 2021 at 11:13:55PM -0800, Jakub Kicinski wrote:
> > On Thu, 18 Nov 2021 10:36:15 -0800 Kees Cook wrote:  
> > > In preparation for FORTIFY_SOURCE performing compile-time and run-time
> > > field bounds checking for memcpy(), memmove(), and memset(), avoid
> > > intentionally writing across neighboring fields.
> > > 
> > > Replace the existing empty member position markers "headers_start" and
> > > "headers_end" with a struct_group(). This will allow memcpy() and sizeof()
> > > to more easily reason about sizes, and improve readability.
> > > 
> > > "pahole" shows no size nor member offset changes to struct sk_buff.
> > > "objdump -d" shows no object code changes (outside of WARNs affected by
> > > source line number changes).  
> > 
> > This adds ~27k of these warnings to W=1 gcc builds:
> > 
> > include/linux/skbuff.h:851:1: warning: directive in macro's argument list  
> 
> Hrm, I can't reproduce this, using several GCC versions and net-next. What
> compiler version[1] and base commit[2] were used here?

gcc version 11.2.1 20210728 (Red Hat 11.2.1-1) (GCC) 

HEAD was at: 3b1abcf12894 Merge tag 'regmap-no-bus-update-bits' of git://...

> [1] https://github.com/kuba-moo/nipa/pull/10
> [2] https://github.com/kuba-moo/nipa/pull/11

Thanks for these! Will pull in as soon as the bot finishes with what
it's chewing on right now.

From kuba at kernel.org  Fri Nov 19 18:53:05 2021
From: kuba at kernel.org (Jakub Kicinski)
Date: Fri, 19 Nov 2021 10:53:05 -0800
Subject: [PATCH] skbuff: Switch structure bounds to struct_group()
In-Reply-To: <20211119104144.7cb1eac6@kicinski-fedora-pc1c0hjn.dhcp.thefacebook.com>
References: <20211118183615.1281978-1-keescook@chromium.org>
 <20211118231355.7a39d22f@kicinski-fedora-pc1c0hjn.dhcp.thefacebook.com>
 <202111191015.509A0BD@keescook>
 <20211119104144.7cb1eac6@kicinski-fedora-pc1c0hjn.dhcp.thefacebook.com>
Message-ID: <20211119105253.1db523b5@kicinski-fedora-pc1c0hjn.dhcp.thefacebook.com>

On Fri, 19 Nov 2021 10:41:44 -0800 Jakub Kicinski wrote:
> On Fri, 19 Nov 2021 10:26:19 -0800 Kees Cook wrote:
> > On Thu, Nov 18, 2021 at 11:13:55PM -0800, Jakub Kicinski wrote:  
> > > This adds ~27k of these warnings to W=1 gcc builds:
> > > 
> > > include/linux/skbuff.h:851:1: warning: directive in macro's argument list    
> > 
> > Hrm, I can't reproduce this, using several GCC versions and net-next. What
> > compiler version[1] and base commit[2] were used here?  
> 
> gcc version 11.2.1 20210728 (Red Hat 11.2.1-1) (GCC) 
> 
> HEAD was at: 3b1abcf12894 Merge tag 'regmap-no-bus-update-bits' of git://...

Ah, damn, I just realized, it's probably sparse! The build sets C=1.

From keescook at chromium.org  Fri Nov 19 19:04:54 2021
From: keescook at chromium.org (Kees Cook)
Date: Fri, 19 Nov 2021 11:04:54 -0800
Subject: [PATCH] skbuff: Switch structure bounds to struct_group()
In-Reply-To: <20211119105253.1db523b5@kicinski-fedora-pc1c0hjn.dhcp.thefacebook.com>
References: <20211118183615.1281978-1-keescook@chromium.org>
 <20211118231355.7a39d22f@kicinski-fedora-pc1c0hjn.dhcp.thefacebook.com>
 <202111191015.509A0BD@keescook>
 <20211119104144.7cb1eac6@kicinski-fedora-pc1c0hjn.dhcp.thefacebook.com>
 <20211119105253.1db523b5@kicinski-fedora-pc1c0hjn.dhcp.thefacebook.com>
Message-ID: <202111191103.074D77AF2@keescook>

On Fri, Nov 19, 2021 at 10:53:05AM -0800, Jakub Kicinski wrote:
> On Fri, 19 Nov 2021 10:41:44 -0800 Jakub Kicinski wrote:
> > On Fri, 19 Nov 2021 10:26:19 -0800 Kees Cook wrote:
> > > On Thu, Nov 18, 2021 at 11:13:55PM -0800, Jakub Kicinski wrote:  
> > > > This adds ~27k of these warnings to W=1 gcc builds:
> > > > 
> > > > include/linux/skbuff.h:851:1: warning: directive in macro's argument list    
> > > 
> > > Hrm, I can't reproduce this, using several GCC versions and net-next. What
> > > compiler version[1] and base commit[2] were used here?  
> > 
> > gcc version 11.2.1 20210728 (Red Hat 11.2.1-1) (GCC) 
> > 
> > HEAD was at: 3b1abcf12894 Merge tag 'regmap-no-bus-update-bits' of git://...
> 
> Ah, damn, I just realized, it's probably sparse! The build sets C=1.

*head desk* I looked right at the "C=1" (noting it was missing for the
clang build) and didn't put it together. Let me see what I need to do to
make sparse happy...

-- 
Kees Cook

From keescook at chromium.org  Sun Nov 21 00:31:47 2021
From: keescook at chromium.org (Kees Cook)
Date: Sat, 20 Nov 2021 16:31:47 -0800
Subject: [PATCH v2 net-next 0/2] skbuff: Switch structure bounds to
 struct_group()
Message-ID: <20211121003149.28397-1-keescook@chromium.org>

Hi,

This is a pair of patches to add struct_group() to struct sk_buff. The
first is needed to work around sparse-specific complaints, and is new
for v2. The second patch is the same as originally sent as v1.

-Kees

Kees Cook (2):
  skbuff: Move conditional preprocessor directives out of struct sk_buff
  skbuff: Switch structure bounds to struct_group()

 drivers/net/wireguard/queueing.h |  4 +--
 include/linux/skbuff.h           | 46 +++++++++++++++-----------------
 net/core/filter.c                | 10 +++----
 net/core/skbuff.c                | 14 ++++------
 4 files changed, 33 insertions(+), 41 deletions(-)

-- 
2.30.2


From keescook at chromium.org  Sun Nov 21 00:31:49 2021
From: keescook at chromium.org (Kees Cook)
Date: Sat, 20 Nov 2021 16:31:49 -0800
Subject: [PATCH v2 net-next 2/2] skbuff: Switch structure bounds to
 struct_group()
In-Reply-To: <20211121003149.28397-1-keescook@chromium.org>
References: <20211121003149.28397-1-keescook@chromium.org>
Message-ID: <20211121003149.28397-3-keescook@chromium.org>

In preparation for FORTIFY_SOURCE performing compile-time and run-time
field bounds checking for memcpy(), memmove(), and memset(), avoid
intentionally writing across neighboring fields.

Replace the existing empty member position markers "headers_start" and
"headers_end" with a struct_group(). This will allow memcpy() and sizeof()
to more easily reason about sizes, and improve readability.

"pahole" shows no size nor member offset changes to struct sk_buff.
"objdump -d" shows no object code changes (outside of WARNs affected by
source line number changes).

Signed-off-by: Kees Cook <keescook at chromium.org>
Reviewed-by: Gustavo A. R. Silva <gustavoars at kernel.org>
Reviewed-by: Jason A. Donenfeld <Jason at zx2c4.com> # drivers/net/wireguard/*
Link: https://lore.kernel.org/lkml/20210728035006.GD35706 at embeddedor
---
 drivers/net/wireguard/queueing.h |  4 +---
 include/linux/skbuff.h           | 10 +++-------
 net/core/skbuff.c                | 14 +++++---------
 3 files changed, 9 insertions(+), 19 deletions(-)

diff --git a/drivers/net/wireguard/queueing.h b/drivers/net/wireguard/queueing.h
index 4ef2944a68bc..52da5e963003 100644
--- a/drivers/net/wireguard/queueing.h
+++ b/drivers/net/wireguard/queueing.h
@@ -79,9 +79,7 @@ static inline void wg_reset_packet(struct sk_buff *skb, bool encapsulating)
 	u8 sw_hash = skb->sw_hash;
 	u32 hash = skb->hash;
 	skb_scrub_packet(skb, true);
-	memset(&skb->headers_start, 0,
-	       offsetof(struct sk_buff, headers_end) -
-		       offsetof(struct sk_buff, headers_start));
+	memset(&skb->headers, 0, sizeof(skb->headers));
 	if (encapsulating) {
 		skb->l4_hash = l4_hash;
 		skb->sw_hash = sw_hash;
diff --git a/include/linux/skbuff.h b/include/linux/skbuff.h
index 0bce88ac799a..b474e5bd71cf 100644
--- a/include/linux/skbuff.h
+++ b/include/linux/skbuff.h
@@ -808,12 +808,10 @@ struct sk_buff {
 	__u8			active_extensions;
 #endif
 
-	/* fields enclosed in headers_start/headers_end are copied
+	/* Fields enclosed in headers group are copied
 	 * using a single memcpy() in __copy_skb_header()
 	 */
-	/* private: */
-	__u32			headers_start[0];
-	/* public: */
+	struct_group(headers,
 
 	/* private: */
 	__u8			__pkt_type_offset[0];
@@ -918,9 +916,7 @@ struct sk_buff {
 	u64			kcov_handle;
 #endif
 
-	/* private: */
-	__u32			headers_end[0];
-	/* public: */
+	); /* end headers group */
 
 	/* These elements must be at the end, see alloc_skb() for details.  */
 	sk_buff_data_t		tail;
diff --git a/net/core/skbuff.c b/net/core/skbuff.c
index ba2f38246f07..3a42b2a3a571 100644
--- a/net/core/skbuff.c
+++ b/net/core/skbuff.c
@@ -992,12 +992,10 @@ void napi_consume_skb(struct sk_buff *skb, int budget)
 }
 EXPORT_SYMBOL(napi_consume_skb);
 
-/* Make sure a field is enclosed inside headers_start/headers_end section */
+/* Make sure a field is contained by headers group */
 #define CHECK_SKB_FIELD(field) \
-	BUILD_BUG_ON(offsetof(struct sk_buff, field) <		\
-		     offsetof(struct sk_buff, headers_start));	\
-	BUILD_BUG_ON(offsetof(struct sk_buff, field) >		\
-		     offsetof(struct sk_buff, headers_end));	\
+	BUILD_BUG_ON(offsetof(struct sk_buff, field) !=		\
+		     offsetof(struct sk_buff, headers.field));	\
 
 static void __copy_skb_header(struct sk_buff *new, const struct sk_buff *old)
 {
@@ -1009,14 +1007,12 @@ static void __copy_skb_header(struct sk_buff *new, const struct sk_buff *old)
 	__skb_ext_copy(new, old);
 	__nf_copy(new, old, false);
 
-	/* Note : this field could be in headers_start/headers_end section
+	/* Note : this field could be in the headers group.
 	 * It is not yet because we do not want to have a 16 bit hole
 	 */
 	new->queue_mapping = old->queue_mapping;
 
-	memcpy(&new->headers_start, &old->headers_start,
-	       offsetof(struct sk_buff, headers_end) -
-	       offsetof(struct sk_buff, headers_start));
+	memcpy(&new->headers, &old->headers, sizeof(new->headers));
 	CHECK_SKB_FIELD(protocol);
 	CHECK_SKB_FIELD(csum);
 	CHECK_SKB_FIELD(hash);
-- 
2.30.2


From keescook at chromium.org  Sun Nov 21 00:31:48 2021
From: keescook at chromium.org (Kees Cook)
Date: Sat, 20 Nov 2021 16:31:48 -0800
Subject: [PATCH v2 net-next 1/2] skbuff: Move conditional preprocessor
 directives out of struct sk_buff
In-Reply-To: <20211121003149.28397-1-keescook@chromium.org>
References: <20211121003149.28397-1-keescook@chromium.org>
Message-ID: <20211121003149.28397-2-keescook@chromium.org>

In preparation for using the struct_group() macro in struct sk_buff,
move the conditional preprocessor directives out of the region of struct
sk_buff that will be enclosed by struct_group(). While GCC and Clang are
happy with conditional preprocessor directives here, sparse is not, even
under -Wno-directive-within-macro[1], as would be seen under a C=1 build:

net/core/filter.c: note: in included file (through include/linux/netlink.h, include/linux/sock_diag.h):
./include/linux/skbuff.h:820:1: warning: directive in macro's argument list
./include/linux/skbuff.h:822:1: warning: directive in macro's argument list
./include/linux/skbuff.h:846:1: warning: directive in macro's argument list
./include/linux/skbuff.h:848:1: warning: directive in macro's argument list

Additionally remove empty macro argument definitions and usage.

"objdump -d" shows no object code differences.

[1] https://www.spinics.net/lists/linux-sparse/msg10857.html

Signed-off-by: Kees Cook <keescook at chromium.org>
---
 include/linux/skbuff.h | 36 +++++++++++++++++++-----------------
 net/core/filter.c      | 10 +++++-----
 2 files changed, 24 insertions(+), 22 deletions(-)

diff --git a/include/linux/skbuff.h b/include/linux/skbuff.h
index 686a666d073d..0bce88ac799a 100644
--- a/include/linux/skbuff.h
+++ b/include/linux/skbuff.h
@@ -792,7 +792,7 @@ struct sk_buff {
 #else
 #define CLONED_MASK	1
 #endif
-#define CLONED_OFFSET()		offsetof(struct sk_buff, __cloned_offset)
+#define CLONED_OFFSET		offsetof(struct sk_buff, __cloned_offset)
 
 	/* private: */
 	__u8			__cloned_offset[0];
@@ -815,18 +815,10 @@ struct sk_buff {
 	__u32			headers_start[0];
 	/* public: */
 
-/* if you move pkt_type around you also must adapt those constants */
-#ifdef __BIG_ENDIAN_BITFIELD
-#define PKT_TYPE_MAX	(7 << 5)
-#else
-#define PKT_TYPE_MAX	7
-#endif
-#define PKT_TYPE_OFFSET()	offsetof(struct sk_buff, __pkt_type_offset)
-
 	/* private: */
 	__u8			__pkt_type_offset[0];
 	/* public: */
-	__u8			pkt_type:3;
+	__u8			pkt_type:3; /* see PKT_TYPE_MAX */
 	__u8			ignore_df:1;
 	__u8			nf_trace:1;
 	__u8			ip_summed:2;
@@ -842,16 +834,10 @@ struct sk_buff {
 	__u8			encap_hdr_csum:1;
 	__u8			csum_valid:1;
 
-#ifdef __BIG_ENDIAN_BITFIELD
-#define PKT_VLAN_PRESENT_BIT	7
-#else
-#define PKT_VLAN_PRESENT_BIT	0
-#endif
-#define PKT_VLAN_PRESENT_OFFSET()	offsetof(struct sk_buff, __pkt_vlan_present_offset)
 	/* private: */
 	__u8			__pkt_vlan_present_offset[0];
 	/* public: */
-	__u8			vlan_present:1;
+	__u8			vlan_present:1;	/* See PKT_VLAN_PRESENT_BIT */
 	__u8			csum_complete_sw:1;
 	__u8			csum_level:2;
 	__u8			csum_not_inet:1;
@@ -950,6 +936,22 @@ struct sk_buff {
 #endif
 };
 
+/* if you move pkt_type around you also must adapt those constants */
+#ifdef __BIG_ENDIAN_BITFIELD
+#define PKT_TYPE_MAX	(7 << 5)
+#else
+#define PKT_TYPE_MAX	7
+#endif
+#define PKT_TYPE_OFFSET		offsetof(struct sk_buff, __pkt_type_offset)
+
+/* if you move pkt_vlan_present around you also must adapt these constants */
+#ifdef __BIG_ENDIAN_BITFIELD
+#define PKT_VLAN_PRESENT_BIT	7
+#else
+#define PKT_VLAN_PRESENT_BIT	0
+#endif
+#define PKT_VLAN_PRESENT_OFFSET	offsetof(struct sk_buff, __pkt_vlan_present_offset)
+
 #ifdef __KERNEL__
 /*
  *	Handling routines are only of interest to the kernel
diff --git a/net/core/filter.c b/net/core/filter.c
index e471c9b09670..0bf912a44099 100644
--- a/net/core/filter.c
+++ b/net/core/filter.c
@@ -301,7 +301,7 @@ static u32 convert_skb_access(int skb_field, int dst_reg, int src_reg,
 		break;
 
 	case SKF_AD_PKTTYPE:
-		*insn++ = BPF_LDX_MEM(BPF_B, dst_reg, src_reg, PKT_TYPE_OFFSET());
+		*insn++ = BPF_LDX_MEM(BPF_B, dst_reg, src_reg, PKT_TYPE_OFFSET);
 		*insn++ = BPF_ALU32_IMM(BPF_AND, dst_reg, PKT_TYPE_MAX);
 #ifdef __BIG_ENDIAN_BITFIELD
 		*insn++ = BPF_ALU32_IMM(BPF_RSH, dst_reg, 5);
@@ -323,7 +323,7 @@ static u32 convert_skb_access(int skb_field, int dst_reg, int src_reg,
 				      offsetof(struct sk_buff, vlan_tci));
 		break;
 	case SKF_AD_VLAN_TAG_PRESENT:
-		*insn++ = BPF_LDX_MEM(BPF_B, dst_reg, src_reg, PKT_VLAN_PRESENT_OFFSET());
+		*insn++ = BPF_LDX_MEM(BPF_B, dst_reg, src_reg, PKT_VLAN_PRESENT_OFFSET);
 		if (PKT_VLAN_PRESENT_BIT)
 			*insn++ = BPF_ALU32_IMM(BPF_RSH, dst_reg, PKT_VLAN_PRESENT_BIT);
 		if (PKT_VLAN_PRESENT_BIT < 7)
@@ -8027,7 +8027,7 @@ static int bpf_unclone_prologue(struct bpf_insn *insn_buf, bool direct_write,
 	 * (Fast-path, otherwise approximation that we might be
 	 *  a clone, do the rest in helper.)
 	 */
-	*insn++ = BPF_LDX_MEM(BPF_B, BPF_REG_6, BPF_REG_1, CLONED_OFFSET());
+	*insn++ = BPF_LDX_MEM(BPF_B, BPF_REG_6, BPF_REG_1, CLONED_OFFSET);
 	*insn++ = BPF_ALU32_IMM(BPF_AND, BPF_REG_6, CLONED_MASK);
 	*insn++ = BPF_JMP_IMM(BPF_JEQ, BPF_REG_6, 0, 7);
 
@@ -8615,7 +8615,7 @@ static u32 bpf_convert_ctx_access(enum bpf_access_type type,
 	case offsetof(struct __sk_buff, pkt_type):
 		*target_size = 1;
 		*insn++ = BPF_LDX_MEM(BPF_B, si->dst_reg, si->src_reg,
-				      PKT_TYPE_OFFSET());
+				      PKT_TYPE_OFFSET);
 		*insn++ = BPF_ALU32_IMM(BPF_AND, si->dst_reg, PKT_TYPE_MAX);
 #ifdef __BIG_ENDIAN_BITFIELD
 		*insn++ = BPF_ALU32_IMM(BPF_RSH, si->dst_reg, 5);
@@ -8640,7 +8640,7 @@ static u32 bpf_convert_ctx_access(enum bpf_access_type type,
 	case offsetof(struct __sk_buff, vlan_present):
 		*target_size = 1;
 		*insn++ = BPF_LDX_MEM(BPF_B, si->dst_reg, si->src_reg,
-				      PKT_VLAN_PRESENT_OFFSET());
+				      PKT_VLAN_PRESENT_OFFSET);
 		if (PKT_VLAN_PRESENT_BIT)
 			*insn++ = BPF_ALU32_IMM(BPF_RSH, si->dst_reg, PKT_VLAN_PRESENT_BIT);
 		if (PKT_VLAN_PRESENT_BIT < 7)
-- 
2.30.2


From pete at peterhyman.com  Fri Nov 19 15:33:34 2021
From: pete at peterhyman.com (Peter Hyman)
Date: Fri, 19 Nov 2021 09:33:34 -0600
Subject: [PATCH] Improvements to wg-quick output for linux.bash.
Message-ID: <20211119093334.3428f3ee@tommyv.localhost>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: RIPEMD160


- From c675c173a8b008bd2853fde8688f4da34271ae18 Mon Sep 17 00:00:00 2001
From: Peter Hyman <pete at peterhyman.com>
Date: Fri, 19 Nov 2021 08:30:29 -0600
Subject: [PATCH] Improvements to wg-quick output for linux.bash.

Replaced use of '<(echo $var)' for 'wg setconf -f' and 'nft -f' commands.
Use of '<(echo $var)' obscured actual input to 'wg' and 'nft' commands
and replaced with /dev/fd/63 which just indicates piped input.
After this commit, output will reflect actual commands for 'nft'
and will echo the config parameters being read by 'wg setconf'.
Config parameters will also hide Private and Preshared keys like
the 'wg' command.

Before
======
[#] wg setconf wg0 /dev/fd/63
[#] nft -f /dev/fd/63

After
=====
[#] wg setconf wg0 /dev/fd/63
wg configuration
[Interface]
PrivateKey = (hidden)
ListenPort = 51820

[Peer]
PublicKey = xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx=
PresharedKey = (hidden)
AllowedIPs = 0.0.0.0/0
Endpoint = xxx.xxx.xxx.xxx:51820
PersistentKeepalive = 25

[#] nft delete table ip wg-quick-wg0

Signed-off-by: Peter Hyman <pete at peterhyman.com>
- ---
 src/wg-quick/linux.bash | 7 +++++--
 1 file changed, 5 insertions(+), 2 deletions(-)

diff --git a/src/wg-quick/linux.bash b/src/wg-quick/linux.bash
index e4d4c4f..f4f7298 100755
- --- a/src/wg-quick/linux.bash
+++ b/src/wg-quick/linux.bash
@@ -192,7 +192,7 @@ remove_firewall() {
 		while read -r table; do
 			[[ $table == *" wg-quick-$INTERFACE" ]] && printf -v nftcmd '%sdelete %s\n' "$nftcmd" "$table"
 		done < <(nft list tables 2>/dev/null)
- -		[[ -z $nftcmd ]] || cmd nft -f <(echo -n "$nftcmd")
+		[[ -z $nftcmd ]] || cmd nft "$nftcmd"
 	fi
 	if type -p iptables >/dev/null; then
 		local line iptables found restore
@@ -239,7 +239,7 @@ add_default() {
 	printf -v nftcmd '%sadd rule %s %s premangle meta l4proto udp meta mark set ct mark \n' "$nftcmd" "$pf" "$nftable"
 	[[ $proto == -4 ]] && cmd sysctl -q net.ipv4.conf.all.src_valid_mark=1
 	if type -p nft >/dev/null; then
- -		cmd nft -f <(echo -n "$nftcmd")
+		cmd nft "$nftcmd"
 	else
 		echo -n "$restore" | cmd $iptables-restore -n
 	fi
@@ -248,7 +248,10 @@ add_default() {
 }
 
 set_config() {
+	local WG_CONFIGTMP
+	WG_CONFIGTMP=$(echo "wg configuration\n$WG_CONFIG" | sed -e 's/\(PrivateKey = \|PresharedKey = \).*$/\1(hidden)/')
 	cmd wg setconf "$INTERFACE" <(echo "$WG_CONFIG")
+	echo -e "$WG_CONFIGTMP"
 }
 
 save_config() {
- -- 
2.34.0


- -- 
Peter Hyman
GPG: 0x467FBF7D
-----BEGIN PGP SIGNATURE-----

iEYEAREDAAYFAmGXw84ACgkQTTfGLUZ/v3079gCgvx2ZCbBfmFH2FQbSXAl2lhPy
svMAmwav2EfkYwJ2jLgBm0ws5j6IQURc
=Ls+S
-----END PGP SIGNATURE-----

From patchwork-bot+netdevbpf at kernel.org  Mon Nov 22 15:40:09 2021
From: patchwork-bot+netdevbpf at kernel.org (patchwork-bot+netdevbpf at kernel.org)
Date: Mon, 22 Nov 2021 15:40:09 +0000
Subject: [PATCH v2 net-next 0/2] skbuff: Switch structure bounds to
 struct_group()
In-Reply-To: <20211121003149.28397-1-keescook@chromium.org>
References: <20211121003149.28397-1-keescook@chromium.org>
Message-ID: <163759560925.30526.865099452539818028.git-patchwork-notify@kernel.org>

Hello:

This series was applied to netdev/net-next.git (master)
by David S. Miller <davem at davemloft.net>:

On Sat, 20 Nov 2021 16:31:47 -0800 you wrote:
> Hi,
> 
> This is a pair of patches to add struct_group() to struct sk_buff. The
> first is needed to work around sparse-specific complaints, and is new
> for v2. The second patch is the same as originally sent as v1.
> 
> -Kees
> 
> [...]

Here is the summary with links:
  - [v2,net-next,1/2] skbuff: Move conditional preprocessor directives out of struct sk_buff
    https://git.kernel.org/netdev/net-next/c/fba84957e2e2
  - [v2,net-next,2/2] skbuff: Switch structure bounds to struct_group()
    https://git.kernel.org/netdev/net-next/c/03f61041c179

You are awesome, thank you!
-- 
Deet-doot-dot, I am a bot.
https://korg.docs.kernel.org/patchwork/pwbot.html



From max.schulze at online.de  Tue Nov 23 09:59:26 2021
From: max.schulze at online.de (Max Schulze)
Date: Tue, 23 Nov 2021 10:59:26 +0100
Subject: [PATCH] wireguard-tools: contrib/reresolve-dns script for Windows
 Powershell
Message-ID: <68de8cb8-81ce-1f51-22fd-9ef20b24f693@online.de>

This is a script I use on windows for dynamic DNS endpoints ("servers").

The big upside is that it supports multiple configurations and does not 
need hard-coded endpoints or "ping-endpoints".

I would like to apply to get it included into wireguard-tools.

I have tested this in win7 and win10. You might add it to the task 
scheduler and run this every 3 minutes or so.

Some care has to be taken as to know which user is running the script. 
The encrypted dpapi-binary file can only be read by the same user that 
created/saved the tunnels, i.e. \system.

To make it easy to maintain this patch, I have tried to stick it as 
close to the linux version as possible. Jason, feel free to modify the 
copyright header as appropriate.

I am in no way the powershell guru, this is more trial-and-error.


NB: Why the limit to 15 chars in the Interface name? Mine had more, thus 
setting the arbitrary (?) limit to 18 chars.


Signed-off-by: Max Schulze <max.schulze at online.de>
---
  contrib/reresolve-dns/reresolve-dns.ps1 | 57 +++++++++++++++++++++++++
  1 file changed, 57 insertions(+)
  create mode 100644 contrib/reresolve-dns/reresolve-dns.ps1

diff --git a/contrib/reresolve-dns/reresolve-dns.ps1 
b/contrib/reresolve-dns/reresolve-dns.ps1
new file mode 100644
index 0000000..951eb33
--- /dev/null
+++ b/contrib/reresolve-dns/reresolve-dns.ps1
@@ -0,0 +1,57 @@
+# SPDX-License-Identifier: GPL-2.0

+#

+# Copyright (C) 2021 Max Schulze. All Rights Reserved.

+# near-literal Translation of the linux version by Jason A. Donenfeld

+

+# to decrypt the dpapi Credentials, you have to be the same user as the 
wireguard tunnel service, i.e. "nt authority\system", check with "whoami"

+# this script might be called by task scheduler as

+#  powershell -NoProfile -NoLogo -NonInteractive -ExecutionPolicy 
Bypass -Command "Get-ChildItem -File 'c:\program 
Files\wireguard\data\configurations\*.dpapi'  | foreach {& C:\<path to 
script>\wireguard_reresolve-dns.ps1 $_.FullName}"

+# if you want to try it in cmd, remember to elevate the user, i.e. with 
psexec from sysutils

+#  psexec -s -i powershell -NoPr...

+

+Set-StrictMode -Version 3

+Add-Type -AssemblyName System.Security

+

+Set-Variable CONFIG_FILE -Value $args[0].ToString().Trim('"')

+

+$byteCrypted = ((Get-Content -LiteralPath $CONFIG_FILE -Encoding Byte 
-ReadCount 0))

+

+$config = 
[System.Security.Cryptography.ProtectedData]::Unprotect($byteCrypted,$null,[System.Security.Cryptography.DataProtectionScope]::LocalMachine)

+

+$config = [System.Text.UTF8Encoding]::UTF8.GetString($config)

+

+Set-Variable Interface -Option Constant -Value $(if ($CONFIG_FILE 
-match '.?([a-zA-Z0-9_=+.-]{1,18})\.conf.dpapi$') { $matches[1] } else { 
$null })

+

+function process_peer () {

+  if (-not $PEER_SECTION -or ($PUBLIC_KEY -eq $null) -or ($ENDPOINT -eq 
$null)) { return }

+  if (-not ((& wg show "$INTERFACE" latest-handshakes) -replace 
$PUBLIC_KEY -match ('[0-9]+'))) { return }

+  if (((Get-Date) - (New-Object -Type DateTime -ArgumentList 
1970,1,1,0,0,0,0).AddSeconds($matches[0]).ToLocalTime()).TotalSeconds 
-le 135) { return }

+  (& wg set "$INTERFACE" peer "$PUBLIC_KEY" endpoint "$ENDPOINT")

+  reset_peer_section

+}

+

+function reset_peer_section () {

+  Set-Variable PEER_SECTION -Value $null

+  Set-Variable PUBLIC_KEY -Value $null

+  Set-Variable ENDPOINT -Value $null

+}

+

+reset_peer_section

+Set-Variable PEER_SECTION -Value $null

+

+foreach ($line in 
$config.Split([Environment]::NewLine,[StringSplitOptions]::RemoveEmptyEntries)) 
{

+  if ($line.Trim().length -gt 0) {

+    $stripped = $line.Trim() -ireplace '\#.*'

+    $key = $stripped -ireplace '=.*'; $key = $key.Trim()

+    $val = $stripped -ireplace '^.*?='; $val = $val.Trim()

+    if ($key -match '\[.*') { process_peer; reset_peer_section; }

+    if ($key -eq '[Peer]') { $PEER_SECTION = $true }

+    if ($PEER_SECTION) {

+      switch ($key) {

+        "PublicKey" { $PUBLIC_KEY = $val; continue; }

+        "Endpoint" { $ENDPOINT = $val; continue; }

+      }

+    }

+  }

+}

+process_peer

-- 
2.33.1

From max.schulze at online.de  Tue Nov 23 10:14:28 2021
From: max.schulze at online.de (Max Schulze)
Date: Tue, 23 Nov 2021 11:14:28 +0100
Subject: [PATCH v2] wireguard-tools: contrib/reresolve-dns script for
 Windows Powershell
In-Reply-To: <68de8cb8-81ce-1f51-22fd-9ef20b24f693@online.de>
References: <68de8cb8-81ce-1f51-22fd-9ef20b24f693@online.de>
Message-ID: <a0506608-2797-7bc2-8786-97678d01832a@online.de>

v2: Resend as-is, patch got garbled by thunderbird -.-

This is a script I use on windows for dynamic DNS endpoints ("servers").

The big upside is that it supports multiple configurations and does not need hard-coded endpoints or "ping-endpoints".

I would like to apply to get it included into wireguard-tools.

I have tested this in win7 and win10. You might add it to the task scheduler and run this every 3 minutes or so.

Some care has to be taken as to know which user is running the script. The encrypted dpapi-binary file can only be read by the same user that created/saved the tunnels, i.e. \system.

To make it easy to maintain this patch, I have tried to stick it as close to the linux version as possible. Jason, feel free to modify the copyright header as appropriate.

I am in no way the powershell guru, this is more trial-and-error.


NB: Why the limit to 15 chars in the Interface name? Mine had more, thus setting the arbitrary (?) limit to 18 chars.


Signed-off-by: Max Schulze <max.schulze at online.de>
---
 contrib/reresolve-dns/reresolve-dns.ps1 | 57 +++++++++++++++++++++++++
 1 file changed, 57 insertions(+)
 create mode 100644 contrib/reresolve-dns/reresolve-dns.ps1

diff --git a/contrib/reresolve-dns/reresolve-dns.ps1 b/contrib/reresolve-dns/reresolve-dns.ps1
new file mode 100644
index 0000000..951eb33
--- /dev/null
+++ b/contrib/reresolve-dns/reresolve-dns.ps1
@@ -0,0 +1,57 @@
+# SPDX-License-Identifier: GPL-2.0
+#
+# Copyright (C) 2021 Max Schulze. All Rights Reserved.
+# near-literal Translation of the linux version by Jason A. Donenfeld
+
+# to decrypt the dpapi Credentials, you have to be the same user as the wireguard tunnel service, i.e. "nt authority\system", check with "whoami"
+# this script might be called by task scheduler as 
+#  powershell -NoProfile -NoLogo -NonInteractive -ExecutionPolicy Bypass -Command "Get-ChildItem -File 'c:\program Files\wireguard\data\configurations\*.dpapi'  | foreach {& C:\<path to script>\wireguard_reresolve-dns.ps1 $_.FullName}"
+# if you want to try it in cmd, remember to elevate the user, i.e. with psexec from sysutils
+#  psexec -s -i powershell -NoPr...
+
+Set-StrictMode -Version 3
+Add-Type -AssemblyName System.Security
+
+Set-Variable CONFIG_FILE -Value $args[0].ToString().Trim('"')
+
+$byteCrypted = ((Get-Content -LiteralPath $CONFIG_FILE -Encoding Byte -ReadCount 0))
+
+$config = [System.Security.Cryptography.ProtectedData]::Unprotect($byteCrypted,$null,[System.Security.Cryptography.DataProtectionScope]::LocalMachine)
+
+$config = [System.Text.UTF8Encoding]::UTF8.GetString($config)
+
+Set-Variable Interface -Option Constant -Value $(if ($CONFIG_FILE -match '.?([a-zA-Z0-9_=+.-]{1,18})\.conf.dpapi$') { $matches[1] } else { $null })
+
+function process_peer () {
+  if (-not $PEER_SECTION -or ($PUBLIC_KEY -eq $null) -or ($ENDPOINT -eq $null)) { return }
+  if (-not ((& wg show "$INTERFACE" latest-handshakes) -replace $PUBLIC_KEY -match ('[0-9]+'))) { return }
+  if (((Get-Date) - (New-Object -Type DateTime -ArgumentList 1970,1,1,0,0,0,0).AddSeconds($matches[0]).ToLocalTime()).TotalSeconds -le 135) { return }
+  (& wg set "$INTERFACE" peer "$PUBLIC_KEY" endpoint "$ENDPOINT")
+  reset_peer_section
+}
+
+function reset_peer_section () {
+  Set-Variable PEER_SECTION -Value $null
+  Set-Variable PUBLIC_KEY -Value $null
+  Set-Variable ENDPOINT -Value $null
+}
+
+reset_peer_section
+Set-Variable PEER_SECTION -Value $null
+
+foreach ($line in $config.Split([Environment]::NewLine,[StringSplitOptions]::RemoveEmptyEntries)) {
+  if ($line.Trim().length -gt 0) {
+    $stripped = $line.Trim() -ireplace '\#.*'
+    $key = $stripped -ireplace '=.*'; $key = $key.Trim()
+    $val = $stripped -ireplace '^.*?='; $val = $val.Trim()
+    if ($key -match '\[.*') { process_peer; reset_peer_section; }
+    if ($key -eq '[Peer]') { $PEER_SECTION = $true }
+    if ($PEER_SECTION) {
+      switch ($key) {
+        "PublicKey" { $PUBLIC_KEY = $val; continue; }
+        "Endpoint" { $ENDPOINT = $val; continue; }
+      }
+    }
+  }
+}
+process_peer
-- 
2.33.1


From wireguard-mail at chil.at  Tue Nov 23 13:27:24 2021
From: wireguard-mail at chil.at (Christoph Loesch)
Date: Tue, 23 Nov 2021 14:27:24 +0100
Subject: [PATCH v2] wireguard-tools: contrib/reresolve-dns script for
 Windows Powershell
In-Reply-To: <a0506608-2797-7bc2-8786-97678d01832a@online.de>
References: <68de8cb8-81ce-1f51-22fd-9ef20b24f693@online.de>
 <a0506608-2797-7bc2-8786-97678d01832a@online.de>
Message-ID: <20c5ff86-be75-b9e2-8dda-0dc5b0ec8f17@chil.at>

Hi,

Am 23.11.2021 um 11:14 schrieb Max Schulze:
> NB: Why the limit to 15 chars in the Interface name? Mine had more, thus setting the arbitrary (?) limit to 18 chars.

as Aaron Jones pointed out in my thread "Android app: length of name":

on Linux, the length of an interface name (e.g. "eth0") is limited to 15 characters

Kind regards,
Christoph


From wireguard-mail at chil.at  Tue Nov 23 13:52:37 2021
From: wireguard-mail at chil.at (Christoph Loesch)
Date: Tue, 23 Nov 2021 14:52:37 +0100
Subject: Wireguard source ip selection issue with multi interfaces
In-Reply-To: <CACu-5+0Z1JPCttbZpM13870N9Wce8K3V2ZRdhV6XFmVk4FJKxQ@mail.gmail.com>
References: <CACu-5+0Z1JPCttbZpM13870N9Wce8K3V2ZRdhV6XFmVk4FJKxQ@mail.gmail.com>
Message-ID: <60b5d439-e117-f3c7-3a6b-8ce46bc48fc2@chil.at>

Hi,

Is this related to my issue with using wrong source ip on outgoing connections?
(https://lists.zx2c4.com/pipermail/wireguard/2021-November/007307.html)

Which bug would that be? Would be interesting in more detail to understand the issue.
I'm having a hard time to reproduce (or somehow "fix") the issue I am facing, since this issue is on only 5 devices from over 20 devices where all are configured same.

The configuration or better all routes do not even have any metric defined so I am unsure what to add/remove to change the behaviour.
Also in my configuration there are multiple interfaces (br0,br1,eth0,eth1,etc..) as it is a router but there are no multiple default routes.
Comparing perfectly fine working devices to one of those five devices which have this issue I am not able to find a difference in IP-config or routes.
Even kernel and firmware versions are all same.

The only difference I see is while looking at tcpdump where a simple ping to the wireguard server's internal IP (172.27.0.1/24 in my case) originates with the public IP of the device.

If it is a known bug, it would be nice to know what (and if) I could try to fix and possibly confirm the bug. Or even better beeing able to reproduce the issue.

Thanks and kind regards,
Christoph

Am 19.11.2021 um 14:10 schrieb ??:
> Hi,
> As Jason said in mailing list
> (https://lists.zx2c4.com/pipermail/wireguard/2017-November/002018.html)
> years ago, if wireguard reply the client with wrong src ip, then it is
> a bug. And now I'm trying to set up a minimal configuration.
>
> Server info:
>
> Ubuntu 20.04 in VMware with two Host-only network adapters:
>
> wacke at Ubuntu:/etc$ lsb_release -a
> No LSB modules are available.
> Distributor ID: Ubuntu
> Description: Ubuntu 20.04.3 LTS
> Release: 20.04
> Codename: focal
>
> wacke at Ubuntu:/etc$ uname -a
> Linux Ubuntu 5.14.8-051408-generic #202109280455 SMP Tue Sep 28
> 04:57:31 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
>
> wacke at Ubuntu:/etc$ sudo ip route show
> 169.254.0.0/16 dev ens33 scope link metric 1000
> 172.17.0.0/16 dev docker0 proto kernel scope link src 172.17.0.1 linkdown
> 192.168.187.0/24 dev ens33 proto kernel scope link src 192.168.187.129
> metric 100
> 192.168.187.0/24 dev ens35 proto kernel scope link src 192.168.187.128
> metric 101
> 192.168.199.0/24 dev wg0 proto kernel scope link src 192.168.199.1
>
> wacke at Ubuntu:/etc$ sudo cat /etc/wireguard/wg0.conf
> [Interface]
> PrivateKey = oLLeTlvwWsfGoyQpDwZo0k7AsMf0LkkfycGPwFjamEA=
> ListenPort = 5999
>
> [Peer]
> PublicKey = JYvK2/FBbbSrDbkH29ejWejkBJS3ch/SFtyZwNf5O1c=
> AllowedIPs = 192.168.199.2/32
> PersistentKeepalive = 25
>
> [Peer]
> PublicKey = keIPXZFgvB79biV74kGc5R9vAHpzyVyQHci8KBSDIUw=
> AllowedIPs = 192.168.199.3/32
> PersistentKeepalive = 25
>
> wacke at Ubuntu:/etc$ sudo wg setconf wg0 /etc/wireguard/wg0.conf
> wacke at Ubuntu:/etc$ sudo ip address add dev wg0 192.168.199.1/24
> wacke at Ubuntu:/etc$ sudo ip link set up dev wg0
> wacke at Ubuntu:/etc$ sudo wg show
> interface: wg0
>    public key: Vkdx0MhQkc9anLaUehR/Df1zuKjxceflxMCAeAaCWCo=
>    private key: (hidden)
>    listening port: 5999
>
> peer: JYvK2/FBbbSrDbkH29ejWejkBJS3ch/SFtyZwNf5O1c=
>    endpoint: 192.168.187.1:35560
>    allowed ips: 192.168.199.2/32
>    transfer: 2.89 KiB received, 1.80 KiB sent
>    persistent keepalive: every 25 seconds
>
> peer: keIPXZFgvB79biV74kGc5R9vAHpzyVyQHci8KBSDIUw=
>    allowed ips: 192.168.199.3/32
>    persistent keepalive: every 25 seconds
>
> Client info:
>
> HOME-Server:/NAS/Software/Software # uname -a
> Linux HOME-Server 5.14.14-2-default #1 SMP Thu Oct 21 05:05:03 UTC
> 2021 (2b5383f) x86_64 x86_64 x86_64 GNU/Linux
>
> HOME-Server:/NAS/Software/Software # cat wg0.conf
> [Interface]
> PrivateKey = 4Pw1cetxd9TdfH3TSab+9UcRBlHwZ1o/vmrUgkerZls=
>
> [Peer]
> PublicKey = Vkdx0MhQkc9anLaUehR/Df1zuKjxceflxMCAeAaCWCo=
> Endpoint = 192.168.187.128:5999
> AllowedIPs = 192.168.199.0/24
> PersistentKeepalive = 25
>
> HOME-Server:/NAS/Software/Software # ip link add wg0 type wireguard
> HOME-Server:/NAS/Software/Software # ip address add dev wg0 192.168.199.2/24
> HOME-Server:/NAS/Software/Software # wg setconf wg0 wg0.conf
> HOME-Server:/NAS/Software/Software # ip link set wg0 up
> HOME-Server:/NAS/Software/Software # wg show
> interface: wg0
>    public key: JYvK2/FBbbSrDbkH29ejWejkBJS3ch/SFtyZwNf5O1c=
>    private key: (hidden)
>    listening port: 53058
>
> peer: Vkdx0MhQkc9anLaUehR/Df1zuKjxceflxMCAeAaCWCo=
>    endpoint: 192.168.187.128:5999
>    allowed ips: 192.168.199.0/24
>    transfer: 0 B received, 59.26 KiB sent
>    persistent keepalive: every 25 seconds
>
>
> tcpdump of client side:
>
> HOME-Server:/NAS/Software/Software # tcpdump -i any -vn "(dst port 5999)"
> tcpdump: data link type LINUX_SLL2
> tcpdump: listening on any, link-type LINUX_SLL2 (Linux cooked v2),
> snapshot length 262144 bytes
> 22:10:37.794301 vmnet1 Out IP (tos 0x88, ttl 64, id 54724, offset 0,
> flags [none], proto UDP (17), length 176)
>      192.168.187.1.53058 > 192.168.187.128.5999: UDP, length 148
> 22:10:43.170306 vmnet1 Out IP (tos 0x88, ttl 64, id 55327, offset 0,
> flags [none], proto UDP (17), length 176)
>      192.168.187.1.53058 > 192.168.187.128.5999: UDP, length 148
> 22:10:48.546281 vmnet1 Out IP (tos 0x88, ttl 64, id 56142, offset 0,
> flags [none], proto UDP (17), length 176)
>      192.168.187.1.53058 > 192.168.187.128.5999: UDP, length 148
> 22:10:53.670343 vmnet1 Out IP (tos 0x88, ttl 64, id 57053, offset 0,
> flags [none], proto UDP (17), length 176)
>      192.168.187.1.53058 > 192.168.187.128.5999: UDP, length 148
> 22:11:18.754308 vmnet1 Out IP (tos 0x88, ttl 64, id 58505, offset 0,
> flags [none], proto UDP (17), length 176)
>      192.168.187.1.53058 > 192.168.187.128.5999: UDP, length 148
> 22:11:24.130286 vmnet1 Out IP (tos 0x88, ttl 64, id 58621, offset 0,
> flags [none], proto UDP (17), length 176)
>      192.168.187.1.53058 > 192.168.187.128.5999: UDP, length 148
> 22:11:29.506305 vmnet1 Out IP (tos 0x88, ttl 64, id 58810, offset 0,
> flags [none], proto UDP (17), length 176)
>      192.168.187.1.53058 > 192.168.187.128.5999: UDP, length 148
> 22:11:34.882326 vmnet1 Out IP (tos 0x88, ttl 64, id 59444, offset 0,
> flags [none], proto UDP (17), length 176)
>      192.168.187.1.53058 > 192.168.187.128.5999: UDP, length 148
> ^C
> 8 packets captured
> 8 packets received by filter
> 0 packets dropped by kernel
>
> tcpdump of server side:
>
> wacke at Ubuntu:~$ sudo tcpdump -i any -vn "(src port 5999)"
> tcpdump: listening on any, link-type LINUX_SLL (Linux cooked v1),
> capture size 262144 bytes
> 22:10:11.754862 IP (tos 0x88, ttl 64, id 22024, offset 0, flags
> [none], proto UDP (17), length 120)
>      192.168.187.129.5999 > 192.168.187.1.53058: UDP, length 92
> 22:10:16.874400 IP (tos 0x88, ttl 64, id 22399, offset 0, flags
> [none], proto UDP (17), length 120)
>      192.168.187.129.5999 > 192.168.187.1.53058: UDP, length 92
> 22:10:21.994433 IP (tos 0x88, ttl 64, id 23057, offset 0, flags
> [none], proto UDP (17), length 120)
>      192.168.187.129.5999 > 192.168.187.1.53058: UDP, length 92
> 22:10:27.370558 IP (tos 0x88, ttl 64, id 23912, offset 0, flags
> [none], proto UDP (17), length 120)
>      192.168.187.129.5999 > 192.168.187.1.53058: UDP, length 92
> 22:10:32.494605 IP (tos 0x88, ttl 64, id 24944, offset 0, flags
> [none], proto UDP (17), length 120)
>      192.168.187.129.5999 > 192.168.187.1.53058: UDP, length 92
> 22:10:37.610779 IP (tos 0x88, ttl 64, id 25327, offset 0, flags
> [none], proto UDP (17), length 120)
>      192.168.187.129.5999 > 192.168.187.1.53058: UDP, length 92
> 22:10:42.986834 IP (tos 0x88, ttl 64, id 25498, offset 0, flags
> [none], proto UDP (17), length 120)
>      192.168.187.129.5999 > 192.168.187.1.53058: UDP, length 92
> 22:10:48.362908 IP (tos 0x88, ttl 64, id 26521, offset 0, flags
> [none], proto UDP (17), length 120)
>      192.168.187.129.5999 > 192.168.187.1.53058: UDP, length 92
> 22:10:53.486942 IP (tos 0x88, ttl 64, id 27214, offset 0, flags
> [none], proto UDP (17), length 120)
>      192.168.187.129.5999 > 192.168.187.1.53058: UDP, length 92
> 22:11:18.571437 IP (tos 0x88, ttl 64, id 31112, offset 0, flags
> [none], proto UDP (17), length 120)
>      192.168.187.129.5999 > 192.168.187.1.53058: UDP, length 92
> 22:11:23.947460 IP (tos 0x88, ttl 64, id 31897, offset 0, flags
> [none], proto UDP (17), length 120)
>      192.168.187.129.5999 > 192.168.187.1.53058: UDP, length 92
> 22:11:29.323608 IP (tos 0x88, ttl 64, id 32715, offset 0, flags
> [none], proto UDP (17), length 120)
>      192.168.187.129.5999 > 192.168.187.1.53058: UDP, length 92
> 22:11:34.699676 IP (tos 0x88, ttl 64, id 33840, offset 0, flags
> [none], proto UDP (17), length 120)
>      192.168.187.129.5999 > 192.168.187.1.53058: UDP, length 92
> ^C
> 13 packets captured
> 13 packets received by filter
> 0 packets dropped by kernel
>
> As the tcpdump log showed, the wireguard server always chose the ip
> with lowest metric as source ip (192.168.187.129) to reply to the
> client (while the client tried to connect 192.168.187.128).
>
> Hope the info above can help to debug. Many thanks.

From wireguard-mail at chil.at  Tue Nov 23 16:26:04 2021
From: wireguard-mail at chil.at (Christoph Loesch)
Date: Tue, 23 Nov 2021 17:26:04 +0100
Subject: client uses wrong source ip for outgoing connections
In-Reply-To: <744d7291-e43b-4e8c-76be-c78c11204e17@chil.at>
References: <d60ce240-5bb3-eed7-85a0-31d05cb0133e@chil.at>
 <744d7291-e43b-4e8c-76be-c78c11204e17@chil.at>
Message-ID: <c93393bb-eda1-2588-c52e-14d5cd219364@chil.at>

Hi,

I could at least (temporary) fix this issue by adding the correct src IP to the routes like shown in the following example.
Now I just don't fully understand, what causes wireguard to select the IP from the wrong interface. Or based on what it selects the IP in the first place.
Even ping gives the warning: "ping: Warning: source address might be selected on device other than wg0."
That warning goes away when the routes have the correct IP set as src.
-> But I can definitely say that wireguard somehow selects the wrong IP for outgoing packets.
Just that I don't know why this happens only on 5 out of over 20 devices with same configuration..

ip route del <NET>
ip route add <NET> dev <ALIAS_DEV> src <SRC_IP>

information about interfaces:
root at zi1-router:~# ip a sh wg0
18: wg0: <POINTOPOINT,NOARP,UP,LOWER_UP> mtu 1420 qdisc noqueue state UNKNOWN group default qlen 1000
 ??? link/none
root at zi1-router:~# ip -4 a sh br0
11: br0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
 ??? inet 78.41.x.y/32 scope global br0
 ?????? valid_lft forever preferred_lft forever
root at zi1-router:~# ip -4 a sh br1
12: br1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
 ??? inet 10.34.0.100/24 brd 10.34.0.255 scope global br1
 ?????? valid_lft forever preferred_lft forever

root at zi1-router:~# ip r sh dev wg0
10.5.44.0/24 scope link
172.27.0.0/24 scope link
root at zi1-router:~# ip r d 172.27.0.0/24
root at zi1-router:~# ip r d 10.5.44.0/24
root at zi1-router:~# ip r a 172.27.0.0/24 dev wg0 src 10.34.0.100
root at zi1-router:~# ip r a 10.5.44.0/24 dev wg0 src 10.34.0.100
root at zi1-router:~# ip r sh dev wg0
10.5.44.0/24 scope link src 10.34.0.100
172.27.0.0/24 scope link src 10.34.0.100

root at zi1-router:~# ping 172.27.0.1
PING 172.27.0.1 (172.27.0.1) 56(84) bytes of data.
64 bytes from 172.27.0.1: icmp_seq=1 ttl=64 time=13.1 ms

Kind regards,
Christoph

Am 19.11.2021 um 01:11 schrieb Christoph Loesch:
> if relevant, some more details about interface and routes from good and bad example to compare:
>
> root at eng196-router:~# ip a sh wg0
> 46: wg0: <POINTOPOINT,NOARP,UP,LOWER_UP> mtu 1420 qdisc noqueue state UNKNOWN group default qlen 1000
> ??? link/none
> root at eng196-router:~# ip r sh dev wg0
> 10.5.44.0/24 scope link
> 172.27.0.0/24 scope link
> root at eng196-router:~# ip a sh br1
> 11: br1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
> ??? link/ether 44:d9:e7:x:y:z brd ff:ff:ff:ff:ff:ff
> ??? inet 10.29.85.100/24 brd 10.29.85.255 scope global br1
> ?????? valid_lft forever preferred_lft forever
> ??? inet6 fe80::7c4c:1dff:fe84:fece/64 scope link
> ?????? valid_lft forever preferred_lft forever
>
> root at zi1-router:~# ip a sh wg0
> 18: wg0: <POINTOPOINT,NOARP,UP,LOWER_UP> mtu 1420 qdisc noqueue state UNKNOWN group default qlen 1000
> ??? link/none
> root at zi1-router:~# ip r sh dev wg0
> 10.5.44.0/24 scope link
> 172.27.0.0/24 scope link
> root at zi1-router:~# ip a sh br1
> 12: br1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
> ??? link/ether 74:83:c2:x:y:z brd ff:ff:ff:ff:ff:ff
> ??? inet 10.34.0.100/24 brd 10.34.0.255 scope global br1
> ?????? valid_lft forever preferred_lft forever
> ??? inet6 fe80::2c2e:76ff:fedc:d8e/64 scope link
> ?????? valid_lft forever preferred_lft forever
>
> Am 19.11.2021 um 00:40 schrieb Christoph Loesch:
>> Hi,
>>
>> I am using wireguard on about 20 EdgeRouters (based on Debian stretch).
>> Each router has exact same configuration (apart from router ip addresses and wireguard keys/passphrases).
>> Works very well on most of them but on five routers wireguard uses the wrong ip address for outgoing connections over the tunnel.
>> All routers use kernel 4.14.54-UBNT and wireguard-tools v1.0.20210914
>> Wireguard debian package is from github/WireGuard/wireguard-vyatta-ubnt
>>
>> On the problematic routers the public ip address is used for the tunnel instead the private ip address.
>> Interestingly even in the bad example the wg tunnel is running and the server can reach the routers(=wg clients), but not the other way round.
>>
>> In the following examples 172.27.0.1 is the wireguard server internal ip address.
>> Routers use ip addresses in the 10.0.0.0/8 range for the wg tunnel which are allowed on the server.
>> I already even debugged this with tcpdump where I found out it uses the wrong ip.
>> But looking at a simple ping you also notice the wrong ip after the word "from".
>>
>> Good example:
>> eng196-router:~$ \ping -I wg0 -c1 172.27.0.1
>> ping: Warning: source address might be selected on device other than wg0.
>> PING 172.27.0.1 (172.27.0.1) from 10.29.85.100 wg0: 56(84) bytes of data.
>> 64 bytes from 172.27.0.1: icmp_seq=1 ttl=64 time=6.82 ms
>> --- 172.27.0.1 ping statistics ---
>> 1 packets transmitted, 1 received, 0% packet loss, time 0ms
>> rtt min/avg/max/mdev = 6.826/6.826/6.826/0.000 ms
>>
>> Bad example:
>> zi1-router:~$ \ping -I wg0 -c1 172.27.0.1
>> ping: Warning: source address might be selected on device other than wg0.
>> PING 172.27.0.1 (172.27.0.1) from 78.41.x.y wg0: 56(84) bytes of data.
>> --- 172.27.0.1 ping statistics ---
>> 1 packets transmitted, 0 received, 100% packet loss, time 0ms
>>
>> Configurations:
>> eng196-router:~# wg
>> interface: wg0
>> ? public key: SoV2obcH0qWfCRY3gZbkLNeMa1QRcnhNDCeiI9weszA=
>> ? private key: (hidden)
>> ? listening port: 58205
>> peer: 1syRMYD1jIVFMUMm5hF/j0MzjMQmuC5mlcT1VVugIkU=
>> ? preshared key: (hidden)
>> ? endpoint: 86.59.x.y:1024
>> ? allowed ips: 172.27.0.0/24, 10.5.44.0/24
>> ? latest handshake: 53 seconds ago
>> ? transfer: 24.57 MiB received, 26.48 MiB sent
>> ? persistent keepalive: every 25 seconds
>>
>> zi1-router:~# wg
>> interface: wg0
>> ? public key: aYtVhblpR0XSsAb/dXF3zM9Hu+LxlvrR5RWFU2psF3M=
>> ? private key: (hidden)
>> ? listening port: 45514
>> peer: 1syRMYD1jIVFMUMm5hF/j0MzjMQmuC5mlcT1VVugIkU=
>> ? preshared key: (hidden)
>> ? endpoint: 86.59.x.y:51820
>> ? allowed ips: 172.27.0.0/24, 10.5.44.0/24
>> ? latest handshake: 13 seconds ago
>> ? transfer: 1.79 MiB received, 6.26 MiB sent
>> ? persistent keepalive: every 25 seconds
>>
>> What could cause the wrong selection?
>> Why does that work for most routers but for some not? There must be some difference or something gets confused up by specific ip addresses I guess?
>> How could I debug this further to find the difference and/or cause for this problem?
>>
>> Thanks for any hints and kind regards,
>> Christoph
>>

From wg-lists at bluematt.me  Tue Nov 23 18:32:52 2021
From: wg-lists at bluematt.me (Matt Corallo)
Date: Tue, 23 Nov 2021 13:32:52 -0500
Subject: Incorrect Source Addr Selection On Initiate and Asymmetric Routing
In-Reply-To: <eeee1fa3-8b42-1122-3123-d3b07b150036@bluematt.me>
References: <6fc9765d-f4ef-84d2-c65a-97bab58e3e4b@bluematt.me>
 <eeee1fa3-8b42-1122-3123-d3b07b150036@bluematt.me>
Message-ID: <3c8c2e52-7a1a-2397-d6f4-769032b93044@bluematt.me>

Hit this problem again today on 5.10, seems like there's renewed interest in fixing wg's source 
address selection - any chance one of the recent patches may address this?

On 8/18/20 11:26, Matt Corallo wrote:
> [Resending this few-month-old mail because apparently the list bounced it the first time.]
> 
> 
> Oops, should have mentioned, this may have always been the case, with only recent addition of 
> asymmetric routing leading
> me to identify it, but its at least been the case on 5.6.X and currently is the case on 5.7.6.
> 
> Matt
> 
> On 6/28/20 3:03 PM, Matt Corallo wrote:
>> I run wireguard on some endpoints with anycast IP addresses (which mostly workes seamlessly, which 
>> is awesome!), however
>> of late it seems the source address selection in Wireguard incorrectly selects the default source 
>> address when it most
>> recently received packet(s) to a different address.
>>
>> Most of the routes on such boxes have an explicit default source that is different from the 
>> anycast addresses, as
>> otherwise regular connections from such boxes would fail, eg:
>> 1.0.0.0/24 via XXX dev XXX src (non-anycast-address) metric 32
>>
>> Ive observed wireguard selecting the default source in two cases:
>>
>> a) when the server is the one sending the handshake initiation due to the handshake timer, it 
>> appears the server selects
>> a new source address based on the default. I haen't had practical issues with this, but its worth 
>> noting, and probably
>> fixing.
>>
>> b) when the path outbound to the client is different from the path inbound. In my case, inbound v4 
>> traffic from my phone
>> on T-Mobile US (which passes through CG-NAT) comes into my server on one interface, but the path 
>> back out to TMO is via
>> a different interface. In this case, wireguard selects the default source address and sends a 
>> packet which T-Mobile's
>> CG-NAT drops as there is no NAT entry for it.
>>
>> Matt
>>

From Robert.Dahlem at gmx.net  Mon Nov 22 17:20:12 2021
From: Robert.Dahlem at gmx.net (Robert Dahlem)
Date: Mon, 22 Nov 2021 18:20:12 +0100
Subject: Dynamic routes
Message-ID: <3c70475b-64e5-acf7-819c-94f721fed8b6@gmx.net>

Hello,

I'm searching for a way to dynamically add or delete routes when a
tunnel comes up or goes down. I do not mean "interface wg0 comes up",
but explicitly "tunnel comes up" in the sense that both peers see each
other, key exchange has been successful and packets would be transported
through the tunnel.

The idea behind this is to have redundant tunnels. Let's say a Wireguard
client has a preferred server and a secondary server. Packets should be
routed through the preferred router while that tunnel is established and
through the secondary server when the preferred server is not available
for some reason.

I could always install Quagga or Bird and use OSPF. But that might be a
bit of an overkill for a simple active/passive failover.

Is there any way to react on tunnel state events in Wireguard?

Regards,
Robert

From minus at mnus.de  Tue Nov 23 19:03:32 2021
From: minus at mnus.de (Thomas Renoth)
Date: Tue, 23 Nov 2021 20:03:32 +0100
Subject: [PATCH wireguard-go] conn: linux: fix incorrect endpoint address
Message-ID: <20211123190331.101346-1-minus@mnus.de>

Signed-off-by: Thomas Renoth <minus at mnus.de>
---
`wg show` just shows 0.0.0.0:<port> for peer endpoints currently.

 conn/bind_linux.go | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/conn/bind_linux.go b/conn/bind_linux.go
index 975a6ab..da0670a 100644
--- a/conn/bind_linux.go
+++ b/conn/bind_linux.go
@@ -275,7 +275,7 @@ func (end *LinuxSocketEndpoint) SrcIP() netip.Addr {
 
 func (end *LinuxSocketEndpoint) DstIP() netip.Addr {
 	if !end.isV6 {
-		return netip.AddrFrom4(end.src4().Src)
+		return netip.AddrFrom4(end.dst4().Addr)
 	} else {
 		return netip.AddrFrom16(end.dst6().Addr)
 	}
-- 
2.34.0


From Jason at zx2c4.com  Tue Nov 23 21:01:29 2021
From: Jason at zx2c4.com (Jason A. Donenfeld)
Date: Tue, 23 Nov 2021 22:01:29 +0100
Subject: [PATCH wireguard-go] conn: linux: fix incorrect endpoint address
In-Reply-To: <20211123190331.101346-1-minus@mnus.de>
References: <20211123190331.101346-1-minus@mnus.de>
Message-ID: <CAHmME9pEf9=6C3eAXfanftBt4LJ4viRszQc5oG3Jt9H7Fxd68A@mail.gmail.com>

Thank you for catching this regression!

Jason

From jonathon.fernyhough at york.ac.uk  Tue Nov 23 22:40:06 2021
From: jonathon.fernyhough at york.ac.uk (Jonathon Fernyhough)
Date: Tue, 23 Nov 2021 22:40:06 +0000
Subject: Dynamic routes
In-Reply-To: <3c70475b-64e5-acf7-819c-94f721fed8b6@gmx.net>
References: <3c70475b-64e5-acf7-819c-94f721fed8b6@gmx.net>
Message-ID: <edfa128e-7c6d-c329-5d63-4a3b32c2517c@york.ac.uk>

On 22/11/2021 17:20, Robert Dahlem wrote:
> The idea behind this is to have redundant tunnels. Let's say a Wireguard
> client has a preferred server and a secondary server. Packets should be
> routed through the preferred router while that tunnel is established and
> through the secondary server when the preferred server is not available
> for some reason.

Would route metrics work for this?
-------------- next part --------------
A non-text attachment was scrubbed...
Name: OpenPGP_signature
Type: application/pgp-signature
Size: 840 bytes
Desc: OpenPGP digital signature
URL: <http://lists.zx2c4.com/pipermail/wireguard/attachments/20211123/3673098e/attachment.sig>

From Robert.Dahlem at gmx.net  Tue Nov 23 23:03:02 2021
From: Robert.Dahlem at gmx.net (Robert Dahlem)
Date: Wed, 24 Nov 2021 00:03:02 +0100
Subject: Dynamic routes
In-Reply-To: <edfa128e-7c6d-c329-5d63-4a3b32c2517c@york.ac.uk>
References: <3c70475b-64e5-acf7-819c-94f721fed8b6@gmx.net>
 <edfa128e-7c6d-c329-5d63-4a3b32c2517c@york.ac.uk>
Message-ID: <3b6549b1-7ed0-00fc-2951-fc77049faf13@gmx.net>

Hi,

On 23.11.2021 23:40, Jonathon Fernyhough wrote:

>> The idea behind this is to have redundant tunnels. Let's say a Wireguard
>> client has a preferred server and a secondary server. Packets should be
>> routed through the preferred router while that tunnel is established and
>> through the secondary server when the preferred server is not available
>> for some reason.

> Would route metrics work for this?

Yes and no.

Metrics do useful things when there are identical prefixes with
different metrics over *working* paths. So when both tunnels are up
metrics can choose the preferred path.

With Wireguard (as far as I understand it) routes exist independently
from tunnel state. If the path through the preferred server has a better
metric, but doesn't work, the system does not fall back to the path
through the secondary server.

Metrics work fine when routes are installed/uninstalled dynamically. Say
you have a system with LAN connectivity (best metric), Wifi connectivity
(medium metric) and mobile (UMTS/LTE) connectivity (worst metric). All
of them give you a default route. If all of them are actually connected,
the system would use the route through the LAN. As soon as you unplug
the LAN cable, the LAN routes disappear and the Wifi routes are
selected. Plug the cable back in, the LAN routes reappear and the system
uses the LAN route.

Yes, metrics would work for this. But only if the routes don't appear
before a tunnel comes up and disappear when the tunnel goes down.

Regards,
Robert

From minus at mnus.de  Tue Nov 23 23:23:10 2021
From: minus at mnus.de (minus)
Date: Wed, 24 Nov 2021 00:23:10 +0100
Subject: [PATCH wireguard-go] conn: linux: fix incorrect endpoint address
In-Reply-To: <CAHmME9pEf9=6C3eAXfanftBt4LJ4viRszQc5oG3Jt9H7Fxd68A@mail.gmail.com>
References: <20211123190331.101346-1-minus@mnus.de>
 <CAHmME9pEf9=6C3eAXfanftBt4LJ4viRszQc5oG3Jt9H7Fxd68A@mail.gmail.com>
Message-ID: <fef7daf2-adf7-e042-4977-46fda04aa07d@mnus.de>

On 23/11/2021 22.01, Jason A. Donenfeld wrote:
> Thank you for catching this regression!You're welcome. Unfortunately, I found another one: On an old Debian 8 
system, wireguard-go does not receive or send packets anymore, nor does 
`wg show` show any sent/received stats. On a current Arch system it 
works fine. git bisect points to e3134bf (device: defer state machine 
transitions until configuration is complete) as first bad commit. I 
haven't further tried reproducing it, hoping that it's something obvious 
to you.

From ch at ntrv.dk  Wed Nov 24 11:18:46 2021
From: ch at ntrv.dk (Chriztoffer Hansen)
Date: Wed, 24 Nov 2021 12:18:46 +0100
Subject: Dynamic routes
In-Reply-To: <3c70475b-64e5-acf7-819c-94f721fed8b6@gmx.net>
References: <3c70475b-64e5-acf7-819c-94f721fed8b6@gmx.net>
Message-ID: <CA+cYV6s2jMh4z5eUz-AAY5MO9CBpszWZtiUbkWY8XeFw7pLaFg@mail.gmail.com>

On Mon, 22 Nov 2021 at 18:20, Robert Dahlem <Robert.Dahlem at gmx.net> wrote:
> I could always install Quagga or Bird and use OSPF. But that might be a
> bit of an overkill for a simple active/passive failover.

RIP, OSPF, BGP, EIGRP. Either can work. And would IMO be sufficiently
simple to set up and maintain. Either way, you will need a daemon to
maintain monitoring of the packets being able to flow over the tunnel.
If you select a routing demon or another daemon. ... take a pick.


From mike at pineview.net  Wed Nov 24 12:04:20 2021
From: mike at pineview.net (Mike O'Connor)
Date: Wed, 24 Nov 2021 22:34:20 +1030
Subject: Dynamic routes
In-Reply-To: <CA+cYV6s2jMh4z5eUz-AAY5MO9CBpszWZtiUbkWY8XeFw7pLaFg@mail.gmail.com>
References: <3c70475b-64e5-acf7-819c-94f721fed8b6@gmx.net>
 <CA+cYV6s2jMh4z5eUz-AAY5MO9CBpszWZtiUbkWY8XeFw7pLaFg@mail.gmail.com>
Message-ID: <3171350c-543d-92c9-c897-3cb752a811cf@pineview.net>

On 24/11/21 9:48 pm, Chriztoffer Hansen wrote:
> On Mon, 22 Nov 2021 at 18:20, Robert Dahlem <Robert.Dahlem at gmx.net> wrote:
>> I could always install Quagga or Bird and use OSPF. But that might be a
>> bit of an overkill for a simple active/passive failover.
> RIP, OSPF, BGP, EIGRP. Either can work. And would IMO be sufficiently
> simple to set up and maintain. Either way, you will need a daemon to
> maintain monitoring of the packets being able to flow over the tunnel.
> If you select a routing demon or another daemon. ... take a pick.
>
Correct, not the job of Wireguard.


Cheers

Mike


From Robert.Dahlem at gmx.net  Wed Nov 24 15:07:22 2021
From: Robert.Dahlem at gmx.net (Robert Dahlem)
Date: Wed, 24 Nov 2021 16:07:22 +0100
Subject: Dynamic routes
In-Reply-To: <3171350c-543d-92c9-c897-3cb752a811cf@pineview.net>
References: <3c70475b-64e5-acf7-819c-94f721fed8b6@gmx.net>
 <CA+cYV6s2jMh4z5eUz-AAY5MO9CBpszWZtiUbkWY8XeFw7pLaFg@mail.gmail.com>
 <3171350c-543d-92c9-c897-3cb752a811cf@pineview.net>
Message-ID: <f9e5c379-f199-e362-9ad7-3df580e45f93@gmx.net>

Hi,

On 24.11.2021 13:04, Mike O'Connor wrote:

> On 24/11/21 9:48 pm, Chriztoffer Hansen wrote:

>>> I could always install Quagga or Bird and use OSPF. But that might be a
>>> bit of an overkill for a simple active/passive failover.

>> RIP, OSPF, BGP, EIGRP. Either can work. And would IMO be sufficiently
>> simple to set up and maintain. Either way, you will need a daemon to
>> maintain monitoring of the packets being able to flow over the tunnel.
>> If you select a routing demon or another daemon. ... take a pick.

> Correct, not the job of Wireguard.

I can live with that. Thanks for clearing up.

Regards,
Robert


From bandry at ut1.org  Wed Nov 24 15:21:23 2021
From: bandry at ut1.org (Bruno UT1)
Date: Wed, 24 Nov 2021 16:21:23 +0100
Subject: [Windows Client] Out of date Title scare my users
Message-ID: <1da227e6-5b12-40ca-a6de-3117835a2f16@ut1.org>

Hi,

Thank you again for your great work.

I have a suggestion for the Windows Client (maybe applicable for others).

I install Wireguard in my university on about 500 computers in 2 phases:

Phase 1 : validation

Phase 2 : production

So my end users have not, most of the time, the last version. They have 
"Out of date" in the Wireguard window title and some of them call the 
helpdesk, thinking they have a problem.

I can't deploy all versions as soon as they are available, I can't 
activate automatic update and my users have no admin rights. So my 
suggestion is to add a new admin registry key to hide update, or not 
check them at all.

Is that possible ?

Regards


From Jason at zx2c4.com  Wed Nov 24 15:42:59 2021
From: Jason at zx2c4.com (Jason A. Donenfeld)
Date: Wed, 24 Nov 2021 16:42:59 +0100
Subject: [Windows Client] Out of date Title scare my users
In-Reply-To: <1da227e6-5b12-40ca-a6de-3117835a2f16@ut1.org>
References: <1da227e6-5b12-40ca-a6de-3117835a2f16@ut1.org>
Message-ID: <CAHmME9oqvV9E9cVXeF5nVpumE0fJgdASaBqG6aCPyS-NLq9rXQ@mail.gmail.com>

I agree the situation is a bit ridiculous. I'll revert the change that
added this: https://git.zx2c4.com/wireguard-windows/commit/?id=82129ba288f7561c89bb80e04841ffb46bc29889

I'm CCing Diab, who originally requested the change, in case he wants
to argue with you about it. But in the absence of that, I'll revert.

From peter at libassi.se  Thu Nov 25 05:16:28 2021
From: peter at libassi.se (Peter Libassi)
Date: Thu, 25 Nov 2021 06:16:28 +0100
Subject: [ANNOUNCE] wireguard-freebsd snapshot v0.0.20211105 is available
Message-ID: <EFAEC1BE-AC42-4656-9B0F-6B2FAE2FFBD4@libassi.se>


I've done some preliminary tests with the new if_wg kernel module on amd64 with freebsd 13-p4 and 14.

It works fine AFAICT. My previous issue's with wireguard-freebsd are also resolved. 

Good work!

Thanks
Peter

From liuhangbin at gmail.com  Thu Nov 25 12:34:28 2021
From: liuhangbin at gmail.com (Hangbin Liu)
Date: Thu, 25 Nov 2021 20:34:28 +0800
Subject: [PATCH wireguard] wireguard: selftests: refactor the test
 structure
In-Reply-To: <CAHmME9pNFe7grqhW7=YQgRq10g4K5bqVuJrq0HonEVNbQSRuYg@mail.gmail.com>
References: <20211116081359.975655-1-liuhangbin@gmail.com>
 <CAHmME9pNFe7grqhW7=YQgRq10g4K5bqVuJrq0HonEVNbQSRuYg@mail.gmail.com>
Message-ID: <YZ+C1MWdWQvd66ic@Laptop-X1>

On Tue, Nov 16, 2021 at 03:35:40PM +0100, Jason A. Donenfeld wrote:
> Hi Hangbin,
> 
> I don't know how interested in this I am. Splitting things into two
> files means more confusing maintenance, and categorizing sections
> strictly into functions means there's more overhead when adding tests
> (e.g. "where do they fit?"), because the categories you've chosen are
> fairly broad, rather than being functions for each specific test. I'd
> be more amenable to something _entirely_ granular, because that'd be
> consistent, or what we have now, which is just linear. Full
> granularity, though, has its own downsides, of increased clutter.
> Alternatively, if you'd like to add some comments around the different
> areas to better document what's happening, perhaps that'd accomplish
> the same thing as this patch.
> 

Hi Jason,

May be my timezone is not very fit for yours. So I will copy my IRC replies
in the mail to moving on our kselftest topic.

The reason I did this patch is because I want to make the test more clear
and able to run each test case separately. My though is to make the
wireguard test looks like tools/testing/selftests/net/fib_tests.sh.(Of course
this could be discussed).

Because the linear structure makes reader hard to find out what test it does.
The function name in my current patch is also a little broad to look, which
could to be updated. After updating, I'd like to make the test has 2 parts,
functional tests and regression test. Functional tests for big part of function
tests and regression test for small specific issues.

BTW, one downside about current linear structure I think is that when someone
want to add a new test, he need to read through the whole test to know that
kind of topology at last. But with function structure, when we want to add a
new test. We can just do like:
1. set up basic topology
2. configure to specific topo for testing, or just skip the first step and
   configure to specific topo directly.
3. Do test
4. Clean up environment or reset to basic topology

I think this would make adding new test case easier. What do you think?

Thanks
Hangbin

From lazerl0rd at thezest.dev  Thu Nov 25 14:23:12 2021
From: lazerl0rd at thezest.dev (lazerl0rd at thezest.dev)
Date: Thu, 25 Nov 2021 14:23:12 +0000
Subject: [Windows Client] Out of date Title scare my users
In-Reply-To: <CAHmME9oqvV9E9cVXeF5nVpumE0fJgdASaBqG6aCPyS-NLq9rXQ@mail.gmail.com>
References: <1da227e6-5b12-40ca-a6de-3117835a2f16@ut1.org>
 <CAHmME9oqvV9E9cVXeF5nVpumE0fJgdASaBqG6aCPyS-NLq9rXQ@mail.gmail.com>
Message-ID: <ccf4971633fdd62807ff51ddf7de1299@purelymail.com>

Dear Bruno,

Whilst I understand the frustration that having hundreds of users can 
cause, I don't believe simply reverting the change [as proposed by 
Jason] is the correct solution. I've come up with a few alternative 
solutions, but before I present them I'd just like to give a brief 
introduction into why I requested that change in the first place.

WireGuard on Windows exclusively provides a GUI to users of the 
Administrators group, as well as a limited GUI to users of the Network 
Configuration Operators group when the `LimitedOperatorUI` DWORD is set. 
The latter is helpful for users who wish to separate their personal and 
administrator accounts (to protect themselves against the plethora of 
UAC exploits, amongst other security issues) where otherwise the user 
would have to switch accounts to switch tunnels. However, the GUI shown 
to Network Configuration Operators lacked any information about updates. 
This lead to users in such setups to not be informed about any updates 
unless they switched out to the Administrator account and or kept an eye 
on the releases online. This is quite a problem as users could be 
running ancient versions of WireGuard for relatively long periods of 
time without the knowledge that they are doing so (some users may even 
assume WireGuard automatically updates). As such, I asked Jason if he 
could add the ability for non-admins to at least be informed of an 
update which lead to where we are today.

After speaking to Jason "off the mailing list", he stated he wouldn't 
like to add any more configuration options (via the Registry or within 
the GUI) nor any metadata to updates so bearing that in mind I came up 
with a few alternatives:

1) Rewording the update prompt for non-admins to appear less 
"aggressive". Currently, the prompt is "Please ask the system 
administrator to update." but this could be changed to something along 
the lines of "There is an update available. The system administrator 
will update when necessary." which should reduce most, if not all, users 
from contacting you unnecessarily. I can throw up a patch for this if 
Jason agrees.

2) Avoiding users seeing the UI at all, where unnecessary. If your users 
do not need *control* of the WireGuard configuration, then avoiding 
showing them the UI altogether could be an option. I don't know your 
system as well as you do, of course, so I can't assure that this 
solution is valid. However, having hundreds of users as Network 
Configuration Operators sounds a little "worrying" to me.

3) Showing an even more limited UI for unprivileged users. If the users 
still need some form of UI, then an even more limited UI could be 
presented to users not part of the Administrators nor the Network 
Configuration Operators groups. This would lack any form of control, and 
could still be under the same `LimitedOperatorUI` Registry DWORD, or not 
if is deemed "safe enough for the masses". If it is, you could say the 
semantics refer to "Limited [User or Network] Operator UI".

4) Updates could be hidden from the UI for N days after an update or N 
updates (preferably two in this case, so that it doesn't pile up) for 
Network Configuration Operators. This provides you [and any other 
sysadmins] with a "buffer zone" to apply the updates before users 
contact you about them. This could also be teamed up with 1) to further 
reduce the likelihood of users contacting you. I'm not a large fan of 
this "solution", however, since WireGuard for Windows lacks any metadata 
to differentiate important and optional updates which can lead to a 
security patch or critical bug-fix being ignored for some time.

5) Creating a separate group which are able to switch tunnels. For users 
who just need the GUI to switch tunnels, having a group specific to such 
behavior named something along the lines of "WireGuard Operators" could 
be helpful.

Hopefully at least one of these suffices for you so that we can meet a 
mid-point of sorts that matches both your criteria as well as my own.

Thank for your time,
Diab Neiroukh

PS: Whilst it may seem a pain, I believe that a balance should be 
achieved between the sysadmins and users where if the former forgets to 
apply an update "for too long" then the users contact them as a 
reminder. After all, we're all humans and we do forget sometimes. The 
solutions 1) - with a prompt such as "There is an update available. The 
system administrator should update soon." - and 4) match up to this 
quite nicely.

On 2021-11-24 15:42, Jason A. Donenfeld wrote:
> I agree the situation is a bit ridiculous. I'll revert the change that
> added this:
> https://git.zx2c4.com/wireguard-windows/commit/?id=82129ba288f7561c89bb80e04841ffb46bc29889
> 
> I'm CCing Diab, who originally requested the change, in case he wants
> to argue with you about it. But in the absence of that, I'll revert.

From bandry at ut1.org  Thu Nov 25 16:07:33 2021
From: bandry at ut1.org (Bruno UT1)
Date: Thu, 25 Nov 2021 17:07:33 +0100
Subject: [Windows Client] Out of date Title scare my users
In-Reply-To: <ccf4971633fdd62807ff51ddf7de1299@purelymail.com>
References: <1da227e6-5b12-40ca-a6de-3117835a2f16@ut1.org>
 <CAHmME9oqvV9E9cVXeF5nVpumE0fJgdASaBqG6aCPyS-NLq9rXQ@mail.gmail.com>
 <ccf4971633fdd62807ff51ddf7de1299@purelymail.com>
Message-ID: <406a9568-8f70-b222-4e08-23fc5884847e@ut1.org>

Hi Diab,

Thank you for the detailed explanation.

My suggestion to add a new registry key seemed easy, but I understand a 
limited number of options is important.

 From your alternatives :

1) Probably the easiest to implement. It's not my favorite, but it would 
be better no doubt.

2) I used Network Configuration Operators for everyone because no other 
option was available. It's not great as you said, but users need to 
start and stop the VPN sometimes. I can't remove all rights.

3) Do you suggest a "read only" interface ? I looks like the opposite of 
what I need, the less users see the better.

4) It seems pretty complicate for a small improvement. Like you I'm not 
fan, but why not if Jason like it. It can but combined with the first 
option.

5) I like this option, but I supposed it wasn't possible. If it is, a 
group that can only access to the list of tunnels and start or stop them 
is a good solution for my environment. Removing Network Configuration 
Operators rights would be an improvement.

To summarize, the option 5 is what I'm looking for from the beginning. 
But if it's to complicate (or impossible) to do, the first one looks 
like a good start.

Rewording can be done easily and quickly (before a better solution is 
chosen, if possible). In that case, I indicate that I mainly use the 
french localization (it explains my poor english language level). Maybe 
the french version seems more aggressive that the english one? I don't 
know who leads this language translation, but I suggest him (or her) to 
change the Windows title "Obsol?te" (out of date) to something softer, 
or nothing in the title just the update tab.

Thank for your time too,

Bruno ANDRY


Le 25/11/2021 ? 15:23, lazerl0rd at thezest.dev a ?crit?:
> Dear Bruno,
>
> Whilst I understand the frustration that having hundreds of users can 
> cause, I don't believe simply reverting the change [as proposed by 
> Jason] is the correct solution. I've come up with a few alternative 
> solutions, but before I present them I'd just like to give a brief 
> introduction into why I requested that change in the first place.
>
> WireGuard on Windows exclusively provides a GUI to users of the 
> Administrators group, as well as a limited GUI to users of the Network 
> Configuration Operators group when the `LimitedOperatorUI` DWORD is 
> set. The latter is helpful for users who wish to separate their 
> personal and administrator accounts (to protect themselves against the 
> plethora of UAC exploits, amongst other security issues) where 
> otherwise the user would have to switch accounts to switch tunnels. 
> However, the GUI shown to Network Configuration Operators lacked any 
> information about updates. This lead to users in such setups to not be 
> informed about any updates unless they switched out to the 
> Administrator account and or kept an eye on the releases online. This 
> is quite a problem as users could be running ancient versions of 
> WireGuard for relatively long periods of time without the knowledge 
> that they are doing so (some users may even assume WireGuard 
> automatically updates). As such, I asked Jason if he could add the 
> ability for non-admins to at least be informed of an update which lead 
> to where we are today.
>
> After speaking to Jason "off the mailing list", he stated he wouldn't 
> like to add any more configuration options (via the Registry or within 
> the GUI) nor any metadata to updates so bearing that in mind I came up 
> with a few alternatives:
>
> 1) Rewording the update prompt for non-admins to appear less 
> "aggressive". Currently, the prompt is "Please ask the system 
> administrator to update." but this could be changed to something along 
> the lines of "There is an update available. The system administrator 
> will update when necessary." which should reduce most, if not all, 
> users from contacting you unnecessarily. I can throw up a patch for 
> this if Jason agrees.
>
> 2) Avoiding users seeing the UI at all, where unnecessary. If your 
> users do not need *control* of the WireGuard configuration, then 
> avoiding showing them the UI altogether could be an option. I don't 
> know your system as well as you do, of course, so I can't assure that 
> this solution is valid. However, having hundreds of users as Network 
> Configuration Operators sounds a little "worrying" to me.
>
> 3) Showing an even more limited UI for unprivileged users. If the 
> users still need some form of UI, then an even more limited UI could 
> be presented to users not part of the Administrators nor the Network 
> Configuration Operators groups. This would lack any form of control, 
> and could still be under the same `LimitedOperatorUI` Registry DWORD, 
> or not if is deemed "safe enough for the masses". If it is, you could 
> say the semantics refer to "Limited [User or Network] Operator UI".
>
> 4) Updates could be hidden from the UI for N days after an update or N 
> updates (preferably two in this case, so that it doesn't pile up) for 
> Network Configuration Operators. This provides you [and any other 
> sysadmins] with a "buffer zone" to apply the updates before users 
> contact you about them. This could also be teamed up with 1) to 
> further reduce the likelihood of users contacting you. I'm not a large 
> fan of this "solution", however, since WireGuard for Windows lacks any 
> metadata to differentiate important and optional updates which can 
> lead to a security patch or critical bug-fix being ignored for some time.
>
> 5) Creating a separate group which are able to switch tunnels. For 
> users who just need the GUI to switch tunnels, having a group specific 
> to such behavior named something along the lines of "WireGuard 
> Operators" could be helpful.
>
> Hopefully at least one of these suffices for you so that we can meet a 
> mid-point of sorts that matches both your criteria as well as my own.
>
> Thank for your time,
> Diab Neiroukh
>
> PS: Whilst it may seem a pain, I believe that a balance should be 
> achieved between the sysadmins and users where if the former forgets 
> to apply an update "for too long" then the users contact them as a 
> reminder. After all, we're all humans and we do forget sometimes. The 
> solutions 1) - with a prompt such as "There is an update available. 
> The system administrator should update soon." - and 4) match up to 
> this quite nicely.
>
> On 2021-11-24 15:42, Jason A. Donenfeld wrote:
>> I agree the situation is a bit ridiculous. I'll revert the change that
>> added this:
>> https://git.zx2c4.com/wireguard-windows/commit/?id=82129ba288f7561c89bb80e04841ffb46bc29889 
>>
>>
>> I'm CCing Diab, who originally requested the change, in case he wants
>> to argue with you about it. But in the absence of that, I'll revert.
>
> !DSPAM:5,619f9c6d262291485912835!
>
>

From Jason at zx2c4.com  Thu Nov 25 16:27:30 2021
From: Jason at zx2c4.com (Jason A. Donenfeld)
Date: Thu, 25 Nov 2021 17:27:30 +0100
Subject: [Windows Client] Out of date Title scare my users
In-Reply-To: <406a9568-8f70-b222-4e08-23fc5884847e@ut1.org>
References: <1da227e6-5b12-40ca-a6de-3117835a2f16@ut1.org>
 <CAHmME9oqvV9E9cVXeF5nVpumE0fJgdASaBqG6aCPyS-NLq9rXQ@mail.gmail.com>
 <ccf4971633fdd62807ff51ddf7de1299@purelymail.com>
 <406a9568-8f70-b222-4e08-23fc5884847e@ut1.org>
Message-ID: <CAHmME9roq+Qp7TFA4h3KZDuTd6DYJ68y2j9__MrX3+FDdtwVbA@mail.gmail.com>

> nothing in the title just the update tab.

If you find that an acceptable compromise, it's fine with me.

From Jason at zx2c4.com  Thu Nov 25 16:38:20 2021
From: Jason at zx2c4.com (Jason A. Donenfeld)
Date: Thu, 25 Nov 2021 17:38:20 +0100
Subject: [PATCH wireguard-go] conn: linux: fix incorrect endpoint address
In-Reply-To: <fef7daf2-adf7-e042-4977-46fda04aa07d@mnus.de>
References: <20211123190331.101346-1-minus@mnus.de>
 <CAHmME9pEf9=6C3eAXfanftBt4LJ4viRszQc5oG3Jt9H7Fxd68A@mail.gmail.com>
 <fef7daf2-adf7-e042-4977-46fda04aa07d@mnus.de>
Message-ID: <CAHmME9qMLM=HedzvADriiWLZXbo0yLoxt=Nk9B=PGcwY9kn0eg@mail.gmail.com>

Hey Minus,

I'm not seeing what could be causing this. Could you triple check that
it really is busted on Debian 8?

Jason

From aplsms at gmail.com  Fri Nov 26 07:33:18 2021
From: aplsms at gmail.com (Andrii Petrenko)
Date: Thu, 25 Nov 2021 23:33:18 -0800
Subject: Wireguard DNS error. 
Message-ID: <FDA1FABB-8432-4259-BFF5-ED3E7EB632C3@gmail.com>

Hello, I have a problem with Wireguard DNS proxy.

Issue looks like: 

Trough DNS proxy

$ host presence.teams.microsoft.com.                                                                                                                                            

Host presence.teams.microsoft.com not found: 2(SERVFAIL)

Trough the upstream DNS server:

$ host presence.teams.microsoft.com. 10.10.10.1 
Using domain server:
Name: 10.10.10.1
Address: 10.10.10.1#53
Aliases:

presence.teams.microsoft.com is an alias for presence.services.sfb.trafficmanager.net.
presence.services.sfb.trafficmanager.net is an alias for a-ups-presence0-prod-azsc.eastus2.cloudapp.azure.com.
a-ups-presence0-prod-azsc.eastus2.cloudapp.azure.com has address 52.114.142.202

Logs from server: 

time="2021-11-26T02:33:56Z" level=debug msg="dns query: dns query for: presence.teams.microsoft.com.:1:1" file="server.go:70"
time="2021-11-26T02:33:56Z" level=error msg="failed lookup record with error: dns: overflowing header size\n;; opcode: QUERY, status: NOERROR, id: 53952\n;; flags: rd; QUERY: 1, ANSWER: 0, AUTHORITY: 0, ADDITIONAL: 0\n\n;; QUESTION SECTION:\n;presence.teams.microsoft.com.\tIN\t A\n" file="server.go:76"

Another error:

$ host ocsp2.apple.com
Host ocsp2.apple.com not found: 2(SERVFAIL)

$ host ocsp2.apple.com 10.10.10.1
Using domain server:
Name: 10.10.10.1
Address: 10.10.10.1#53
Aliases:

ocsp2.apple.com is an alias for ocsp2-lb.apple.com.akadns.net.
ocsp2-lb.apple.com.akadns.net is an alias for ocsp2.g.aaplimg.com.
ocsp2.g.aaplimg.com has address 17.253.5.203
ocsp2.g.aaplimg.com has address 17.253.1.201
ocsp2.g.aaplimg.com has IPv6 address 2620:149:a00:f000::5
ocsp2.g.aaplimg.com has IPv6 address 2620:149:a1c:f000::1


time="2021-11-26T02:32:17Z" level=debug msg="dns query: dns query for: ocsp2.apple.com.:1:1" file="server.go:70"
time="2021-11-26T02:32:17Z" level=error msg="failed lookup record with error: dns: overflow unpacking uint32\n;; opcode: QUERY, status: NOERROR, id: 18718\n;; flags: rd; QUERY: 1, ANSWER: 0, AUTHORITY: 0, ADDITIONAL: 0\n\n;; QUESTION SECTION:\n;ocsp2.apple.com.\tIN\t A\n" file="server.go:76?


How to fix this problem? Please advise.

Thank you,
---
Andrii Petrenko
aplsms at gmail.com	


From Jason at zx2c4.com  Fri Nov 26 08:47:52 2021
From: Jason at zx2c4.com (Jason A. Donenfeld)
Date: Fri, 26 Nov 2021 09:47:52 +0100
Subject: [Windows Client] Out of date Title scare my users
In-Reply-To: <CAHmME9roq+Qp7TFA4h3KZDuTd6DYJ68y2j9__MrX3+FDdtwVbA@mail.gmail.com>
References: <1da227e6-5b12-40ca-a6de-3117835a2f16@ut1.org>
 <CAHmME9oqvV9E9cVXeF5nVpumE0fJgdASaBqG6aCPyS-NLq9rXQ@mail.gmail.com>
 <ccf4971633fdd62807ff51ddf7de1299@purelymail.com>
 <406a9568-8f70-b222-4e08-23fc5884847e@ut1.org>
 <CAHmME9roq+Qp7TFA4h3KZDuTd6DYJ68y2j9__MrX3+FDdtwVbA@mail.gmail.com>
Message-ID: <CAHmME9rwFyJ4W3nnbTWq0e0X_P_FOQx6Z3kcuUhcYuYjkc0atQ@mail.gmail.com>

https://git.zx2c4.com/wireguard-windows/commit/?id=8120d07dd5fc9a5e545419fe13490086ce920f31

Is this okay with both of you?

From lazerl0rd at thezest.dev  Fri Nov 26 09:17:38 2021
From: lazerl0rd at thezest.dev (lazerl0rd at thezest.dev)
Date: Fri, 26 Nov 2021 09:17:38 +0000
Subject: [Windows Client] Out of date Title scare my users
In-Reply-To: <CAHmME9rwFyJ4W3nnbTWq0e0X_P_FOQx6Z3kcuUhcYuYjkc0atQ@mail.gmail.com>
References: <1da227e6-5b12-40ca-a6de-3117835a2f16@ut1.org>
 <CAHmME9oqvV9E9cVXeF5nVpumE0fJgdASaBqG6aCPyS-NLq9rXQ@mail.gmail.com>
 <ccf4971633fdd62807ff51ddf7de1299@purelymail.com>
 <406a9568-8f70-b222-4e08-23fc5884847e@ut1.org>
 <CAHmME9roq+Qp7TFA4h3KZDuTd6DYJ68y2j9__MrX3+FDdtwVbA@mail.gmail.com>
 <CAHmME9rwFyJ4W3nnbTWq0e0X_P_FOQx6Z3kcuUhcYuYjkc0atQ@mail.gmail.com>
Message-ID: <2b0136d9581406195d542696db53d8cd@purelymail.com>

I assume you've chosen the "reword" route instead of any larger changes, 
for the better or worse (though as Bruno said, it would be great if 5) 
is considered somewhere down the line).

Since this route was chosen, I suggest that we also reword the update 
prompt itself as I feel that is equally responsible for users "freaking 
out". After all, it is literally telling users to contact their sysadmin 
instantly for each update. I propose something along the lines of the 
following patch (though I guess il8n will be a bit of a pain):

```
 From 76ea8a81cf327527089bfea8209bf4f2faa1b6cf Mon Sep 17 00:00:00 2001
 From: Diab Neiroukh <lazerl0rd at thezest.dev>
Date: Fri, 26 Nov 2021 09:12:15 +0000
Subject: [PATCH] ui: Don't explicitly tell users to contact their 
sysadmin for
  updates.

The wording used here practically told users to instantly contact their
system administrators for every update. We can reword it to instead to
implicitly suggest that they contact their system administrator if
the update has not been applied for "a relatively long time".

Signed-off-by: Diab Neiroukh <lazerl0rd at thezest.dev>
---
  ui/updatepage.go | 2 +-
  zgotext.go       | 2 +-
  2 files changed, 2 insertions(+), 2 deletions(-)

diff --git a/ui/updatepage.go b/ui/updatepage.go
index 96fc87f3..76a8dced 100644
--- a/ui/updatepage.go
+++ b/ui/updatepage.go
@@ -65,7 +65,7 @@ func NewUpdatePage() (*UpdatePage, error) {
  	button.SetText(l18n.Sprintf("Update Now"))

  	if !IsAdmin {
-		button.SetText(l18n.Sprintf("Please ask the system administrator to 
update."))
+		button.SetText(l18n.Sprintf("There is an update available. The system 
administrator should update soon."))
  		button.SetEnabled(false)
  		status.SetText(l18n.Sprintf("Status: Waiting for administrator"))
  	}
diff --git a/zgotext.go b/zgotext.go
index efbb9a80..e35974aa 100644
--- a/zgotext.go
+++ b/zgotext.go
@@ -235,7 +235,7 @@ var messageKeyToIndex = map[string]int{
  	"Packet with invalid IP version from %v":                              
  215,
  	"Peer":                                                                
  100,
  	"Persistent keepalive:":                                               
  54,
-	"Please ask the system administrator to update.":                      
  275,
+	"There is an update available. The system administrator should update 
soon.":                         275,
  	"Preshared key:":                                                      
  51,
  	"Protocol version must be 1":                                          
  85,
  	"Public key:":                                                         
  46,
-- 
2.34.0
```

On 2021-11-26 08:47, Jason A. Donenfeld wrote:
> https://git.zx2c4.com/wireguard-windows/commit/?id=8120d07dd5fc9a5e545419fe13490086ce920f31
> 
> Is this okay with both of you?

From lazerl0rd at thezest.dev  Thu Nov 25 13:34:16 2021
From: lazerl0rd at thezest.dev (Diab Neiroukh)
Date: Thu, 25 Nov 2021 13:34:16 +0000
Subject: [Windows Client] Out of date Title scare my users
In-Reply-To: <CAHmME9oqvV9E9cVXeF5nVpumE0fJgdASaBqG6aCPyS-NLq9rXQ@mail.gmail.com>
References: <1da227e6-5b12-40ca-a6de-3117835a2f16@ut1.org>
 <CAHmME9oqvV9E9cVXeF5nVpumE0fJgdASaBqG6aCPyS-NLq9rXQ@mail.gmail.com>
Message-ID: <41R43R.3PB4SBRZN49H2@thezest.dev>

Dear Bruno,

Whilst I understand the frustration that having hundreds of users can 
cause, I don't believe simply reverting the change [as proposed by 
Jason] is the correct solution. I've come up with a few alternative 
solutions, but before I present them I'd just like to give a brief 
introduction into why I requested that change in the first place.

WireGuard on Windows exclusively provides a GUI to users of the 
Administrators group, as well as a limited GUI to users of the Network 
Configuration Operators group when the `LimitedOperatorUI` DWORD is 
set. The latter is helpful for users who wish to separate their 
personal and administrator accounts (to protect themselves against the 
plethora of UAC exploits, amongst other security issues) where 
otherwise the user would have to switch accounts to switch tunnels. 
However, the GUI shown to Network Configuration Operators lacked any 
information about updates. This lead to users in such setups to not be 
informed about any updates unless they switched out to the 
Administrator account and or kept an eye on the releases online. This 
is quite a problem as users could be running ancient versions of 
WireGuard for relatively long periods of time without the knowledge 
that they are doing so (some users may even assume WireGuard 
automatically updates). As such, I asked Jason if he could add the 
ability for non-admins to at least be informed of an update which lead 
to where we are today.

After speaking to Jason "off the mailing list", he stated he wouldn't 
like to add any more configuration options (via the Registry or within 
the GUI) nor any metadata to updates so bearing that in mind I came up 
with a few alternatives:

1) Rewording the update prompt for non-admins to appear less 
"aggressive". Currently, the prompt is "Please ask the system 
administrator to update." but this could be changed to something along 
the lines of "There is an update available. The system administrator 
will update when necessary." which should reduce most, if not all, 
users from contacting you unnecessarily. I can throw up a patch for 
this if Jason agrees.

2) Avoiding users seeing the UI at all, where unnecessary. If your 
users do not need *control* of the WireGuard configuration, then 
avoiding showing them the UI altogether could be an option. I don't 
know your system as well as you do, of course, so I can't assure that 
this solution is valid. However, having hundreds of users as Network 
Configuration Operators sounds a little "worrying" to me.

3) Showing an even more limited UI for unprivileged users. If the users 
still need some form of UI, then an even more limited UI could be 
presented to users not part of the Administrators nor the Network 
Configuration Operators groups. This would lack any form of control, 
and could still be under the same `LimitedOperatorUI` Registry DWORD, 
or not if is deemed "safe enough for the masses". If it is, you could 
say the semantics refer to "Limited [User or Network] Operator UI".

4) Updates could be hidden from the UI for N days after an update or N 
updates (preferably two in this case, so that it doesn't pile up) for 
Network Configuration Operators. This provides you [and any other 
sysadmins] with a "buffer zone" to apply the updates before users 
contact you about them. This could also be teamed up with 1) to further 
reduce the likelihood of users contacting you. I'm not a large fan of 
this "solution", however, since WireGuard for Windows lacks any 
metadata to differentiate important and optional updates which can lead 
to a security patch or critical bug-fix being ignored for some time.

5) Creating a separate group which are able to switch tunnels. For 
users who just need the GUI to switch tunnels, having a group specific 
to such behaviour named something along the lines of "WireGuard 
Operators" could be helpful.

Hopefully at least one of these suffices for you so that we can meet a 
mid-point of sorts that matches both your criteria as well as my own.

PS: Whilst it may seem a pain, I believe that a balance should be 
achieved between the sysadmins and users where if the former forgets to 
apply an update "for too long" then the users contact them as a 
reminder. After all, we're all humans and we do forget sometimes. The 
solutions 1) - with a prompt such as "There is an update available. The 
system administrator should update soon." - and 4) match up to this 
quite nicely.

Thank for your time,
Diab Neiroukh

On Wed, Nov 24 2021 at 16:42:59 +0100, Jason A. Donenfeld 
<Jason at zx2c4.com> wrote:
> I agree the situation is a bit ridiculous. I'll revert the change that
> added this: 
> https://git.zx2c4.com/wireguard-windows/commit/?id=82129ba288f7561c89bb80e04841ffb46bc29889
> 
> I'm CCing Diab, who originally requested the change, in case he wants
> to argue with you about it. But in the absence of that, I'll revert.



From ceidem at ceidem.com  Thu Nov 25 18:41:22 2021
From: ceidem at ceidem.com (Chris Eidem)
Date: Thu, 25 Nov 2021 12:41:22 -0600
Subject: Ubuntu client OpenBSD server
Message-ID: <d83162a4-05bf-22f1-f8de-7ec183ed9173@ceidem.com>

I have a Wireguard server set up on an OpenBSD 6.9 server with OpenBSD,
Android, iOS and Ubuntu clients. All clients are connecting, although
the Ubuntu clients only work when the PSK is not enabled in the wg0.conf
file. Has anyone else seen this behavior? Is there any information I
could provide to help figure out why the PSK isn't working on my Linux
clients?

I don't know if wg uses the system's TLS libraries, but if so, I suspect
that the fact gnutls and libressl don't play well together may have
something to do with it. But, that is a guess pulled from my fundament...

  ?- chris


From minus+lists at mnus.de  Mon Nov 29 00:24:06 2021
From: minus+lists at mnus.de (minus)
Date: Mon, 29 Nov 2021 01:24:06 +0100
Subject: wireguard-go fails to send/receive data
In-Reply-To: <CAHmME9qMLM=HedzvADriiWLZXbo0yLoxt=Nk9B=PGcwY9kn0eg@mail.gmail.com>
References: <20211123190331.101346-1-minus@mnus.de>
 <CAHmME9pEf9=6C3eAXfanftBt4LJ4viRszQc5oG3Jt9H7Fxd68A@mail.gmail.com>
 <fef7daf2-adf7-e042-4977-46fda04aa07d@mnus.de>
 <CAHmME9qMLM=HedzvADriiWLZXbo0yLoxt=Nk9B=PGcwY9kn0eg@mail.gmail.com>
Message-ID: <d3d1d445-128e-9acb-b483-53b675fd9591@mnus.de>

Hi Jason,

> I'm not seeing what could be causing this. Could you triple check that
> it really is busted on Debian 8?
I was mistaken, it has nothing to do with Debian 8 specifically. I could 
reproduce it on Arch as well.
When you bring up an interface via wireguard-go and start pinging 
there's a big chance that nothing at all will work starting with commit 
e3134bf6. I've attached a script to reproduce the problem. It doesn't 
always fail, so you may have to run it a few times.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: repro.sh
Type: application/x-shellscript
Size: 810 bytes
Desc: not available
URL: <http://lists.zx2c4.com/pipermail/wireguard/attachments/20211129/75cb0f65/attachment.bin>

From jaquilina at eagleeyet.net  Mon Nov 29 15:47:52 2021
From: jaquilina at eagleeyet.net (Jonathan Aquilina)
Date: Mon, 29 Nov 2021 15:47:52 +0000
Subject: Watchguard T35 site to site with wireguard tunnel
Message-ID: <VE1PR08MB58376D02F4068133D5B52F30A0669@VE1PR08MB5837.eurprd08.prod.outlook.com>

 Good Afternoon All,

I am just doing some brain storming as I am considering revamping my infrastructure.

As an edge device I have a watchguard T35. I am wondering if I could setup a site to site vpn from that device to a watchguard tunnel.

Regards,
Jonathan

From Jason at zx2c4.com  Mon Nov 29 17:34:34 2021
From: Jason at zx2c4.com (Jason A. Donenfeld)
Date: Mon, 29 Nov 2021 12:34:34 -0500
Subject: wireguard-go fails to send/receive data
In-Reply-To: <d3d1d445-128e-9acb-b483-53b675fd9591@mnus.de>
References: <20211123190331.101346-1-minus@mnus.de>
 <CAHmME9pEf9=6C3eAXfanftBt4LJ4viRszQc5oG3Jt9H7Fxd68A@mail.gmail.com>
 <fef7daf2-adf7-e042-4977-46fda04aa07d@mnus.de>
 <CAHmME9qMLM=HedzvADriiWLZXbo0yLoxt=Nk9B=PGcwY9kn0eg@mail.gmail.com>
 <d3d1d445-128e-9acb-b483-53b675fd9591@mnus.de>
Message-ID: <CAHmME9pAvi3P82yKnpfNXtOdTUxYoZasPGkeLpo3angnh7wFfw@mail.gmail.com>

Thanks for the script and the report
https://git.zx2c4.com/wireguard-go/commit/?id=2dd424e2d808703339688ff78e32ed30cd0dfe87

From mmakassikis at freebox.fr  Tue Nov 30 09:22:29 2021
From: mmakassikis at freebox.fr (Marios Makassikis)
Date: Tue, 30 Nov 2021 10:22:29 +0100
Subject: Ubuntu client OpenBSD server
In-Reply-To: <d83162a4-05bf-22f1-f8de-7ec183ed9173@ceidem.com>
References: <d83162a4-05bf-22f1-f8de-7ec183ed9173@ceidem.com>
Message-ID: <CAF6XXKWbYasBFCp5-Fyqp_g-13sXbZvRmVfvve7nKwi7vH60Ng@mail.gmail.com>

On Mon, Nov 29, 2021 at 4:45 PM Chris Eidem <ceidem at ceidem.com> wrote:
>
> I have a Wireguard server set up on an OpenBSD 6.9 server with OpenBSD,
> Android, iOS and Ubuntu clients. All clients are connecting, although
> the Ubuntu clients only work when the PSK is not enabled in the wg0.conf
> file. Has anyone else seen this behavior? Is there any information I
> could provide to help figure out why the PSK isn't working on my Linux
> clients?
>
> I don't know if wg uses the system's TLS libraries, but if so, I suspect
> that the fact gnutls and libressl don't play well together may have
> something to do with it. But, that is a guess pulled from my fundament...
>
>    - chris
>

wg doesn't use TLS libraries at all, so the issue is somewhere else.
Have you checked that the PSK is the same on both ends ?

Which kernel version are you using ?

Enabling debug logs on both ends may shed some light on what is
going on. On OpenBSD, enable using the command:
  ifconfig wgX debug
To disable:
  ifconfig wgX -debug

On Ubuntu, enable using:
  echo 'module wireguard +p' > /sys/kernel/debug/dynamic_debug/control

and disable with:
  echo 'module wireguard -p' > /sys/kernel/debug/dynamic_debug/control

From bandry at ut1.org  Tue Nov 30 10:12:09 2021
From: bandry at ut1.org (Bruno UT1)
Date: Tue, 30 Nov 2021 11:12:09 +0100
Subject: [Windows Client] Out of date Title scare my users
In-Reply-To: <2b0136d9581406195d542696db53d8cd@purelymail.com>
References: <1da227e6-5b12-40ca-a6de-3117835a2f16@ut1.org>
 <CAHmME9oqvV9E9cVXeF5nVpumE0fJgdASaBqG6aCPyS-NLq9rXQ@mail.gmail.com>
 <ccf4971633fdd62807ff51ddf7de1299@purelymail.com>
 <406a9568-8f70-b222-4e08-23fc5884847e@ut1.org>
 <CAHmME9roq+Qp7TFA4h3KZDuTd6DYJ68y2j9__MrX3+FDdtwVbA@mail.gmail.com>
 <CAHmME9rwFyJ4W3nnbTWq0e0X_P_FOQx6Z3kcuUhcYuYjkc0atQ@mail.gmail.com>
 <2b0136d9581406195d542696db53d8cd@purelymail.com>
Message-ID: <091c01e5-30bb-71d1-a863-999d6dbae7dd@ut1.org>

Hi,

I validate what Diab said. It's a good start, hoping option 5 will be 
possible later.

Thank you for changes.

Have a good day.

Le 26/11/2021 ? 10:17, lazerl0rd at thezest.dev a ?crit?:
> I assume you've chosen the "reword" route instead of any larger 
> changes, for the better or worse (though as Bruno said, it would be 
> great if 5) is considered somewhere down the line).
>
> Since this route was chosen, I suggest that we also reword the update 
> prompt itself as I feel that is equally responsible for users 
> "freaking out". After all, it is literally telling users to contact 
> their sysadmin instantly for each update. I propose something along 
> the lines of the following patch (though I guess il8n will be a bit of 
> a pain):
>
> ```
> From 76ea8a81cf327527089bfea8209bf4f2faa1b6cf Mon Sep 17 00:00:00 2001
> From: Diab Neiroukh <lazerl0rd at thezest.dev>
> Date: Fri, 26 Nov 2021 09:12:15 +0000
> Subject: [PATCH] ui: Don't explicitly tell users to contact their 
> sysadmin for
> ?updates.
>
> The wording used here practically told users to instantly contact their
> system administrators for every update. We can reword it to instead to
> implicitly suggest that they contact their system administrator if
> the update has not been applied for "a relatively long time".
>
> Signed-off-by: Diab Neiroukh <lazerl0rd at thezest.dev>
> ---
> ?ui/updatepage.go | 2 +-
> ?zgotext.go?????? | 2 +-
> ?2 files changed, 2 insertions(+), 2 deletions(-)
>
> diff --git a/ui/updatepage.go b/ui/updatepage.go
> index 96fc87f3..76a8dced 100644
> --- a/ui/updatepage.go
> +++ b/ui/updatepage.go
> @@ -65,7 +65,7 @@ func NewUpdatePage() (*UpdatePage, error) {
> ???? button.SetText(l18n.Sprintf("Update Now"))
>
> ???? if !IsAdmin {
> -??????? button.SetText(l18n.Sprintf("Please ask the system 
> administrator to update."))
> +??????? button.SetText(l18n.Sprintf("There is an update available. 
> The system administrator should update soon."))
> ???????? button.SetEnabled(false)
> ???????? status.SetText(l18n.Sprintf("Status: Waiting for 
> administrator"))
> ???? }
> diff --git a/zgotext.go b/zgotext.go
> index efbb9a80..e35974aa 100644
> --- a/zgotext.go
> +++ b/zgotext.go
> @@ -235,7 +235,7 @@ var messageKeyToIndex = map[string]int{
> ???? "Packet with invalid IP version from 
> %v":????????????????????????????? ?215,
> "Peer": ?100,
> ???? "Persistent 
> keepalive:":?????????????????????????????????????????????? ?54,
> -??? "Please ask the system administrator to 
> update.":????????????????????? ?275,
> +??? "There is an update available. The system administrator should 
> update soon.":???????????????????????? 275,
> ???? "Preshared 
> key:":????????????????????????????????????????????????????? ?51,
> ???? "Protocol version must be 
> 1":????????????????????????????????????????? ?85,
> ???? "Public key:": ?46,