From mdt at emdete.de  Mon Oct  4 05:30:44 2021
From: mdt at emdete.de (M. Dietrich)
Date: Mon, 04 Oct 2021 07:30:44 +0200
Subject: Behaviour on MTU problem
In-Reply-To: <08C6DA98-A534-451D-9007-FF8627C48065@gmail.com>
References: <1630347517.szy5mrrs16.astroid@morple.none>
 <08C6DA98-A534-451D-9007-FF8627C48065@gmail.com>
Message-ID: <1633325115.iotxkgm6j2.astroid@morple.none>

Quotation from Art Botterell at August 30, 2021 21:55:
> One reads about fragmentation-induced failures and about 
> various MTU values that did or didn't work in a particular 
> situation.  [...]  what we could look out for in order 
> automatically to detect MTU issues [...] Is there some sort 
> of a ?fragtest? network utility out there?

My request was neither about anekdotic problems with different 
MTU sizes nor any automatic detection of a correct size and 
test utilities.

i was asking why the kernel module is completly locking up if 
a MTU size problem is present.

best regards, Michael

From kristofmattei at outlook.com  Sun Oct  3 22:15:25 2021
From: kristofmattei at outlook.com (Kristof Mattei)
Date: Sun, 3 Oct 2021 22:15:25 +0000
Subject: [wireguard-apple] [iOS] 464xlat networks and On-demand roaming issue
Message-ID: <BY5PR04MB6833078C93B5AE04028574EEA2AD9@BY5PR04MB6833.namprd04.prod.outlook.com>

I have an issue with the wireguard-apple on 464xlat connecting to a DNS endpoint with both an A and an AAAA record.

The following line: https://git.zx2c4.com/wireguard-apple/tree/Sources/WireGuardKit/DNSResolver.swift#n81 causes WireGuard to prefer the IPv4 address.

Is there any reason why WireGuard prefers the IPv4 address? 

Why is this causing trouble? But this is what happens: 

When connecting to IPv6 the IPv4 address gets mapped to an IPv6 address which then acts as an IPv6->IPv4 proxy. The IP looks like [2607:7700:0:1a::17f3:f750]:51820.

This causes issues when roaming from my home WiFi (on which WireGuard is disabled) to cellular (on which WireGuard is set to on-Demand).

The initial connection that is set up for some reason does not work. There are reports about this on Reddit, e.g. https://www.reddit.com/r/WireGuard/comments/nk2o7m/anyone_got_it_working_with_tmobile_lte/

I can fix it by setting the endpoint to the actual IPv6 address, and then it works like a charm, but that fails when I connect to a non-IPv6 network.

Thanks,
Kristof


From xiyou.wangcong at gmail.com  Thu Oct  7 20:55:54 2021
From: xiyou.wangcong at gmail.com (Cong Wang)
Date: Thu, 7 Oct 2021 13:55:54 -0700
Subject: [Patch net] wireguard: preserve skb->mark on ingress side
In-Reply-To: <CAM_iQpV-JKrk8vaHDeD0pXaheN0APUxH5Lp+mGCM=_yZQ1hd4w@mail.gmail.com>
References: <20210928031938.17902-1-xiyou.wangcong@gmail.com>
 <CAHmME9rbfBHUnoifdQV6pOp8MHwowEp7ooOhV-JSJmanRzksLA@mail.gmail.com>
 <CAM_iQpV-JKrk8vaHDeD0pXaheN0APUxH5Lp+mGCM=_yZQ1hd4w@mail.gmail.com>
Message-ID: <CAM_iQpVNP2+7F59-6NDhXfE7qxU6APNdBu5UxcWKMaow05kytA@mail.gmail.com>

Hi, Jason

On Mon, Sep 27, 2021 at 8:27 PM Cong Wang <xiyou.wangcong at gmail.com> wrote:
>
> On Mon, Sep 27, 2021 at 8:22 PM Jason A. Donenfeld <Jason at zx2c4.com> wrote:
> >
> > Hi Cong,
> >
> > I'm not so sure this makes sense, as the inner packet is in fact
> > totally different. If you want to distinguish the ingress interface,
>
> The contents are definitely different, but skb itself is the same.
>
> Please also take a look at other tunnels, they all preserve this
> in similar ways, that is, comparing net namespaces. Any reason
> why wireguard is so different from other tunnels?

Any response?

Thanks.

From nima at kandoo.tech  Tue Oct  5 04:14:23 2021
From: nima at kandoo.tech (Nima Fatemi)
Date: Mon, 4 Oct 2021 21:14:23 -0700
Subject: macOS on-demand leaking during 'connecting' stage
Message-ID: <CABXzbRfw6wCik1MR-V1Gx-F1sN+5xGdSZYQJGMqGchYPA1qYaw@mail.gmail.com>

Hi there,

I was reconfiguring my network so I had a ping window open and
realized that wg on macOS leaks traffic right after disconnection
during the 'activating' phase. Figured I should give this list a heads
up.

Cheers,
Nima Fatemi (he/him)
President, Kandoo
Twitter: @mrphs
Phone: +1 202 318 1984

From uxDWzco-wg at moenia.de  Tue Oct  5 08:39:14 2021
From: uxDWzco-wg at moenia.de (uxDWzco-wg at moenia.de)
Date: Tue, 5 Oct 2021 10:39:14 +0200
Subject: performance: multiple clients on one interface?
Message-ID: <a208c85e-4835-cb77-003c-a040544ec968@moenia.de>

hi,

after have various tests run with 1:1 connections we want to expand it
to multiple connects to one system (linux-based).

due the limitations at least in linux wireguard-IFs can't be part of a
bridge-IF, but if we handle all connections with only one
wireguard-interface, we have to use a single udp-port for all connections...

using same port for all connections means, that for receiving encrypted
packets every configured key must be tried, until the right one is
found, or is this wrong?

so: how many connections are reasonable for a single device, without
running in to trouble due to the time trying all the keys?

or is there some internal optimization after have found a match by
filtering possible keys by src-addr/port, so the complete search is only
done at first connection-try?

it would be very helpful, to get some information on it here.

regards

j.

From Jason at zx2c4.com  Thu Oct  7 23:35:33 2021
From: Jason at zx2c4.com (Jason A. Donenfeld)
Date: Thu, 7 Oct 2021 17:35:33 -0600
Subject: Should we sunset Windows 7 support?
In-Reply-To: <CAHmME9rZHaDRRk2-c9kzhUapTRiVxybkwS2Ung_HgSCAVyOw4Q@mail.gmail.com>
References: <CAHmME9rW1e+rRc8q+-_eLR=U9OTn+Sn84ECJtGFY=Hjg8kzfEQ@mail.gmail.com>
 <K17360-33.1605029899@spamhaus.org>
 <CAHmME9pmKxGYmxxs0pLt=zt2igy6aR-g5xsB=2Zew6oWVY0Z3w@mail.gmail.com>
 <CABtXGiCs1fS3FHAwHL1zohuL1BfDhY7GhZhtLFfiqS2aJstr3g@mail.gmail.com>
 <20201112090313.GB3500@trivini.no>
 <CAH8yC8=azrpN=EOP9XsTDe2NwmqQquSpEuqidtS2rVFw8x+5Ew@mail.gmail.com>
 <CAHmME9rZHaDRRk2-c9kzhUapTRiVxybkwS2Ung_HgSCAVyOw4Q@mail.gmail.com>
Message-ID: <CAHmME9rZeksrVkbn1qOAFEtKs4F872N07EJNU5iHXSsGLi_TSg@mail.gmail.com>

Hey again,

About a year later, WireGuard on Windows keeps becoming more advanced
and integrated into the operating system, with better service
notifications, high speed multi-packet transmission, device arrival
notifications, software device management, and so on... the common
theme being that these are all made possible by Windows 8+ APIs. So
we're now carrying well over 1000 lines of Windows 7 compatibility
code, polyfills, and sometimes outright reimplementations. This is
nobody's idea of fun, and having lots of extra code (that receives
less and less testing) is usually a recipe for things to go wrong. The
added burden of Windows 7 slows progress on newer platforms.

So I really would like to sunset Windows 7 support at some point. The
party cannot go on forever.

However, according to [1], Windows 7 still makes up about 15% of
Windows installs. And I know for a fact that some pretty large
WireGuard deployments are running on Windows 7 boxes, sometimes
installed inside of ATMs... Whether or not that's a good idea, it is
happening. What to do?

My current thinking is to follow the Chromium project, or at least
watch attentively and see the fallout from their decision. After a 6
month extension due to the pandemic, Google is set to retire Windows 7
support in Chrome on January 15, 2022, which is exactly 100 days from
now. There are still some open questions, though: will Microsoft
follow by sunsetting Windows 7 support for Edge, or will they keep it
on life support even longer? Will Mozilla follow Chromium's decision
as well? Is tracking Chromium's decision sensible? It might be: web
standards tend to move awfully fast these days, and without an up to
date web browser, will Windows 7 users upgrade to Windows 11 [or
Linux]? It might not: the most important Windows 7 holdouts may well
be embedded machines that don't care about browsers anyway?

If anybody's thinking on this has evolved, I'd love to hear thoughts.

Thanks,
Jason

[1] https://gs.statcounter.com/windows-version-market-share/desktop/worldwide/

From me at aaronmdjones.net  Fri Oct  8 14:55:45 2021
From: me at aaronmdjones.net (Aaron Jones)
Date: Fri, 8 Oct 2021 14:55:45 +0000
Subject: performance: multiple clients on one interface?
In-Reply-To: <a208c85e-4835-cb77-003c-a040544ec968@moenia.de>
References: <a208c85e-4835-cb77-003c-a040544ec968@moenia.de>
Message-ID: <b53d4f44-b094-a5dd-14cf-55b846b17661@aaronmdjones.net>

On 05/10/2021 08:39, uxDWzco-wg at moenia.de wrote:
> using same port for all connections means, that for receiving encrypted
> packets every configured key must be tried, until the right one is
> found, or is this wrong?

Incorrect. The handshake establishes sender and receiver indexes; these
are reproduced in data packets so that the receiver does one hash table
lookup to determine the decryption key.

This is documented on https://www.wireguard.com/protocol/

> so: how many connections are reasonable for a single device, without
> running in to trouble due to the time trying all the keys?

Up to 1,048,576 peers per interface are supported, limited only by
bandwidth and kernel memory.

-------------- next part --------------
A non-text attachment was scrubbed...
Name: OpenPGP_signature
Type: application/pgp-signature
Size: 833 bytes
Desc: OpenPGP digital signature
URL: <http://lists.zx2c4.com/pipermail/wireguard/attachments/20211008/3acd2ff1/attachment.sig>

From frank.wayne at northwestern.edu  Fri Oct  8 20:50:03 2021
From: frank.wayne at northwestern.edu (Frank Wayne)
Date: Fri, 8 Oct 2021 20:50:03 +0000
Subject: Windows Log Output to Event Viewer or Text File
Message-ID: <SA1PR05MB79668D58D886BB8D36F558FE99B29@SA1PR05MB7966.namprd05.prod.outlook.com>

1. Is it possible to get WireGuard on Windows to output its logs to something other than the binary file "Data\log.bin"? 

2. Is there a way to access the contents of the binary file outside of the WireGuard UI?

Frank Wayne

From Jason at zx2c4.com  Fri Oct  8 22:01:09 2021
From: Jason at zx2c4.com (Jason A. Donenfeld)
Date: Fri, 8 Oct 2021 16:01:09 -0600
Subject: Windows Log Output to Event Viewer or Text File
In-Reply-To: <SA1PR05MB79668D58D886BB8D36F558FE99B29@SA1PR05MB7966.namprd05.prod.outlook.com>
References: <SA1PR05MB79668D58D886BB8D36F558FE99B29@SA1PR05MB7966.namprd05.prod.outlook.com>
Message-ID: <CAHmME9pu=syh5zkig=e1KFQKMvCYfu6e+BhKENdwZARCrpss7Q@mail.gmail.com>

https://git.zx2c4.com/wireguard-windows/about/docs/enterprise.md#diagnostic-logs

From jeremy at pregonetwork.net  Sat Oct  9 06:43:34 2021
From: jeremy at pregonetwork.net (=?UTF-8?B?SsOpcsOpbXkgUHJlZ28=?=)
Date: Sat, 9 Oct 2021 08:43:34 +0200
Subject: [wireguard-apple] [iOS] 464xlat networks and On-demand roaming
 issue
In-Reply-To: <BY5PR04MB6833078C93B5AE04028574EEA2AD9@BY5PR04MB6833.namprd04.prod.outlook.com>
References: <BY5PR04MB6833078C93B5AE04028574EEA2AD9@BY5PR04MB6833.namprd04.prod.outlook.com>
Message-ID: <cdbbd387-a3d3-820e-563f-0631ec0df256@pregonetwork.net>

Hello,

i have the exact same request for android, see my thread here:
https://lists.zx2c4.com/pipermail/wireguard/2021-September/007140.html

I regret that it is not configurable and that it does not use the 
default way of doing the system

Jerem

Le 04/10/2021 ? 00:15, Kristof Mattei a ?crit?:
> I have an issue with the wireguard-apple on 464xlat connecting to a DNS endpoint with both an A and an AAAA record.
>
> The following line: https://git.zx2c4.com/wireguard-apple/tree/Sources/WireGuardKit/DNSResolver.swift#n81 causes WireGuard to prefer the IPv4 address.
>
> Is there any reason why WireGuard prefers the IPv4 address?
>
> Why is this causing trouble? But this is what happens:
>
> When connecting to IPv6 the IPv4 address gets mapped to an IPv6 address which then acts as an IPv6->IPv4 proxy. The IP looks like [2607:7700:0:1a::17f3:f750]:51820.
>
> This causes issues when roaming from my home WiFi (on which WireGuard is disabled) to cellular (on which WireGuard is set to on-Demand).
>
> The initial connection that is set up for some reason does not work. There are reports about this on Reddit, e.g. https://www.reddit.com/r/WireGuard/comments/nk2o7m/anyone_got_it_working_with_tmobile_lte/
>
> I can fix it by setting the endpoint to the actual IPv6 address, and then it works like a charm, but that fails when I connect to a non-IPv6 network.
>
> Thanks,
> Kristof
>


From faustin at fala.red  Sat Oct  9 10:26:59 2021
From: faustin at fala.red (Faustin Lammler)
Date: Sat, 9 Oct 2021 12:26:59 +0200
Subject: Build wireguard-linux-compat on SLES15 (s390x)
Message-ID: <YWFuczOf1dM3B4Pt@falared>

Hi!

I am trying to compile wireguard-linux-compat on SLES15 (s390x) without
success. Maybe I should try to install it directly from packages but I
am not expert with SLES and I did not find how so far.

I managed to install wireguard-tools but the kernel does not provide
the wireguard module, see bellow:
| $ uname -r
| 5.3.18-24.83-default
| $ lsmod | grep wireguard
| $ sudo wg-quick up /etc/wireguard/wg0.conf
| [#] ip link add wg0 type wireguard
| Error: Unknown device type.
| Unable to access interface: Protocol not supported
| [#] ip link delete dev wg0
| Cannot find device "wg0"

Here is the build error:
| $ cd /tmp && git clone https://git.zx2c4.com/wireguard-linux-compat
| $ make -C wireguard-linux-compat/src -j$(nproc)
| make: Entering directory '/tmp/wireguard-linux-compat/src'
|   CC [M]  /tmp/wireguard-linux-compat/src/main.o
|   CC [M]  /tmp/wireguard-linux-compat/src/noise.o
|   CC [M]  /tmp/wireguard-linux-compat/src/device.o
|   CC [M]  /tmp/wireguard-linux-compat/src/peer.o
|   CC [M]  /tmp/wireguard-linux-compat/src/timers.o
|   CC [M]  /tmp/wireguard-linux-compat/src/queueing.o
|   CC [M]  /tmp/wireguard-linux-compat/src/send.o
|   CC [M]  /tmp/wireguard-linux-compat/src/receive.o
|   CC [M]  /tmp/wireguard-linux-compat/src/socket.o
|   CC [M]  /tmp/wireguard-linux-compat/src/peerlookup.o
|   CC [M]  /tmp/wireguard-linux-compat/src/allowedips.o
|   CC [M]  /tmp/wireguard-linux-compat/src/ratelimiter.o
|   CC [M]  /tmp/wireguard-linux-compat/src/cookie.o
|   CC [M]  /tmp/wireguard-linux-compat/src/netlink.o
|   CC [M]  /tmp/wireguard-linux-compat/src/crypto/zinc/chacha20/chacha20.o
|   CC [M]  /tmp/wireguard-linux-compat/src/crypto/zinc/poly1305/poly1305.o
|   CC [M]  /tmp/wireguard-linux-compat/src/crypto/zinc/chacha20poly1305.o
| In file included from <command-line>:
| /tmp/wireguard-linux-compat/src/compat/compat.h:859:31: error: expected identifier or ?(? before ?{? token
|   859 | #define genl_dumpit_info(cb) ({ \
|       |                               ^
| /usr/src/linux-5.3.18-24.83/include/net/genetlink.h:138:1: note: in expansion of macro ?genl_dumpit_info?
|   138 | genl_dumpit_info(struct netlink_callback *cb)
|       | ^~~~~~~~~~~~~~~~
| make[3]: *** [/usr/src/linux-5.3.18-24.83/scripts/Makefile.build:282: /tmp/wireguard-linux-compat/src/netlink.o] Error 1
| make[3]: *** Waiting for unfinished jobs....
| make[2]: *** [/usr/src/linux-5.3.18-24.83/Makefile:1655: _module_/tmp/wireguard-linux-compat/src] Error 2
| make[1]: *** [../../../linux-5.3.18-24.83/Makefile:179: sub-make] Error 2
| make: *** [Makefile:26: module] Error 2
| make: Leaving directory '/tmp/wireguard-linux-compat/src'

Any help would be really appreciated.
Cheers!

-- 
Faustin
GPG: F652 BCD1 1AA8 8975 F010 48A5 390A 2F27 832A 5C79
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: not available
URL: <http://lists.zx2c4.com/pipermail/wireguard/attachments/20211009/5a2f0643/attachment.sig>

From mikma.wg at lists.m7n.se  Sun Oct 10 21:19:22 2021
From: mikma.wg at lists.m7n.se (mikma.wg at lists.m7n.se)
Date: Sun, 10 Oct 2021 23:19:22 +0200
Subject: [PATCH] wireguard-tools: embeddable-wg-library: add named wg_endpoint
 union
Message-ID: <b28e05e1-4209-8b82-1b00-e963b2e8fc85@m7n.se>

Hello,

It would be useful to have a named wg_endpoint type in the embeddable wg 
library. It's added by the attached patch.

/Mikael

-------------- next part --------------
A non-text attachment was scrubbed...
Name: 0001-embeddable-wg-library-add-named-wg_endpoint-union.patch
Type: text/x-patch
Size: 1313 bytes
Desc: not available
URL: <http://lists.zx2c4.com/pipermail/wireguard/attachments/20211010/6171e848/attachment.bin>

From gmertes at outlook.be  Tue Oct 12 13:52:45 2021
From: gmertes at outlook.be (Gert Mertes)
Date: Tue, 12 Oct 2021 13:52:45 +0000
Subject: Keepalive packets transmitted by default
Message-ID: <DM5PR1201MB0012B195C77099DA856C713CC5B69@DM5PR1201MB0012.namprd12.prod.outlook.com>

Hi,

My Windows client (0.4.11) will still sporadically send (and receive) keepalive packets over an idle tunnel, even though PersistentKeepalive is not set in the config of any peer in the tunnel. Explicitly setting it to 0 also has the same result. I?m wondering if (i) the above is expected behaviour and (ii) is it possible to completely disable keepalive packets?

Thanks,
Gert

From frank.wayne at northwestern.edu  Tue Oct 12 21:39:31 2021
From: frank.wayne at northwestern.edu (Frank Wayne)
Date: Tue, 12 Oct 2021 21:39:31 +0000
Subject: Windows Log Output to Event Viewer or Text File
In-Reply-To: <CAHmME9pu=syh5zkig=e1KFQKMvCYfu6e+BhKENdwZARCrpss7Q@mail.gmail.com>
References: <SA1PR05MB79668D58D886BB8D36F558FE99B29@SA1PR05MB7966.namprd05.prod.outlook.com>
 <CAHmME9pu=syh5zkig=e1KFQKMvCYfu6e+BhKENdwZARCrpss7Q@mail.gmail.com>
Message-ID: <PH0PR05MB7962AD22BE333E844C0A79B799B69@PH0PR05MB7962.namprd05.prod.outlook.com>

Thanks for the link, Jason.

That's pretty awful. It is only possible to get the last 2048 events and no way to get just the events since the last update. There is no way for an aggregator to simply collect WireGuard logs on Windows.

Frank Wayne

-----Original Message-----
From: Jason A. Donenfeld <Jason at zx2c4.com> 
Sent: Friday, 8 October, 2021 17:01
To: Frank Wayne <frank.wayne at northwestern.edu>
Cc: WireGuard mailing list <wireguard at lists.zx2c4.com>
Subject: Re: Windows Log Output to Event Viewer or Text File

https://urldefense.com/v3/__https://git.zx2c4.com/wireguard-windows/about/docs/enterprise.md*diagnostic-logs__;Iw!!Dq0X2DkFhyF93HkjWTBQKhk!EnH5c-_WQuKGxDYwm737RpDPw0mKqlouc_l-2cGs4omjuN-SafYq-HHpnKKjvknPsn0SLhpq$ 

From Jason at zx2c4.com  Tue Oct 12 21:40:57 2021
From: Jason at zx2c4.com (Jason A. Donenfeld)
Date: Tue, 12 Oct 2021 15:40:57 -0600
Subject: Windows Log Output to Event Viewer or Text File
In-Reply-To: <PH0PR05MB7962AD22BE333E844C0A79B799B69@PH0PR05MB7962.namprd05.prod.outlook.com>
References: <SA1PR05MB79668D58D886BB8D36F558FE99B29@SA1PR05MB7966.namprd05.prod.outlook.com>
 <CAHmME9pu=syh5zkig=e1KFQKMvCYfu6e+BhKENdwZARCrpss7Q@mail.gmail.com>
 <PH0PR05MB7962AD22BE333E844C0A79B799B69@PH0PR05MB7962.namprd05.prod.outlook.com>
Message-ID: <CAHmME9pgbxyAxgBAfgLMVquzmPC9fiBzjWhQ5zJV+3bR2aR9dQ@mail.gmail.com>

On Tue, Oct 12, 2021 at 3:39 PM Frank Wayne
<frank.wayne at northwestern.edu> wrote:
> That's pretty awful. It is only possible to get the last 2048 events and no way to get just the events since the last update. There is no way for an aggregator to simply collect WireGuard logs on Windows.

Your "that's pretty awful" aside, is what you're asking for some kind
of CLI "follow" mode that doesn't terminate and spits out logs to
stdout perpetually?

Jason

From sbonar at gmail.com  Tue Oct 12 23:59:43 2021
From: sbonar at gmail.com (Scott Bonar)
Date: Tue, 12 Oct 2021 17:59:43 -0600
Subject: Wintun.dll (amd64) GetProcAddress failure
Message-ID: <CABGW2bUxOaOBuChZTS4w7+3OX0WNvypjYc-+cSNgQO-zWQHaFw@mail.gmail.com>

I will be the first to admit I maybe doing something wrong, but the
DLL loads just fine, however when I try to init the API functions the
only one that is successful is WintunCreateAdapter.
All other calls to GetProcAddress() return NULL and LastError is
coming back as 0.  Any help would be appreciated.

Thanks

From Jason at zx2c4.com  Wed Oct 13 00:13:27 2021
From: Jason at zx2c4.com (Jason A. Donenfeld)
Date: Tue, 12 Oct 2021 18:13:27 -0600
Subject: Wintun.dll (amd64) GetProcAddress failure
In-Reply-To: <CABGW2bUxOaOBuChZTS4w7+3OX0WNvypjYc-+cSNgQO-zWQHaFw@mail.gmail.com>
References: <CABGW2bUxOaOBuChZTS4w7+3OX0WNvypjYc-+cSNgQO-zWQHaFw@mail.gmail.com>
Message-ID: <CAHmME9rd7uKcbKY0dXKLOPKVdxqrULOOCMvtjgFiaXK1ki1KHA@mail.gmail.com>

If you're using the just-releases Wintun 0.14, make sure you're using
the wintun.h from that release, alongside the InitializeWintun
function from example.c.

From sbonar at gmail.com  Wed Oct 13 00:30:08 2021
From: sbonar at gmail.com (Scott Bonar)
Date: Wed, 13 Oct 2021 00:30:08 +0000
Subject: Wintun.dll (amd64) GetProcAddress failure
In-Reply-To: <CAHmME9rd7uKcbKY0dXKLOPKVdxqrULOOCMvtjgFiaXK1ki1KHA@mail.gmail.com>
References: <CABGW2bUxOaOBuChZTS4w7+3OX0WNvypjYc-+cSNgQO-zWQHaFw@mail.gmail.com>
 <CAHmME9rd7uKcbKY0dXKLOPKVdxqrULOOCMvtjgFiaXK1ki1KHA@mail.gmail.com>
Message-ID: <MW2PR18MB225138007844015EDBF28F8CADB79@MW2PR18MB2251.namprd18.prod.outlook.com>

Thanks.  I don?t know what version of example.c and wintun.h I was looking at verse what comes in the release zip.
But they are for sure different.  Now that I have the correct files things appear to work better and I?m not going to even
Try and track down what version I was looking at.

?On 10/12/21, 6:13 PM, "Jason A. Donenfeld" <Jason at zx2c4.com> wrote:

    If you're using the just-releases Wintun 0.14, make sure you're using
    the wintun.h from that release, alongside the InitializeWintun
    function from example.c.

From me at aaronmdjones.net  Wed Oct 13 02:08:40 2021
From: me at aaronmdjones.net (Aaron Jones)
Date: Wed, 13 Oct 2021 02:08:40 +0000
Subject: Keepalive packets transmitted by default
In-Reply-To: <DM5PR1201MB0012B195C77099DA856C713CC5B69@DM5PR1201MB0012.namprd12.prod.outlook.com>
References: <DM5PR1201MB0012B195C77099DA856C713CC5B69@DM5PR1201MB0012.namprd12.prod.outlook.com>
Message-ID: <21958b52-894e-cb79-d831-159d7fda04e6@aaronmdjones.net>

On 12/10/2021 13:52, Gert Mertes wrote:
> Hi,
>
> My Windows client (0.4.11) will still sporadically send (and receive)
> keepalive packets over an idle tunnel, even though PersistentKeepalive
> is not set in the config of any peer in the tunnel. Explicitly setting
> it to 0 also has the same result. I?m wondering if (i) the above is
> expected behaviour and (ii) is it possible to completely disable
> keepalive packets?
>
> Thanks,
> Gert

This is the expected behaviour, and it is not possible to disable it.

It's occurring because there is a unidirectional data transfer happening
(e.g. UDP), and the other side is responding with the keepalive because
it hasn't already done so recently (like it would with e.g. a TCP ACK)
and has nothing to send.

The reasons are described in sections 6.2 and 6.5 of the WireGuard
whitepaper.


-------------- next part --------------
A non-text attachment was scrubbed...
Name: OpenPGP_signature
Type: application/pgp-signature
Size: 833 bytes
Desc: OpenPGP digital signature
URL: <http://lists.zx2c4.com/pipermail/wireguard/attachments/20211013/f77b672f/attachment.sig>

From eugenio at eutampieri.eu  Wed Oct 13 11:02:36 2021
From: eugenio at eutampieri.eu (Eugenio Tampieri)
Date: Wed, 13 Oct 2021 13:02:36 +0200
Subject: wireguard-apple localization
Message-ID: <83B7BA7B-85A0-4461-B4E5-91EA33BBF2C4@eutampieri.eu>

Hello,

I?d like to provide a new set of localized strings in Italian, since the current one is quite broken, with a few strings missing and others not exactly correct.
I?ve noticed that the repo tries to sync the localized strings with an online platform and I was wondering what?s the best way to contribute.

Regards,

Eugenio Tampieri

From frank.wayne at northwestern.edu  Wed Oct 13 13:29:58 2021
From: frank.wayne at northwestern.edu (Frank Wayne)
Date: Wed, 13 Oct 2021 13:29:58 +0000
Subject: Windows Log Output to Event Viewer or Text File
In-Reply-To: <CAHmME9pgbxyAxgBAfgLMVquzmPC9fiBzjWhQ5zJV+3bR2aR9dQ@mail.gmail.com>
References: <SA1PR05MB79668D58D886BB8D36F558FE99B29@SA1PR05MB7966.namprd05.prod.outlook.com>
 <CAHmME9pu=syh5zkig=e1KFQKMvCYfu6e+BhKENdwZARCrpss7Q@mail.gmail.com>
 <PH0PR05MB7962AD22BE333E844C0A79B799B69@PH0PR05MB7962.namprd05.prod.outlook.com>
 <CAHmME9pgbxyAxgBAfgLMVquzmPC9fiBzjWhQ5zJV+3bR2aR9dQ@mail.gmail.com>
Message-ID: <PH0PR05MB79623EC9C12B4AE732B04E9299B79@PH0PR05MB7962.namprd05.prod.outlook.com>


>> On Tue, Oct 12, 2021 at 3:39 PM Frank Wayne <frank.wayne at northwestern.edu> wrote:
>> That's pretty awful. It is only possible to get the last 2048 events and no way to get just the events since the last update. There is no way for an aggregator to simply collect WireGuard logs on Windows.

> Your "that's pretty awful" aside, is what you're asking for some kind of CLI "follow" mode that doesn't terminate and spits out logs to stdout perpetually?

> Jason

No. I'm not sure that would be much of an improvement.

In Linux (under systemd), kernel logs are accessible in journald, can be forwarded to (r)syslog, and from there to a text file or external syslog or wherever.

In Windows, logs would ideally get sent to Event Logging into a WireGuard log. That way, the user or administrator can use Event Viewer to view the log, forward the log,  or use a collector (like Splunk) to retrieve and aggregate the events. Using a proprietary log makes it difficult to monitor this or any other app.

I'm not sure why WireGuard doesn't use Windows Event Logging. I can't imagine that a proprietary log format would fly in Linux, or even be contemplated. Is there something that precludes the use of Event Logging by WireGuard?

Frank Wayne

From Jason at zx2c4.com  Wed Oct 13 18:16:55 2021
From: Jason at zx2c4.com (Jason A. Donenfeld)
Date: Wed, 13 Oct 2021 12:16:55 -0600
Subject: Windows Log Output to Event Viewer or Text File
In-Reply-To: <PH0PR05MB79623EC9C12B4AE732B04E9299B79@PH0PR05MB7962.namprd05.prod.outlook.com>
References: <SA1PR05MB79668D58D886BB8D36F558FE99B29@SA1PR05MB7966.namprd05.prod.outlook.com>
 <CAHmME9pu=syh5zkig=e1KFQKMvCYfu6e+BhKENdwZARCrpss7Q@mail.gmail.com>
 <PH0PR05MB7962AD22BE333E844C0A79B799B69@PH0PR05MB7962.namprd05.prod.outlook.com>
 <CAHmME9pgbxyAxgBAfgLMVquzmPC9fiBzjWhQ5zJV+3bR2aR9dQ@mail.gmail.com>
 <PH0PR05MB79623EC9C12B4AE732B04E9299B79@PH0PR05MB7962.namprd05.prod.outlook.com>
Message-ID: <CAHmME9q3j=2HG-4v5tpHRM9DCKpw+5iSMyzu30jAUHS1r=eDHw@mail.gmail.com>

Hi Frank,

On Wed, Oct 13, 2021 at 7:30 AM Frank Wayne
<frank.wayne at northwestern.edu> wrote:
> >> On Tue, Oct 12, 2021 at 3:39 PM Frank Wayne <frank.wayne at northwestern.edu> wrote:
> >> That's pretty awful. It is only possible to get the last 2048 events and no way to get just the events since the last update. There is no way for an aggregator to simply collect WireGuard logs on Windows.
>
> > Your "that's pretty awful" aside, is what you're asking for some kind of CLI "follow" mode that doesn't terminate and spits out logs to stdout perpetually?
>
> > Jason
>
> No. I'm not sure that would be much of an improvement.

Why not? It would make it possible to tail the log more easily and
pipe it into whatever log collection daemon you want. And if that's
indeed still not an improvement, what is the relevance of your
previous mention, "It is only possible to get the last 2048 events and
no way to get just the events since the last update"? I find the tone
of your messages quite abrasive rather than informative. Can you slow
down a bit and try to describe the constraints and requirements of
your system, and then we can try to figure out what would be a good
path toward realizing a good design there? Let's start with: what's
missing in a tail mode that you can pipe to whatever, other than,
"it's not ms event logging"?

> In Linux (under systemd), kernel logs are accessible in journald, can be forwarded to (r)syslog, and from there to a text file or external syslog or wherever.

I'm pretty sure systemd-journald epolls on /dev/kmsg. In other words,
it aggregates logs from different sources. That's not a whole lot
different from a follow mode, right? But why are you talking about
Linux, or about kernel logs for that matter?

> I can't imagine that a proprietary log format would fly in Linux, or even be contemplated.

On Linux, wireguard.ko simply uses printk, like other kernel drivers,
and wg-quick(8) uses stdio, like many userspace programs. But why are
you talking about Linux? What's the relevance?

You've used the word "proprietary" but I think "bespoke" might be more
clear. There are open source implementations in C, C#, and Go in the
git repos, and it should be rather trivial to parse in any other
language too.

> In Windows, logs would ideally get sent to Event Logging into a WireGuard log. That way, the user or administrator can use Event Viewer to view the log, forward the log,  or use a collector (like Splunk) to retrieve and aggregate the events.
> I'm not sure why WireGuard doesn't use Windows Event Logging. Is there something that precludes the use of Event Logging by WireGuard?

Event Logging appears to be rather slow and clunky, and I'm not sure
I'd be too happy about blocking on that during packet events. It's
also very cumbersome to use -- especially for things like crash dumps,
which require a separate process or dll -- and the boilerplate
involved doesn't look very appealing. In contrast, memory mapping a
file, memcpy'ing buffers into it, and getting timestamps by reading
0x7ffe0014, means no calls to libraries or any interactions with other
moving components that might be in an undefined state.

Event Logging might yet be possible to use, though. But it seems to me
that'd be some significant research and development work, to figuring
out how it could work in a lightweight way, and also revisiting the
way wgnt logs things.

If this is something you'd like to work on, I'd be happy to review
patches and read descriptions of a simplified event logging
implementation. You're probably not the only user with this concern,
and in theory I'd be open to considering it, provided there's a way to
make it less clunky than initially meets the eye.

Jason

From clint at openziti.org  Thu Oct 14 00:40:43 2021
From: clint at openziti.org (Clint Dovholuk)
Date: Wed, 13 Oct 2021 20:40:43 -0400
Subject: wintun.dll + go CreateTun crash
Message-ID: <5656bb014ee2161f6c8ea5f35d354563@mail.gmail.com>

Our project calls
	tun.CreateTUN(interfaceName, mtu).

I upgraded to windows 11 tonight and this call now crashes at:
	
golang.zx2c4.com/wireguard at v0.0.20201119-0.20201209004655-310ae107c346/tun
/wintun/wintun_windows.go:89

which is this system call:
	r0, _, e1 := syscall.Syscall(procWintunOpenAdapter.Addr(), 2,
uintptr(unsafe.Pointer(pool)), uintptr(unsafe.Pointer(ifname16)), 0)

I'm going to try to debug why myself but figured I'd reach out to the
mailing list in case this was a known issue that's already solved or being
worked on?

Thanks (I can supply the full stack if you like but I wouldn't think it'd
help and decided to keep this mail shorter)
-Clint

From Jason at zx2c4.com  Thu Oct 14 01:30:53 2021
From: Jason at zx2c4.com (Jason A. Donenfeld)
Date: Wed, 13 Oct 2021 19:30:53 -0600
Subject: wintun.dll + go CreateTun crash
In-Reply-To: <5656bb014ee2161f6c8ea5f35d354563@mail.gmail.com>
References: <5656bb014ee2161f6c8ea5f35d354563@mail.gmail.com>
Message-ID: <CAHmME9ogMJfvvHWf065-QO4WrC9-FTNzE+4g6+=wdjJ=SJmpCQ@mail.gmail.com>

If you're using the latest wireguard-go, make sure you're also using
the latest wintun.dll (v0.14 as of writing). You can download the dll
from wintun.net.

From uxDWzco-wg at moenia.de  Thu Oct 14 02:45:32 2021
From: uxDWzco-wg at moenia.de (uxDWzco-wg at moenia.de)
Date: Thu, 14 Oct 2021 04:45:32 +0200
Subject: linux: bridging/bonding not possible
Message-ID: <78fa512a-82c6-0b0c-d759-162d31a134b4@moenia.de>

hi.

as I understand, linux needs the ability to change hardware-addresses on
netdevs to put them into a bridge or bond, but wireguard-netdevs on
linux don't support hw-addresses at all (at least in kernel 5.10).

is it possible (or even planned) to add hw-addresses to the
wireguard-netdevs or does this interfere with the concept of wireguard?

regards

j.

From tanioka404 at gmail.com  Thu Oct 14 03:48:10 2021
From: tanioka404 at gmail.com (Eiji Tanioka)
Date: Thu, 14 Oct 2021 12:48:10 +0900
Subject: [PATCH] wireguard-tools: embeddable-wg-library: add named
 wg_endpoint union
In-Reply-To: <b28e05e1-4209-8b82-1b00-e963b2e8fc85@m7n.se>
References: <b28e05e1-4209-8b82-1b00-e963b2e8fc85@m7n.se>
Message-ID: <CAA2ss9HGhumPcQ-wNZMvf1fmyrTxgv5L3rBsuToaBKdaVL9vJw@mail.gmail.com>

Hi,

Your patch is being sent in an undesirable way, so I think it will not
be reviewed.
In this page, contributing to wireguard-tools requires
`git-send-email`, so you should include the patch file in the body of
the email, rather than attaching it.
https://www.wireguard.com/repositories/

Here is an example how to use `git send-email`
https://git-scm.com/docs/git-send-email#_examples

Hope this help.

2021?10?11?(?) 6:24 <mikma.wg at lists.m7n.se>:
>
> Hello,
>
> It would be useful to have a named wg_endpoint type in the embeddable wg
> library. It's added by the attached patch.
>
> /Mikael
>

From tanioka404 at gmail.com  Thu Oct 14 03:55:06 2021
From: tanioka404 at gmail.com (Eiji Tanioka)
Date: Thu, 14 Oct 2021 12:55:06 +0900
Subject: wireguard-apple localization
In-Reply-To: <83B7BA7B-85A0-4461-B4E5-91EA33BBF2C4@eutampieri.eu>
References: <83B7BA7B-85A0-4461-B4E5-91EA33BBF2C4@eutampieri.eu>
Message-ID: <CAA2ss9EZnrN78rvLbSrY-4x0o9k91FMXucoh+KndYUZ+E3BYbQ@mail.gmail.com>

Hi,

When I did Japanese localization before, I was told to use Crowdin.
https://crowdin.com/project/WireGuard

I think you have to create an account and contact the project manager.

Hope this helps.

2021?10?13?(?) 21:02 Eugenio Tampieri <eugenio at eutampieri.eu>:
>
> Hello,
>
> I?d like to provide a new set of localized strings in Italian, since the current one is quite broken, with a few strings missing and others not exactly correct.
> I?ve noticed that the repo tries to sync the localized strings with an online platform and I was wondering what?s the best way to contribute.
>
> Regards,
>
> Eugenio Tampieri

From Jason at zx2c4.com  Thu Oct 14 05:46:40 2021
From: Jason at zx2c4.com (Jason A. Donenfeld)
Date: Wed, 13 Oct 2021 23:46:40 -0600
Subject: Note: wireguard-nt 0.10 and wintun 0.14 have slightly different API
 from prior versions
Message-ID: <CAHmME9oOuJn+Q9GsQEH=mBJXza4VdQ31FUYi-ZbPJT6sceKwkA@mail.gmail.com>

Hi,

Version 0.10 of wireguard-nt and version 0.14 of wintun are using a
slightly different API and ABI than previous versions. As such, make
sure you're using the wireguard.h/wintun.h header file that comes with
the DLL that you download. And if you're using the Go packages, make
sure you've paired the latest commit with the latest DLL.

Thanks,
Jason

From Jason at zx2c4.com  Thu Oct 14 05:56:56 2021
From: Jason at zx2c4.com (Jason A. Donenfeld)
Date: Wed, 13 Oct 2021 23:56:56 -0600
Subject: Windows stuck in boot after WireGuard update (switch to
 WireGuardNT?)
In-Reply-To: <CAHmME9rknanKEdk8L5PJxbg1OZejN59z5xuh5y54O7y+zWb1nw@mail.gmail.com>
References: <e0d4dcbb-b0c8-1214-ff0c-c856d168d67f@physik.uni-bonn.de>
 <CAHmME9oSSzJU4H8jc__HiFsexpB6jJUKM84kN+_9WesoTSMMsQ@mail.gmail.com>
 <CAHmME9rknanKEdk8L5PJxbg1OZejN59z5xuh5y54O7y+zWb1nw@mail.gmail.com>
Message-ID: <CAHmME9qSUqo4Wj8U4G3oznxnGB5_7oJkMULwSg4oVVgoRgQ05w@mail.gmail.com>

Hi folks,

I assume that after >3 weeks of silence, the problem is fixed. That's good.

The root cause was that wgnt's initialization waited for winsock to
become available, but winsock was waiting for network interfaces like
wgnt to become available first, so they deadlocked for a half hour.
The solution, deployed over three weeks ago, was to defer wgnt's usage
of winsock until userspace requested it, which is necessarily after
everything else. This apparently works and is the correct fix.

But for whatever reason, the impact of this bug has really haunted me,
and so I didn't stop working on it or thinking about it even after it
was fixed correctly. The larger question is: why is a wgnt adapter
coming up at boot time at all? It's clearly not needed, as it is
something that anyway is controlled by a service. And indeed wgnt
adapters are supposed to be cleaned up when the service exits, but
this cleanup might not be committed if it happens during shutdown.
This is why sometimes the bug happened and sometimes it didn't.

So how can we solve the larger issue of never having wgnt adapters
exist at boot time, and always have them tied to the service lifetime?
It turns out Win8+ has an API for this called SwDevice, which
"unplugs" adapters when their owning process exits (or corresponding
handles close). This is used by various internal Windows components
like rasmans.dll and seems quite robust. It's also very different and
comes with its own gotchas, compared to what we were doing before, and
using it is no small undertaking. So over the last 3 weeks or so, I've
rewritten the entire userspace API portion of wgnt (and wintun) in
order to use this API where available (with ugly fallbacks for Win7).
The result is that the entire class of bugs, of which this bug here
was one, should be eliminated all together. And in the process, I've
made a lot of other things about adapter installation more reliable
and quicker too.

This work shipped with wgnt 0.10 and wintun 0.14, and is part of
wireguard for windows 0.4.11.

Regards,
Jason

From eugenio at eutampieri.eu  Thu Oct 14 07:19:43 2021
From: eugenio at eutampieri.eu (Eugenio Tampieri)
Date: Thu, 14 Oct 2021 09:19:43 +0200
Subject: wireguard-apple localization
Message-ID: <BCF80CA9-7681-4A1F-8F6F-26763779D554@eutampieri.eu>

I?ve had a look at Crowdin and I?ve contributed a few strings. However, crowdin is missing a few strings, such as tunnelStatusAddendumOnDemand and tunnelListCaptionOnDemand.
Kindly,

Eugenio Tampieri

> Il giorno 14 ott 2021, alle ore 05:55, Eiji Tanioka <tanioka404 at gmail.com> ha scritto:
> 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 2479 bytes
Desc: not available
URL: <http://lists.zx2c4.com/pipermail/wireguard/attachments/20211014/d79dbc44/attachment.p7s>

From rm at romanrm.net  Thu Oct 14 07:53:18 2021
From: rm at romanrm.net (Roman Mamedov)
Date: Thu, 14 Oct 2021 12:53:18 +0500
Subject: linux: bridging/bonding not possible
In-Reply-To: <78fa512a-82c6-0b0c-d759-162d31a134b4@moenia.de>
References: <78fa512a-82c6-0b0c-d759-162d31a134b4@moenia.de>
Message-ID: <20211014125318.0e569dfc@nvm>

On Thu, 14 Oct 2021 04:45:32 +0200
uxDWzco-wg at moenia.de wrote:

> as I understand, linux needs the ability to change hardware-addresses on
> netdevs to put them into a bridge or bond, but wireguard-netdevs on
> linux don't support hw-addresses at all (at least in kernel 5.10).
> 
> is it possible (or even planned) to add hw-addresses to the
> wireguard-netdevs or does this interfere with the concept of wireguard?

Hello,

It is not a matter of hw-addresses;

Wireguard is L3 interface, transferring IPv4 and IPv6 packets.

For bridging you would need an L2 interface, which transfers Ethernet frames.

It is possible to do a bridge with WG, by using an L2-over-L3 tunnel such as
VXLAN or GRETAP over WG, and bridging that. Of course this leads to additional
overhead and MTU reduction.

If you would prefer to have an L2 VPN directly, there are other solutions such
as Tinc and OpenVPN.

-- 
With respect,
Roman

From heroxbd at gentoo.org  Thu Oct 14 03:25:02 2021
From: heroxbd at gentoo.org (Benda Xu)
Date: Thu, 14 Oct 2021 11:25:02 +0800
Subject: linux: bridging/bonding not possible
In-Reply-To: <78fa512a-82c6-0b0c-d759-162d31a134b4@moenia.de>
 (uxDWzco-wg@moenia.de's message of "Thu, 14 Oct 2021 04:45:32 +0200")
References: <78fa512a-82c6-0b0c-d759-162d31a134b4@moenia.de>
Message-ID: <87bl3sl241.fsf@proton.d.airelinux.org>

Hi uxDWzco,

uxDWzco-wg at moenia.de writes:

> as I understand, linux needs the ability to change hardware-addresses
> on netdevs to put them into a bridge or bond, but wireguard-netdevs on
> linux don't support hw-addresses at all (at least in kernel 5.10).
>
> is it possible (or even planned) to add hw-addresses to the
> wireguard-netdevs or does this interfere with the concept of
> wireguard?

Bridging is an layer 2 network concept. But wireguard is a layer 3
connection.

For more information on OSI layer model, refer to,

  https://en.wikipedia.org/wiki/OSI_model

Cheers,
Benda

From wireguard at qupfer.de  Thu Oct 14 07:12:27 2021
From: wireguard at qupfer.de (wireguard at qupfer.de)
Date: Thu, 14 Oct 2021 09:12:27 +0200
Subject: linux: bridging/bonding not possible
In-Reply-To: <76173@imapsync>
References: <76173@imapsync>
Message-ID: <84cf0d10-471a-5e25-a160-79dd7cdfc367@qupfer.de>

Hi,

 >> is it possible (or even planned) to add hw-addresses to the
 >> wireguard-netdevs or does this interfere with the concept of wireguard?

I hope I say nothing wrong, but its not (directly) possible and probably 
not plant.

Wireguard is a so called Layer-3 VPN, bridging is a Layer-2 thing. So it 
will no work together.
But you could use an (un)secure Layer-2-VPN (like L2TP) and transport it 
through wireguard. (similar to the often used L2TP over IPsec).
You could also take a look to softether vpn 
(https://www.softether.org/), which also includes a Layer-2 VPN. But I 
have no clue about the security quality.


From svenne at kracon.dk  Thu Oct 14 08:30:27 2021
From: svenne at kracon.dk (Svenne Krap)
Date: Thu, 14 Oct 2021 10:30:27 +0200
Subject: Source IP for multihomed peer
Message-ID: <d0e1773d-80d5-7676-7943-2eff8d57beaa@kracon.dk>

Hi,

I have it a problem, that seems like the following is happening.


BoxA has multiple ip-addresses with different internet providers (i.e. 
multihomed)

BoxB is a normal single-homed dynamic peer (i.e. no fixed address), as 
is BoxC.

BoxB? and boxC both have hardcoded address1 ('boxA1')? as its peer


What seems to happen is:

1) BoxB writes sends to BoxA1? (address 1)

2) BoxA responds with BoxA2? (address 2)

3) BoxB disregards the traffic.


BoxC contacts boxA on BoxA1 and due to routing (due to BoxC's network) 
it gets replies with the right address ('boxA1'), and everything works 
as expected.



My question is twofold:

1) Does the above seem like a likely chain of events?

2) Is there any way to force the source ip of the connection from boxA 
to always use address boxA1 ?

 From the documentation Listenport only seems like the portnumber and 
there seems to be no way to set the source ip.


Regards


Svenne


From frank.wayne at northwestern.edu  Thu Oct 14 17:41:31 2021
From: frank.wayne at northwestern.edu (Frank Wayne)
Date: Thu, 14 Oct 2021 17:41:31 +0000
Subject: Windows Log Output to Event Viewer or Text File
In-Reply-To: <CAHmME9q3j=2HG-4v5tpHRM9DCKpw+5iSMyzu30jAUHS1r=eDHw@mail.gmail.com>
References: <SA1PR05MB79668D58D886BB8D36F558FE99B29@SA1PR05MB7966.namprd05.prod.outlook.com>
 <CAHmME9pu=syh5zkig=e1KFQKMvCYfu6e+BhKENdwZARCrpss7Q@mail.gmail.com>
 <PH0PR05MB7962AD22BE333E844C0A79B799B69@PH0PR05MB7962.namprd05.prod.outlook.com>
 <CAHmME9pgbxyAxgBAfgLMVquzmPC9fiBzjWhQ5zJV+3bR2aR9dQ@mail.gmail.com>
 <PH0PR05MB79623EC9C12B4AE732B04E9299B79@PH0PR05MB7962.namprd05.prod.outlook.com>
 <CAHmME9q3j=2HG-4v5tpHRM9DCKpw+5iSMyzu30jAUHS1r=eDHw@mail.gmail.com>
Message-ID: <PH0PR05MB79629688EB84BF07A2AEF54F99B89@PH0PR05MB7962.namprd05.prod.outlook.com>

Jason,

> Why not? It would make it possible to tail the log more easily and pipe it into whatever log collection daemon you want. And if that's indeed still not an improvement, what is the relevance of your previous mention, "It is only possible to get the last 2048 events and no way to get just the events since the last update"? I find the tone of your messages quite abrasive rather than informative. Can you slow down a bit and try to describe the constraints and requirements of your system, and then we can try to figure out what would be a good path toward realizing a good design there? Let's start with: what's missing in a tail mode that you can pipe to whatever, other than, "it's not ms event logging"?

The requirements are providing a simple, timely, and complete feed of WireGuard events to a log aggregator.

If I could get "the events since the last update", then a scheduled process could run to copy the events to some text log. That data could be aggregated. Even so, it assumes that the number of events that occur between iterations of the scheduled job does not exceed 2048. It makes things more complicated and the chance of losing events remains. As such, this does not fit the requirements well, as it is not simple (it requires the scheduling of a job, the choice of intervals based on the expected rate of events, and may be different for each host), nor timely (since events are delayed by the duration between iterations of the scheduled job), nor complete (since it is prone to event overflow). That's why I don't consider tailing the log *much* of an improvement. It would be an improvement to be sure, just not like what is available in Linux.

Since I have mainly worked on collecting logs from Linux WireGuard hosts where the process of getting the logs into a file is simple, timely, and complete with little effort, my reaction to the Windows version was that, relative to Linux, the logging is "awful" to deal with. I apologize for using that term blithely and that it came across as abrasive. That was not my intention. I did not mean to characterize any component of WireGuard as awful, and certainly the logging for the Windows version works well (as far as I can tell), it just doesn't fit the logging paradigms that Linux or Windows applications use that lend themselves to aggregation.

I have no particular love for Windows Event Log, but it is the de facto logging facility in Windows. If the logs went to a text file, it would be even better, frankly, but there's this thing called Windows Event Log that many (most?) applications use. If WireGuard used it, the requirements would be met. My questions might imply a preference toward Windows Event Log, but it is so owing only to that feature's ubiquity.

> I'm pretty sure systemd-journald epolls on /dev/kmsg. In other words, it aggregates logs from different sources. That's not a whole lot different from a follow mode, right? But why are you talking about Linux, or about kernel logs for that matter?

Not too different from follow mode, but journald (Linux's binary log file solution) *can* forward to rsyslog. Windows WireGuard's binary log file solution can't. I'm talking about Linux because its WireGuard logging meets the requirements. Linux WireGuard is easy to deal with because it uses standard Linux logging components. I'm talking about kernel logs because Linux WireGuard logs are kernel messages; i.e., they have a facility number of 0.

> On Linux, wireguard.ko simply uses printk, like other kernel drivers, and wg-quick(8) uses stdio, like many userspace programs. But why are you talking about Linux? What's the relevance?

Again, I'm talking about Linux because Linux WireGuard uses Linux logging facilities that are easy to work with and Windows WireGuard does not use Windows logging facilities that would be easy to work with.

> You've used the word "proprietary" but I think "bespoke" might be more clear. There are open source implementations in C, C#, and Go in the git repos, and it should be rather trivial to parse in any other language too.

I agree, but it is harder than it could be, and still prone to overflow.

> Event Logging appears to be rather slow and clunky, and I'm not sure I'd be too happy about blocking on that during packet events. It's also very cumbersome to use -- especially for things like crash dumps, which require a separate process or dll -- and the boilerplate involved doesn't look very appealing. In contrast, memory mapping a file, memcpy'ing buffers into it, and getting timestamps by reading 0x7ffe0014, means no calls to libraries or any interactions with other moving components that might be in an undefined state.

> Event Logging might yet be possible to use, though. But it seems to me that'd be some significant research and development work, to figuring out how it could work in a lightweight way, and also revisiting the way wgnt logs things.

I don't pretend to understand the performance or convenience implications of using Windows Event Log instead. If using it would impact WireGuard performance, avoiding it is understandable. Thanks for describing how WireGuard writes the logs, which is both elegant and efficient. I just wish there were an easier way to reliably get the events into a sequential log (file?) of indefinite length.

Thank you for taking the time to respond to my questions. My objective was first to know that I wasn't missing something, and then more about understanding why things are the way they are than about influencing change. I imagine that other people have or will have a need to monitor WireGuard on Windows and would benefit from a simpler, external interface to the log events. I hope you keep this in mind, but for now I will try to make do with what I have.

Frank Wayne


-----Original Message-----
From: Jason A. Donenfeld <Jason at zx2c4.com> 
Sent: Wednesday, 13 October, 2021 13:17
To: Frank Wayne <frank.wayne at northwestern.edu>
Cc: WireGuard mailing list <wireguard at lists.zx2c4.com>
Subject: Re: Windows Log Output to Event Viewer or Text File

Hi Frank,

On Wed, Oct 13, 2021 at 7:30 AM Frank Wayne <frank.wayne at northwestern.edu> wrote:
> >> On Tue, Oct 12, 2021 at 3:39 PM Frank Wayne <frank.wayne at northwestern.edu> wrote:
> >> That's pretty awful. It is only possible to get the last 2048 events and no way to get just the events since the last update. There is no way for an aggregator to simply collect WireGuard logs on Windows.
>
> > Your "that's pretty awful" aside, is what you're asking for some kind of CLI "follow" mode that doesn't terminate and spits out logs to stdout perpetually?
>
> > Jason
>
> No. I'm not sure that would be much of an improvement.

Why not? It would make it possible to tail the log more easily and pipe it into whatever log collection daemon you want. And if that's indeed still not an improvement, what is the relevance of your previous mention, "It is only possible to get the last 2048 events and no way to get just the events since the last update"? I find the tone of your messages quite abrasive rather than informative. Can you slow down a bit and try to describe the constraints and requirements of your system, and then we can try to figure out what would be a good path toward realizing a good design there? Let's start with: what's missing in a tail mode that you can pipe to whatever, other than, "it's not ms event logging"?

> In Linux (under systemd), kernel logs are accessible in journald, can be forwarded to (r)syslog, and from there to a text file or external syslog or wherever.

I'm pretty sure systemd-journald epolls on /dev/kmsg. In other words, it aggregates logs from different sources. That's not a whole lot different from a follow mode, right? But why are you talking about Linux, or about kernel logs for that matter?

> I can't imagine that a proprietary log format would fly in Linux, or even be contemplated.

On Linux, wireguard.ko simply uses printk, like other kernel drivers, and wg-quick(8) uses stdio, like many userspace programs. But why are you talking about Linux? What's the relevance?

You've used the word "proprietary" but I think "bespoke" might be more clear. There are open source implementations in C, C#, and Go in the git repos, and it should be rather trivial to parse in any other language too.

> In Windows, logs would ideally get sent to Event Logging into a WireGuard log. That way, the user or administrator can use Event Viewer to view the log, forward the log,  or use a collector (like Splunk) to retrieve and aggregate the events.
> I'm not sure why WireGuard doesn't use Windows Event Logging. Is there something that precludes the use of Event Logging by WireGuard?

Event Logging appears to be rather slow and clunky, and I'm not sure I'd be too happy about blocking on that during packet events. It's also very cumbersome to use -- especially for things like crash dumps, which require a separate process or dll -- and the boilerplate involved doesn't look very appealing. In contrast, memory mapping a file, memcpy'ing buffers into it, and getting timestamps by reading 0x7ffe0014, means no calls to libraries or any interactions with other moving components that might be in an undefined state.

Event Logging might yet be possible to use, though. But it seems to me that'd be some significant research and development work, to figuring out how it could work in a lightweight way, and also revisiting the way wgnt logs things.

If this is something you'd like to work on, I'd be happy to review patches and read descriptions of a simplified event logging implementation. You're probably not the only user with this concern, and in theory I'd be open to considering it, provided there's a way to make it less clunky than initially meets the eye.

Jason

From coder at poorlab.com  Thu Oct 14 18:40:02 2021
From: coder at poorlab.com (StarBrilliant)
Date: Thu, 14 Oct 2021 18:40:02 +0000
Subject: Windows Log Output to Event Viewer or Text File
In-Reply-To: <PH0PR05MB79629688EB84BF07A2AEF54F99B89@PH0PR05MB7962.namprd05.prod.outlook.com>
References: <SA1PR05MB79668D58D886BB8D36F558FE99B29@SA1PR05MB7966.namprd05.prod.outlook.com>
 <CAHmME9pu=syh5zkig=e1KFQKMvCYfu6e+BhKENdwZARCrpss7Q@mail.gmail.com>
 <PH0PR05MB7962AD22BE333E844C0A79B799B69@PH0PR05MB7962.namprd05.prod.outlook.com>
 <CAHmME9pgbxyAxgBAfgLMVquzmPC9fiBzjWhQ5zJV+3bR2aR9dQ@mail.gmail.com>
 <PH0PR05MB79623EC9C12B4AE732B04E9299B79@PH0PR05MB7962.namprd05.prod.outlook.com>
 <CAHmME9q3j=2HG-4v5tpHRM9DCKpw+5iSMyzu30jAUHS1r=eDHw@mail.gmail.com>
 <PH0PR05MB79629688EB84BF07A2AEF54F99B89@PH0PR05MB7962.namprd05.prod.outlook.com>
Message-ID: <b1b2fc28-fee0-4801-ac42-fcd8d985e837@www.fastmail.com>

On Thu, Oct 14, 2021, at 17:41, Frank Wayne wrote:
> On Wed, Oct 13, 2021, at 18:16, Jason A. Donenfeld wrote:
> > Event Logging appears to be rather slow and clunky [...]

In fact, Windows Event Logging has two APIs: ETW and WPP.
The ETW API is, indeed, slow and clunky.
However, the WPP API is very high-performance. The trace function in Windows native TCP stack is implemented with WPP. 

If someone like Frank has the time and ability, they could check this MSDN documentation and try to implement it:
https://docs.microsoft.com/en-us/windows-hardware/drivers/devtest/wpp-software-tracing
However, I also think this feature is *not* a prioritized task, at least for average users.

I am not sure if I get Jason's idea: Is current Wireguard driver using a ring buffer of 2,048 messages for logging?
I am not sure if it has a notify mechanism: Otherwise, the userspace collector will have to poll the logs. Polling too fast consumes power, polling too slow may skip messages.


Best wishes,
StarBrilliant

From frank.wayne at northwestern.edu  Thu Oct 14 19:40:41 2021
From: frank.wayne at northwestern.edu (Frank Wayne)
Date: Thu, 14 Oct 2021 19:40:41 +0000
Subject: Windows Log Output to Event Viewer or Text File
In-Reply-To: <b1b2fc28-fee0-4801-ac42-fcd8d985e837@www.fastmail.com>
References: <SA1PR05MB79668D58D886BB8D36F558FE99B29@SA1PR05MB7966.namprd05.prod.outlook.com>
 <CAHmME9pu=syh5zkig=e1KFQKMvCYfu6e+BhKENdwZARCrpss7Q@mail.gmail.com>
 <PH0PR05MB7962AD22BE333E844C0A79B799B69@PH0PR05MB7962.namprd05.prod.outlook.com>
 <CAHmME9pgbxyAxgBAfgLMVquzmPC9fiBzjWhQ5zJV+3bR2aR9dQ@mail.gmail.com>
 <PH0PR05MB79623EC9C12B4AE732B04E9299B79@PH0PR05MB7962.namprd05.prod.outlook.com>
 <CAHmME9q3j=2HG-4v5tpHRM9DCKpw+5iSMyzu30jAUHS1r=eDHw@mail.gmail.com>
 <PH0PR05MB79629688EB84BF07A2AEF54F99B89@PH0PR05MB7962.namprd05.prod.outlook.com>
 <b1b2fc28-fee0-4801-ac42-fcd8d985e837@www.fastmail.com>
Message-ID: <PH0PR05MB7962D9FE88186BD52AE9774699B89@PH0PR05MB7962.namprd05.prod.outlook.com>

> I am not sure if I get Jason's idea: Is current Wireguard driver using a ring buffer of 2,048 messages for logging?

Yes. There is a file (relative to the installation directory) at Data\log.bin. It contains a WORD with "0BADBABE" (base 16, converted to big endian; a signature?), another WORD with something, followed by (in my case) 2050 structures of [a QWORD epoch time (with nanosecond precision) followed by 512 bytes of event text (zero padded)]. When I export the file in the WireGuard UI, it produces a list of 2048 events.

> I am not sure if it has a notify mechanism: Otherwise, the userspace collector will have to poll the logs. Polling too fast consumes power, polling too slow may skip messages.

Hear, hear! Alas, it does not have a notify mechanism.

Frank Wayne

-----Original Message-----
From: WireGuard <wireguard-bounces at lists.zx2c4.com> On Behalf Of StarBrilliant
Sent: Thursday, 14 October, 2021 13:40
To: wireguard at lists.zx2c4.com
Subject: Re: Windows Log Output to Event Viewer or Text File

On Thu, Oct 14, 2021, at 17:41, Frank Wayne wrote:
> On Wed, Oct 13, 2021, at 18:16, Jason A. Donenfeld wrote:
> > Event Logging appears to be rather slow and clunky [...]

In fact, Windows Event Logging has two APIs: ETW and WPP.
The ETW API is, indeed, slow and clunky.
However, the WPP API is very high-performance. The trace function in Windows native TCP stack is implemented with WPP. 

If someone like Frank has the time and ability, they could check this MSDN documentation and try to implement it:
https://urldefense.com/v3/__https://docs.microsoft.com/en-us/windows-hardware/drivers/devtest/wpp-software-tracing__;!!Dq0X2DkFhyF93HkjWTBQKhk!CMEOhPSNaRk9va55Sq3P6hrPlsaEZR9cKugdVaKMMSFkQVvmvAwTk-w9efcePl7WnfDRnWcQ$ 
However, I also think this feature is *not* a prioritized task, at least for average users.

I am not sure if I get Jason's idea: Is current Wireguard driver using a ring buffer of 2,048 messages for logging?
I am not sure if it has a notify mechanism: Otherwise, the userspace collector will have to poll the logs. Polling too fast consumes power, polling too slow may skip messages.


Best wishes,
StarBrilliant

From Jason at zx2c4.com  Thu Oct 14 19:52:50 2021
From: Jason at zx2c4.com (Jason A. Donenfeld)
Date: Thu, 14 Oct 2021 13:52:50 -0600
Subject: Windows Log Output to Event Viewer or Text File
In-Reply-To: <b1b2fc28-fee0-4801-ac42-fcd8d985e837@www.fastmail.com>
References: <SA1PR05MB79668D58D886BB8D36F558FE99B29@SA1PR05MB7966.namprd05.prod.outlook.com>
 <CAHmME9pu=syh5zkig=e1KFQKMvCYfu6e+BhKENdwZARCrpss7Q@mail.gmail.com>
 <PH0PR05MB7962AD22BE333E844C0A79B799B69@PH0PR05MB7962.namprd05.prod.outlook.com>
 <CAHmME9pgbxyAxgBAfgLMVquzmPC9fiBzjWhQ5zJV+3bR2aR9dQ@mail.gmail.com>
 <PH0PR05MB79623EC9C12B4AE732B04E9299B79@PH0PR05MB7962.namprd05.prod.outlook.com>
 <CAHmME9q3j=2HG-4v5tpHRM9DCKpw+5iSMyzu30jAUHS1r=eDHw@mail.gmail.com>
 <PH0PR05MB79629688EB84BF07A2AEF54F99B89@PH0PR05MB7962.namprd05.prod.outlook.com>
 <b1b2fc28-fee0-4801-ac42-fcd8d985e837@www.fastmail.com>
Message-ID: <CAHmME9qjs=38jss-DEwtOJ0tm3m9mn+4tsC8jdRk7AsUyTUPaQ@mail.gmail.com>

On Thu, Oct 14, 2021 at 12:43 PM StarBrilliant <coder at poorlab.com> wrote:
> In fact, Windows Event Logging has two APIs: ETW and WPP.
> The ETW API is, indeed, slow and clunky.
> However, the WPP API is very high-performance. The trace function in Windows native TCP stack is implemented with WPP.

Yes. I have no interest in using binary WPP traces. The kernel driver
now mimics linux's, having the exact same messaged logs in a simple
printk-like buffer.

> If someone like Frank has the time and ability, they could check this MSDN documentation and try to implement it:
> https://docs.microsoft.com/en-us/windows-hardware/drivers/devtest/wpp-software-tracing

Not interested. I won't take patches for that.

> I am not sure if I get Jason's idea: Is current Wireguard driver using a ring buffer of 2,048 messages for logging?

No. Frank is conflating the kernel driver and a simple userspace
service. The userspace service uses a very simple ringlogger format,
with multiple implementations, used for years on different platforms.
The kernel driver doesn't have an on-disk format; it uses a ring
buffer of sorts, but so far that remains irrelevant to this
discussion.

Jason

From Jason at zx2c4.com  Thu Oct 14 20:02:49 2021
From: Jason at zx2c4.com (Jason A. Donenfeld)
Date: Thu, 14 Oct 2021 14:02:49 -0600
Subject: Windows Log Output to Event Viewer or Text File
In-Reply-To: <PH0PR05MB79629688EB84BF07A2AEF54F99B89@PH0PR05MB7962.namprd05.prod.outlook.com>
References: <SA1PR05MB79668D58D886BB8D36F558FE99B29@SA1PR05MB7966.namprd05.prod.outlook.com>
 <CAHmME9pu=syh5zkig=e1KFQKMvCYfu6e+BhKENdwZARCrpss7Q@mail.gmail.com>
 <PH0PR05MB7962AD22BE333E844C0A79B799B69@PH0PR05MB7962.namprd05.prod.outlook.com>
 <CAHmME9pgbxyAxgBAfgLMVquzmPC9fiBzjWhQ5zJV+3bR2aR9dQ@mail.gmail.com>
 <PH0PR05MB79623EC9C12B4AE732B04E9299B79@PH0PR05MB7962.namprd05.prod.outlook.com>
 <CAHmME9q3j=2HG-4v5tpHRM9DCKpw+5iSMyzu30jAUHS1r=eDHw@mail.gmail.com>
 <PH0PR05MB79629688EB84BF07A2AEF54F99B89@PH0PR05MB7962.namprd05.prod.outlook.com>
Message-ID: <CAHmME9rMPgJ3E82nLoXyFYDT7DyjkLsdV3ange9sd_Qm+1-BNw@mail.gmail.com>

Hi Frank,

>From what I can see, none of what you've written describes what's
lacking in a potential command that would tail the wireguard log and
print it to standard out, perpetually. At least I couldn't figure it
out from a close read. As I wrote before, you could then pipe this
into event log or into any place else. You spoke more about why
snapshotted logs are problematic, and I'm inclined to agree with you
for a few reasons (though not the one you mentioned). But what I'm
suggesting is a tail mode, that keeps spitting out new logs as they
arrive. Are pipes problematic? Is there no ingestor that reads data
from stdin that would be convenient to you? Where does the tail
approach fall short? Does the message-based approach of event log
clash with the line-based approach of wireguard's unix-like logs? I
would like to fully understand what about this approach fails to meet
your design criteria.

One thing we could pretty easily do is add a "WireGuardEventLogger"
service, {un,}installable with:

> wireguard /installeventlogger
> wireguard /uninstalleventlogger

which would then scoop up the binary log as it grows and spit it out
to event logger, in real time. This wouldn't be too hard to do.
However, I would really like to first understand precisely what the
shortcomings would be in a simpler tail subcommand. That seems a lot
more versatile and simpler to implement too.

Thanks,
Jason

From frank.wayne at northwestern.edu  Thu Oct 14 21:45:01 2021
From: frank.wayne at northwestern.edu (Frank Wayne)
Date: Thu, 14 Oct 2021 21:45:01 +0000
Subject: Windows Log Output to Event Viewer or Text File
In-Reply-To: <CAHmME9rMPgJ3E82nLoXyFYDT7DyjkLsdV3ange9sd_Qm+1-BNw@mail.gmail.com>
References: <SA1PR05MB79668D58D886BB8D36F558FE99B29@SA1PR05MB7966.namprd05.prod.outlook.com>
 <CAHmME9pu=syh5zkig=e1KFQKMvCYfu6e+BhKENdwZARCrpss7Q@mail.gmail.com>
 <PH0PR05MB7962AD22BE333E844C0A79B799B69@PH0PR05MB7962.namprd05.prod.outlook.com>
 <CAHmME9pgbxyAxgBAfgLMVquzmPC9fiBzjWhQ5zJV+3bR2aR9dQ@mail.gmail.com>
 <PH0PR05MB79623EC9C12B4AE732B04E9299B79@PH0PR05MB7962.namprd05.prod.outlook.com>
 <CAHmME9q3j=2HG-4v5tpHRM9DCKpw+5iSMyzu30jAUHS1r=eDHw@mail.gmail.com>
 <PH0PR05MB79629688EB84BF07A2AEF54F99B89@PH0PR05MB7962.namprd05.prod.outlook.com>
 <CAHmME9rMPgJ3E82nLoXyFYDT7DyjkLsdV3ange9sd_Qm+1-BNw@mail.gmail.com>
Message-ID: <PH0PR05MB79620040C1298C80677C4BF999B89@PH0PR05MB7962.namprd05.prod.outlook.com>

Jason,

> From what I can see, none of what you've written describes what's lacking in a potential command that would tail the wireguard log and print it to standard out, perpetually. At least I couldn't figure it out from a close read. As I wrote before, you could then pipe this into event log or into any place else. You spoke more about why snapshotted logs are problematic, and I'm inclined to agree with you for a few reasons (though not the one you mentioned). But what I'm suggesting is a tail mode, that keeps spitting out new logs as they arrive. Are pipes problematic? Is there no ingestor that reads data from stdin that would be convenient to you? Where does the tail approach fall short? Does the message-based approach of event log clash with the line-based approach of wireguard's unix-like logs? I would like to fully understand what about this approach fails to meet your design criteria.

I think tail mode could work. It would be easier than parsing the binary file. I can write a script that runs wireguard.exe continuously with the tail option and collect stdout. Maybe it can be made to work universally with minimal effort, so that it works problem-free on whatever WireGuard hosts I push it to. If you write it, I'll try it.

> One thing we could pretty easily do is add a "WireGuardEventLogger"
service, {un,}installable with:

>> wireguard /installeventlogger
>> wireguard /uninstalleventlogger

>which would then scoop up the binary log as it grows and spit it out to event logger, in real time. This wouldn't be too hard to do.
>However, I would really like to first understand precisely what the shortcomings would be in a simpler tail subcommand. That seems a lot more versatile and simpler to implement too.

The service approach is interesting, too. It would be simpler to ingest.

In terms of shortcomings, consider the typical organization that wants to collect logs with (for example) Splunk.

They want to collect Exchange Mail logs. The Splunk administrator writes a monitor stanza for each Event Log and each directory with text logs, pushes it to the servers, and log events begin pouring in to indexers.

They want to collect Postfix logs. Same process, different OS.

They want to collect appliance syslogs. Set up forwarding to a Splunk listener and done.

They want to collect WireGuard logs. Well, now that's different, but only on Windows. Here, you need to write a script that runs software external to Splunk, and runs it continuously, and collects the output. Will every host have WireGuard installed? Forever? The script will have to check that the product is installed or else continuously generate errors. Is wireguard.exe in the PATH? For the user that Splunk runs under? Does that user have permissions to the WireGuard program directory? Should the script check the registry for the executable's location if it's not in the path? Can we run that script on endpoints without checking each team's security policy regarding in-house software running executables outside of its scope?

I don't know what all the shortcomings of a tail subcommand are, but these questions come to mind even before any development is started. So is tail more versatile and simpler? For whom?

That said, I think it would be a good start.

Frank Wayne

-----Original Message-----
From: Jason A. Donenfeld <Jason at zx2c4.com> 
Sent: Thursday, 14 October, 2021 15:03
To: Frank Wayne <frank.wayne at northwestern.edu>
Cc: WireGuard mailing list <wireguard at lists.zx2c4.com>
Subject: Re: Windows Log Output to Event Viewer or Text File

Hi Frank,

From what I can see, none of what you've written describes what's lacking in a potential command that would tail the wireguard log and print it to standard out, perpetually. At least I couldn't figure it out from a close read. As I wrote before, you could then pipe this into event log or into any place else. You spoke more about why snapshotted logs are problematic, and I'm inclined to agree with you for a few reasons (though not the one you mentioned). But what I'm suggesting is a tail mode, that keeps spitting out new logs as they arrive. Are pipes problematic? Is there no ingestor that reads data from stdin that would be convenient to you? Where does the tail approach fall short? Does the message-based approach of event log clash with the line-based approach of wireguard's unix-like logs? I would like to fully understand what about this approach fails to meet your design criteria.

One thing we could pretty easily do is add a "WireGuardEventLogger"
service, {un,}installable with:

> wireguard /installeventlogger
> wireguard /uninstalleventlogger

which would then scoop up the binary log as it grows and spit it out to event logger, in real time. This wouldn't be too hard to do.
However, I would really like to first understand precisely what the shortcomings would be in a simpler tail subcommand. That seems a lot more versatile and simpler to implement too.

Thanks,
Jason

From Jason at zx2c4.com  Thu Oct 14 21:56:48 2021
From: Jason at zx2c4.com (Jason A. Donenfeld)
Date: Thu, 14 Oct 2021 15:56:48 -0600
Subject: Windows Log Output to Event Viewer or Text File
In-Reply-To: <PH0PR05MB79620040C1298C80677C4BF999B89@PH0PR05MB7962.namprd05.prod.outlook.com>
References: <SA1PR05MB79668D58D886BB8D36F558FE99B29@SA1PR05MB7966.namprd05.prod.outlook.com>
 <CAHmME9pu=syh5zkig=e1KFQKMvCYfu6e+BhKENdwZARCrpss7Q@mail.gmail.com>
 <PH0PR05MB7962AD22BE333E844C0A79B799B69@PH0PR05MB7962.namprd05.prod.outlook.com>
 <CAHmME9pgbxyAxgBAfgLMVquzmPC9fiBzjWhQ5zJV+3bR2aR9dQ@mail.gmail.com>
 <PH0PR05MB79623EC9C12B4AE732B04E9299B79@PH0PR05MB7962.namprd05.prod.outlook.com>
 <CAHmME9q3j=2HG-4v5tpHRM9DCKpw+5iSMyzu30jAUHS1r=eDHw@mail.gmail.com>
 <PH0PR05MB79629688EB84BF07A2AEF54F99B89@PH0PR05MB7962.namprd05.prod.outlook.com>
 <CAHmME9rMPgJ3E82nLoXyFYDT7DyjkLsdV3ange9sd_Qm+1-BNw@mail.gmail.com>
 <PH0PR05MB79620040C1298C80677C4BF999B89@PH0PR05MB7962.namprd05.prod.outlook.com>
Message-ID: <CAHmME9q9VB7eEsZCFUqo8XG7qj1pe2RwgLezJEHueF4_AjmA4A@mail.gmail.com>

Hi Frank,

On Thu, Oct 14, 2021 at 3:45 PM Frank Wayne
<frank.wayne at northwestern.edu> wrote:
> The service approach is interesting, too. It would be simpler to ingest.

I started writing this, and got something basically working, but got a
bit tripped up in annoying lifetime issues of registering an event log
"source" in the registry, and the boggling configurability there. I'm
kind of shying away from it after an initial stab...

To answer the more concrete questions about a tail approach:

> Will every host have WireGuard installed? Forever?

This is a tougie, I guess in the same way that scooping up
non-streamed file-based logs are: at some point you have to do a sweep
to see if there are new files to grab, or in this case, if wireguard
has been installed. So on one hand, "polling" is pretty gnarly, but on
the other hand, you do that anyway for file-based logs I imagine.
There are probably other SCM-based or MSI-event based ways of doing
this without polling that are more complicated.

Does Splunk have any condition logic like, "do this collection routine
if file F exists; otherwise wait to do it until it exists"? If that
kind of thing is built in, then you're done. If not I agree this is an
annoying point.

> Is wireguard.exe in the PATH? For the user that Splunk runs under?

Yes. It's added to the system PATH.

> Should the script check the registry for the executable's location if it's not in the path?

No. The installer always sets PATH.

> Does that user have permissions to the WireGuard program directory?

Hmm. Here indeed is where the granular decoupled permission system of
Event Log comes in handy, I suppose. But in theory you should be able
to do the same for wireguard's log:
"%PROGRAMFILES%\wireguard\data\log.bin" is just a file path like any
other, and you can adjust its permissions accordingly. By default its
parent directory is O:SYG:SYD:PAI(A;OICI;FA;;;SY)(A;OICI;FA;;;BA), but
you could set a more particular ACL for log.bin itself.

> Can we run that script on endpoints without checking each team's security policy regarding in-house software running executables outside of its scope?

I imagine probably so in the sense that wireguard.exe is already
executed several times in a few different funny roles, so things might
already be sufficiently permissive. Have you seen any wireguard.exe
policies being passed around places? If you've got a link, I'd be
curious to see what people are doing.

Jason

From heroxbd at gentoo.org  Fri Oct 15 02:39:01 2021
From: heroxbd at gentoo.org (Benda Xu)
Date: Fri, 15 Oct 2021 10:39:01 +0800
Subject: Source IP for multihomed peer
In-Reply-To: <d0e1773d-80d5-7676-7943-2eff8d57beaa@kracon.dk> (Svenne Krap's
 message of "Thu, 14 Oct 2021 10:30:27 +0200")
References: <d0e1773d-80d5-7676-7943-2eff8d57beaa@kracon.dk>
Message-ID: <87wnmfj9kq.fsf@proton.d.airelinux.org>

Hi Svenne,

We have met exactly the same problem.

Svenne Krap <svenne at kracon.dk> writes:

> [...]
>
> My question is twofold:
>
> 1) Does the above seem like a likely chain of events?
>
> 2) Is there any way to force the source ip of the connection from boxA
> to always use address boxA1 ?
>
> From the documentation Listenport only seems like the portnumber and
> there seems to be no way to set the source ip.

It has been discussed on the list several times.  But Jason seems not
convinced of the necessity of address binding.

  https://lists.zx2c4.com/pipermail/wireguard/2017-May/001280.html
  https://lists.zx2c4.com/pipermail/wireguard/2019-March/003938.html
  https://lists.zx2c4.com/pipermail/wireguard/2018-June/003013.html
  https://lists.zx2c4.com/pipermail/wireguard/2017-November/002017.html

Rulin and I tried to implement an address binding feature at,

  https://github.com/FireflyTang/linux-wireguard-bind

It was verified to work with Linux-5.7.

Yours,
Benda

From ch at ntrv.dk  Fri Oct 15 07:57:20 2021
From: ch at ntrv.dk (Chriztoffer Hansen)
Date: Fri, 15 Oct 2021 09:57:20 +0200
Subject: Source IP for multihomed peer
In-Reply-To: <87wnmfj9kq.fsf@proton.d.airelinux.org>
References: <d0e1773d-80d5-7676-7943-2eff8d57beaa@kracon.dk>
 <87wnmfj9kq.fsf@proton.d.airelinux.org>
Message-ID: <CA+cYV6t=doigei7mH-77oZrq=_uMmCn_kRxhfXku+XC75moU=A@mail.gmail.com>

On Fri, 15 Oct 2021 at 04:39, Benda Xu <heroxbd at gentoo.org> wrote:
> > From the documentation Listenport only seems like the portnumber and
> > there seems to be no way to set the source ip.
>
> It has been discussed on the list several times.  But Jason seems not
> convinced of the necessity of address binding.
>
>   https://lists.zx2c4.com/pipermail/wireguard/2017-May/001280.html
>   https://lists.zx2c4.com/pipermail/wireguard/2019-March/003938.html
>   https://lists.zx2c4.com/pipermail/wireguard/2018-June/003013.html
>   https://lists.zx2c4.com/pipermail/wireguard/2017-November/002017.html
>
> Rulin and I tried to implement an address binding feature at,
>
>   https://github.com/FireflyTang/linux-wireguard-bind
>
> It was verified to work with Linux-5.7.

In the prototyping patch you developed for WireGuard, did you consider
prototyping being able to bind to an interface, instead of explicitly
specifying an IP address? An example case for being able to bind to an
interface could be a multi-wan connected firewall/router with dynamic
public IP addresses offered by the upstream provider no at least one
connection. E.g. Primary fiber line, backup DSL line.

/Chriztoffer


From heroxbd at gentoo.org  Fri Oct 15 08:25:11 2021
From: heroxbd at gentoo.org (Benda Xu)
Date: Fri, 15 Oct 2021 16:25:11 +0800
Subject: Source IP for multihomed peer
In-Reply-To: <CA+cYV6t=doigei7mH-77oZrq=_uMmCn_kRxhfXku+XC75moU=A@mail.gmail.com>
 (Chriztoffer Hansen's message of "Fri, 15 Oct 2021 09:57:20 +0200")
References: <d0e1773d-80d5-7676-7943-2eff8d57beaa@kracon.dk>
 <87wnmfj9kq.fsf@proton.d.airelinux.org>
 <CA+cYV6t=doigei7mH-77oZrq=_uMmCn_kRxhfXku+XC75moU=A@mail.gmail.com>
Message-ID: <871r4mk848.fsf@proton.d.airelinux.org>

Hi Chriztoffer,

Chriztoffer Hansen <ch at ntrv.dk> writes:

> In the prototyping patch you developed for WireGuard, did you consider
> prototyping being able to bind to an interface, instead of explicitly
> specifying an IP address? 

No we did not.  We wanted to keep the patch as small as possible.  

That said, we could manage it by an auxiliary script.

> An example case for being able to bind to an interface could be a
> multi-wan connected firewall/router with dynamic public IP addresses
> offered by the upstream provider no at least one
> connection. E.g. Primary fiber line, backup DSL line.

Yeah, that's a common use case.

Cheers,
Benda

From svenne at kracon.dk  Fri Oct 15 08:54:59 2021
From: svenne at kracon.dk (Svenne Krap)
Date: Fri, 15 Oct 2021 10:54:59 +0200
Subject: Source IP for multihomed peer
In-Reply-To: <87wnmfj9kq.fsf@proton.d.airelinux.org>
References: <d0e1773d-80d5-7676-7943-2eff8d57beaa@kracon.dk>
 <87wnmfj9kq.fsf@proton.d.airelinux.org>
Message-ID: <5ec4906f-db2b-363a-db38-ac7179af83b5@kracon.dk>

On 15.10.2021 04.39, Benda Xu wrote:
> We have met exactly the same problem.

Have you tried SNAT from iptables? Is that a viable work-around?


Svenne


From toke at toke.dk  Fri Oct 15 10:14:31 2021
From: toke at toke.dk (Toke =?utf-8?Q?H=C3=B8iland-J=C3=B8rgensen?=)
Date: Fri, 15 Oct 2021 12:14:31 +0200
Subject: Source IP for multihomed peer
In-Reply-To: <d0e1773d-80d5-7676-7943-2eff8d57beaa@kracon.dk>
References: <d0e1773d-80d5-7676-7943-2eff8d57beaa@kracon.dk>
Message-ID: <87ee8m1to8.fsf@toke.dk>

> 2) Is there any way to force the source ip of the connection from boxA 
> to always use address boxA1 ?

In theory this should be possible to enforce via policy routing. Just
tried this on a simple veth setup:

# ip a add 10.11.1.1/24 dev veth0
# ip a add 10.11.2.1/24 dev veth0
# ping 10.11.1.2 -c 1
12:09:22.385888 IP 10.11.1.1 > 10.11.1.2: ICMP echo request, id 15, seq 1, length 64
12:09:22.385903 IP 10.11.1.2 > 10.11.1.1: ICMP echo reply, id 15, seq 1, length 64

# ip r add 10.11.1.2 src 10.11.2.1 dev veth0
# ping 10.11.1.2 -c 1
12:09:53.251386 IP 10.11.2.1 > 10.11.1.2: ICMP echo request, id 16, seq 1, length 64
12:09:53.251403 IP 10.11.1.2 > 10.11.2.1: ICMP echo reply, id 16, seq 1, length 64

I think this ought to work for wireguard's source selection as well. If
you don't have a particular destination, you should be able to do
something similar based on sports with ip-rule using the wireguard
source port:

# ip rule add sport 1234 lookup 100
# ip route add table 100 default via 1.2.3.4 src 3.4.5.6

That last bit I didn't test, though...

-Toke

From ch at ntrv.dk  Fri Oct 15 11:14:45 2021
From: ch at ntrv.dk (Chriztoffer Hansen)
Date: Fri, 15 Oct 2021 13:14:45 +0200
Subject: Source IP for multihomed peer
In-Reply-To: <87ee8m1to8.fsf@toke.dk>
References: <d0e1773d-80d5-7676-7943-2eff8d57beaa@kracon.dk>
 <87ee8m1to8.fsf@toke.dk>
Message-ID: <CA+cYV6vZsbkKyPGvkEF+WbJgi5KQhS46X0yvB3ExfXzr8g4i3Q@mail.gmail.com>

On Fri, 15 Oct 2021 at 12:14, Toke H?iland-J?rgensen <toke at toke.dk> wrote:
> > 2) Is there any way to force the source ip of the connection from boxA
> > to always use address boxA1 ?
>
> In theory this should be possible to enforce via policy routing. Just
> tried this on a simple veth setup:
>
> # ip a add 10.11.1.1/24 dev veth0
> # ip a add 10.11.2.1/24 dev veth0
> # ping 10.11.1.2 -c 1
> 12:09:22.385888 IP 10.11.1.1 > 10.11.1.2: ICMP echo request, id 15, seq 1, length 64
> 12:09:22.385903 IP 10.11.1.2 > 10.11.1.1: ICMP echo reply, id 15, seq 1, length 64
>
> # ip r add 10.11.1.2 src 10.11.2.1 dev veth0
> # ping 10.11.1.2 -c 1
> 12:09:53.251386 IP 10.11.2.1 > 10.11.1.2: ICMP echo request, id 16, seq 1, length 64
> 12:09:53.251403 IP 10.11.1.2 > 10.11.2.1: ICMP echo reply, id 16, seq 1, length 64
>
> I think this ought to work for wireguard's source selection as well. If
> you don't have a particular destination, you should be able to do
> something similar based on sports with ip-rule using the wireguard
> source port:
>
> # ip rule add sport 1234 lookup 100
> # ip route add table 100 default via 1.2.3.4 src 3.4.5.6
>
> That last bit I didn't test, though...

Will have to test this later.

If this works. This suggestion would be a great enhancement to wireguard-tools?


From frank.wayne at northwestern.edu  Fri Oct 15 13:25:21 2021
From: frank.wayne at northwestern.edu (Frank Wayne)
Date: Fri, 15 Oct 2021 13:25:21 +0000
Subject: Windows Log Output to Event Viewer or Text File
In-Reply-To: <CAHmME9q9VB7eEsZCFUqo8XG7qj1pe2RwgLezJEHueF4_AjmA4A@mail.gmail.com>
References: <SA1PR05MB79668D58D886BB8D36F558FE99B29@SA1PR05MB7966.namprd05.prod.outlook.com>
 <CAHmME9pu=syh5zkig=e1KFQKMvCYfu6e+BhKENdwZARCrpss7Q@mail.gmail.com>
 <PH0PR05MB7962AD22BE333E844C0A79B799B69@PH0PR05MB7962.namprd05.prod.outlook.com>
 <CAHmME9pgbxyAxgBAfgLMVquzmPC9fiBzjWhQ5zJV+3bR2aR9dQ@mail.gmail.com>
 <PH0PR05MB79623EC9C12B4AE732B04E9299B79@PH0PR05MB7962.namprd05.prod.outlook.com>
 <CAHmME9q3j=2HG-4v5tpHRM9DCKpw+5iSMyzu30jAUHS1r=eDHw@mail.gmail.com>
 <PH0PR05MB79629688EB84BF07A2AEF54F99B89@PH0PR05MB7962.namprd05.prod.outlook.com>
 <CAHmME9rMPgJ3E82nLoXyFYDT7DyjkLsdV3ange9sd_Qm+1-BNw@mail.gmail.com>
 <PH0PR05MB79620040C1298C80677C4BF999B89@PH0PR05MB7962.namprd05.prod.outlook.com>
 <CAHmME9q9VB7eEsZCFUqo8XG7qj1pe2RwgLezJEHueF4_AjmA4A@mail.gmail.com>
Message-ID: <PH0PR05MB7962AD9044A813DE82AE796A99B99@PH0PR05MB7962.namprd05.prod.outlook.com>

Jason,

Please understand that the questions, although real concerns, are rhetorical in the context of this discussion. I know the answers to them, but they are intended to serve as an example of why I consider the tail option only a partial solution. This is not to dissuade you from implementing it; it is merely a response to your incomprehension of "why it fails to meet" my expectations of convenience, etc.
 
Frank Wayne

From matthew.poletiek at gmail.com  Fri Oct 15 19:58:41 2021
From: matthew.poletiek at gmail.com (Matthew Poletiek)
Date: Fri, 15 Oct 2021 14:58:41 -0500
Subject: [WARNING - iOS 15] Update WireGuard to >=1.0.15-26 *before*
 updating to iOS 15
In-Reply-To: <CAHmME9p9XxAhh4WgZgRiMRFxahp2wabPr-KuW=AdVQyRsHaD+A@mail.gmail.com>
References: <CAHmME9p9XxAhh4WgZgRiMRFxahp2wabPr-KuW=AdVQyRsHaD+A@mail.gmail.com>
Message-ID: <CAJhSV8uZ3nGOL9-oUu=wL1CiY7Eeux0UYzsL4psMWeR1D73VMQ@mail.gmail.com>

Hi,

Thanks for the update Jason.

Unfortunately I wasn't reading and didn't prepare properly. In XCode,
does anyone know the best way to upgrade the wireguard-apple package
so my custom app can leverage the latest fixes?

Thanks again,
-------------------------------------------
Matthew Poletiek
303.810.9082
matthew.poletiek at gmail.com
www.matthewpoletiek.com



On Thu, Sep 23, 2021 at 9:25 PM Jason A. Donenfeld <Jason at zx2c4.com> wrote:
>
> Hello list,
>
> Due to buggy changes Apple has made to the Keychain on iOS 15, older
> versions of WireGuard on iOS will *eat your VPN configurations* when
> upgrading to iOS 15. This has been fixed by [1], which is now
> available as of version ?1.0.15-26. If you are still on iOS 14, before
> updating to iOS 15, please remember to update the WireGuard app to
> this latest version in the App Store.
>
> Sorry for the hassle.
>
> Jason
>
> [1] https://git.zx2c4.com/wireguard-apple/commit/?id=03a59ff38e96fb3bb5dde2f15fe42198d1dfb995

From matthew.poletiek at gmail.com  Fri Oct 15 21:17:07 2021
From: matthew.poletiek at gmail.com (Matthew Poletiek)
Date: Fri, 15 Oct 2021 16:17:07 -0500
Subject: [WARNING - iOS 15] Update WireGuard to >=1.0.15-26 *before*
 updating to iOS 15
In-Reply-To: <CAJhSV8uZ3nGOL9-oUu=wL1CiY7Eeux0UYzsL4psMWeR1D73VMQ@mail.gmail.com>
References: <CAHmME9p9XxAhh4WgZgRiMRFxahp2wabPr-KuW=AdVQyRsHaD+A@mail.gmail.com>
 <CAJhSV8uZ3nGOL9-oUu=wL1CiY7Eeux0UYzsL4psMWeR1D73VMQ@mail.gmail.com>
Message-ID: <CAJhSV8vEJs4zhBTzKP2n8n4db87bBmBpFys9C5UYPkMdDbX+pg@mail.gmail.com>

Disregard. I figured it out. I have a different issue now, but am
working on more details. Tunnel immediately disconnects after
connecting.

-------------------------------------------
Matthew Poletiek
303.810.9082
matthew.poletiek at gmail.com
www.matthewpoletiek.com



On Fri, Oct 15, 2021 at 2:58 PM Matthew Poletiek
<matthew.poletiek at gmail.com> wrote:
>
> Hi,
>
> Thanks for the update Jason.
>
> Unfortunately I wasn't reading and didn't prepare properly. In XCode,
> does anyone know the best way to upgrade the wireguard-apple package
> so my custom app can leverage the latest fixes?
>
> Thanks again,
> -------------------------------------------
> Matthew Poletiek
> 303.810.9082
> matthew.poletiek at gmail.com
> www.matthewpoletiek.com
>
>
>
> On Thu, Sep 23, 2021 at 9:25 PM Jason A. Donenfeld <Jason at zx2c4.com> wrote:
> >
> > Hello list,
> >
> > Due to buggy changes Apple has made to the Keychain on iOS 15, older
> > versions of WireGuard on iOS will *eat your VPN configurations* when
> > upgrading to iOS 15. This has been fixed by [1], which is now
> > available as of version ?1.0.15-26. If you are still on iOS 14, before
> > updating to iOS 15, please remember to update the WireGuard app to
> > this latest version in the App Store.
> >
> > Sorry for the hassle.
> >
> > Jason
> >
> > [1] https://git.zx2c4.com/wireguard-apple/commit/?id=03a59ff38e96fb3bb5dde2f15fe42198d1dfb995

From aavery77 at hotmail.com  Sat Oct 16 20:59:30 2021
From: aavery77 at hotmail.com (Aaron Avery)
Date: Sat, 16 Oct 2021 15:59:30 -0500
Subject: [PATCH] Fixed null pointer exception when user namespace is empty
Message-ID: <DM6PR01MB4444144DA39B5197C71D9762D7BA9@DM6PR01MB4444.prod.exchangelabs.com>

---
I compiled the Wireguard kernel module for my QNAP NAS running
version 4.14.24. When creating the network device, it got a null pointer
exception. I figured out that the user namespace is null on this system
and was being passed into ns_capable as-is, crashing the kernel (somewhat).
After applying this change, I finally have Wireguard up and running
after years of wishing I had it available instead of OpenVPN.

I'm not a Linux expert so if there's a better way to handle this
situation (such as checking for root instead of CAP_NET_ADMIN when
user_ns doesn't exist), let me know and I can try it and submit
a different patch.
Otherwise, it seems like this could be applied to both
wireguard-linux-compat and wireguard-linux for maximum system
compatibility going forward.

 src/netlink.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/src/netlink.c b/src/netlink.c
index ef239ab..688e41f 100644
--- a/src/netlink.c
+++ b/src/netlink.c
@@ -513,7 +513,7 @@ static int wg_set_device(struct sk_buff *skb, struct genl_info *info)
 		struct net *net;
 		rcu_read_lock();
 		net = rcu_dereference(wg->creating_net);
-		ret = !net || !ns_capable(net->user_ns, CAP_NET_ADMIN) ? -EPERM : 0;
+		ret = !net || (net->user_ns && !ns_capable(net->user_ns, CAP_NET_ADMIN)) ? -EPERM : 0;
 		rcu_read_unlock();
 		if (ret)
 			goto out;
-- 
2.33.0


From Jason at zx2c4.com  Sun Oct 17 00:52:32 2021
From: Jason at zx2c4.com (Jason A. Donenfeld)
Date: Sat, 16 Oct 2021 18:52:32 -0600
Subject: [PATCH] Fixed null pointer exception when user namespace is empty
In-Reply-To: <DM6PR01MB4444144DA39B5197C71D9762D7BA9@DM6PR01MB4444.prod.exchangelabs.com>
References: <DM6PR01MB4444144DA39B5197C71D9762D7BA9@DM6PR01MB4444.prod.exchangelabs.com>
Message-ID: <CAHmME9p=qmQhJ5SQRhorjZJzVDpaX7BK1EA7douTwY856y9t1A@mail.gmail.com>

Hi Aaron,

That patch is missing your Signed-off-by line, so I won't be able to
take it as-is.

However, I also wonder if it makes sense. I just grepped the entire
kernel and I couldn't find any other instances of net->user_ns being
NULL checked. Is it possible that there's a bug in QNAP's kernel
somewhere? Or you're compiling against the wrong .config so the struct
offsets are wrong? Or something else? When should net->user_ns be
NULL?

Thanks,
Jason

From Jason at zx2c4.com  Sun Oct 17 05:08:52 2021
From: Jason at zx2c4.com (Jason A. Donenfeld)
Date: Sat, 16 Oct 2021 23:08:52 -0600
Subject: [ANNOUNCE] WireGuardNT, a high-performance WireGuard
 implementation for the Windows kernel
In-Reply-To: <YQgrCfqIL2SWfF6G@zx2c4.com>
References: <YQgrCfqIL2SWfF6G@zx2c4.com>
Message-ID: <CAHmME9rL7FT=fmrV-Vv7RgWd5dvoPo29tK8ctbhG7KddvLXZ7A@mail.gmail.com>

Hi folks,

As described in August:

On Mon, Aug 2, 2021 at 11:28 AM Jason A. Donenfeld <Jason at zx2c4.com> wrote:
>   Phase 3) WireGuardNT is enabled, and wireguard-go/Wintun is removed from
>            the client. [Do note: as projects and codebases, both Wintun and
>            wireguard-go will continue to be maintained, as they have
>            applications and uses outside of our WireGuard client, and Wintun
>            has uses outside of WireGuard in general.]

After a considerable amount of testing and 11 releases, we are now in
phase 3, and the transition is complete. WireGuard for Windows v0.5 is
being released as we speak, and contains WireGuardNT 0.10.1, which is
now fully WHQL Certified and will run on all flavors of Windows
Server.

Thank you very much to everyone who helped test and stabilize this code.

Regards,
Jason

From aavery77 at hotmail.com  Sun Oct 17 20:27:04 2021
From: aavery77 at hotmail.com (Aaron Avery)
Date: Sun, 17 Oct 2021 20:27:04 +0000
Subject: [PATCH] Fixed null pointer exception when user namespace is empty
In-Reply-To: <CAHmME9p=qmQhJ5SQRhorjZJzVDpaX7BK1EA7douTwY856y9t1A@mail.gmail.com>
References: <DM6PR01MB4444144DA39B5197C71D9762D7BA9@DM6PR01MB4444.prod.exchangelabs.com>
 <CAHmME9p=qmQhJ5SQRhorjZJzVDpaX7BK1EA7douTwY856y9t1A@mail.gmail.com>
Message-ID: <DM6PR01MB44446FF0D5FB0D11873C337ED7BB9@DM6PR01MB4444.prod.exchangelabs.com>

Thank you, Jason. Sorry I missed the Signed-off-by. As I said, I'm not that familiar with Linux development.
I'm quite sure I have the correct .config for my QNAP's kernel, and the WG kernel module I built works perfectly, with this patch. All of the *_NS entries in their config file are enabled.

After more research, I agree with you completely that this could easily be a QNAP-specific bug so shouldn't be merged. As a NAS, they probably have some custom network driver code and may have failed to initialize user_ns in this situation. The only somewhat-relevant kernel commit I could find for rtnetlink.c is https://github.com/torvalds/linux/commit/f428fe4a04cc339166c8bbd489789760de3a0cee and version 4.14.24 is more recent than that.

A lot of people like me use a QNAP NAS as their always-on "home server". Hopefully Google will crawl this message list so that tech-savvy people searching for Wireguard QNAP will find this patch and have a chance to use WG to remotely access their home network.

- Aaron

From: Jason A. Donenfeld <Jason at zx2c4.com>
Sent: Saturday, October 16, 2021 7:52 PM
To: Aaron Avery <aavery77 at hotmail.com>
Cc: WireGuard mailing list <wireguard at lists.zx2c4.com>
Subject: Re: [PATCH] Fixed null pointer exception when user namespace is empty 
?
Hi Aaron,

That patch is missing your Signed-off-by line, so I won't be able to
take it as-is.

However, I also wonder if it makes sense. I just grepped the entire
kernel and I couldn't find any other instances of net->user_ns being
NULL checked. Is it possible that there's a bug in QNAP's kernel
somewhere? Or you're compiling against the wrong .config so the struct
offsets are wrong? Or something else? When should net->user_ns be
NULL?

Thanks,
Jason

From david at kerr.net  Mon Oct 18 01:04:48 2021
From: david at kerr.net (David Kerr)
Date: Sun, 17 Oct 2021 21:04:48 -0400
Subject: [PATCH] Fixed null pointer exception when user namespace is empty
In-Reply-To: <DM6PR01MB44446FF0D5FB0D11873C337ED7BB9@DM6PR01MB4444.prod.exchangelabs.com>
References: <DM6PR01MB4444144DA39B5197C71D9762D7BA9@DM6PR01MB4444.prod.exchangelabs.com>
 <CAHmME9p=qmQhJ5SQRhorjZJzVDpaX7BK1EA7douTwY856y9t1A@mail.gmail.com>
 <DM6PR01MB44446FF0D5FB0D11873C337ED7BB9@DM6PR01MB4444.prod.exchangelabs.com>
Message-ID: <CAJJxGdFsd+xr4rVBg79kQdx2Zqx1owfkQwbeuxkJ7khCVjyv2w@mail.gmail.com>

My QNAP system just recently did a system update to QTS version 5
which now includes Wireguard as part of their QVPN service.  I have
not tried it, but it is there along with the usual suspects of
OpenVPN, IPSec, PPTP, etc.

DAK.


On Sun, Oct 17, 2021 at 4:30 PM Aaron Avery <aavery77 at hotmail.com> wrote:
>
> Thank you, Jason. Sorry I missed the Signed-off-by. As I said, I'm not that familiar with Linux development.
> I'm quite sure I have the correct .config for my QNAP's kernel, and the WG kernel module I built works perfectly, with this patch. All of the *_NS entries in their config file are enabled.
>
> After more research, I agree with you completely that this could easily be a QNAP-specific bug so shouldn't be merged. As a NAS, they probably have some custom network driver code and may have failed to initialize user_ns in this situation. The only somewhat-relevant kernel commit I could find for rtnetlink.c is https://github.com/torvalds/linux/commit/f428fe4a04cc339166c8bbd489789760de3a0cee and version 4.14.24 is more recent than that.
>
> A lot of people like me use a QNAP NAS as their always-on "home server". Hopefully Google will crawl this message list so that tech-savvy people searching for Wireguard QNAP will find this patch and have a chance to use WG to remotely access their home network.
>
> - Aaron
>
> From: Jason A. Donenfeld <Jason at zx2c4.com>
> Sent: Saturday, October 16, 2021 7:52 PM
> To: Aaron Avery <aavery77 at hotmail.com>
> Cc: WireGuard mailing list <wireguard at lists.zx2c4.com>
> Subject: Re: [PATCH] Fixed null pointer exception when user namespace is empty
>
> Hi Aaron,
>
> That patch is missing your Signed-off-by line, so I won't be able to
> take it as-is.
>
> However, I also wonder if it makes sense. I just grepped the entire
> kernel and I couldn't find any other instances of net->user_ns being
> NULL checked. Is it possible that there's a bug in QNAP's kernel
> somewhere? Or you're compiling against the wrong .config so the struct
> offsets are wrong? Or something else? When should net->user_ns be
> NULL?
>
> Thanks,
> Jason

From and at mullvad.net  Tue Oct 19 09:54:17 2021
From: and at mullvad.net (Andrej Mihajlov)
Date: Tue, 19 Oct 2021 11:54:17 +0200
Subject: [wireguard-apple] [iOS] Changing network fails with
 includeAllNetworks (Kill Switch)
In-Reply-To: <85F39D56-E800-4EAE-B159-85335D150A4A@mullvad.net>
References: <B3A4A27C-1FBF-4098-B38F-9F87F8C115B5@gmail.com>
 <5569D8DF-A7E8-4A13-BA6B-913785819E4E@mullvad.net>
 <CB84BD03-DB8A-4855-94CC-898100C0B97E@gmail.com>
 <9D47F45A-59C7-4FB0-A9F8-2F13A1D4AB1A@mullvad.net>
 <E65FFE7D-9BCC-4EEF-9CF0-FA46CE6B9096@gmail.com>
 <85F39D56-E800-4EAE-B159-85335D150A4A@mullvad.net>
Message-ID: <66E63699-8CC0-4382-ADD3-1AC878898D8C@mullvad.net>

Follow up on this. It looks like the VPN connection breaks on my iPad running iPadOS 15 after changing DNS settings via WireGuardKit. 

Tested on iPadOS 15.1 beta today and it seems to be stable.

> On 28 Sep 2021, at 13:03, Andrej Mihajlov <and at mullvad.net> wrote:
> 
> Hi,
> 
> I can confirm that it behaves correctly on iOS 15 (tested on iPhone 12) and iOS 15.1 beta (tested on iPhone 7). Tested by toggling cellular/wi-fi and airplane mode on both devices and network monitor seems to be functioning properly. 
> 
> I haven?t tested this patch on iOS 14.8, but I had previously tested it on iOS 14.5 (IIRC) and it didn?t work there, that's why this patch is scoped to iOS 15+.
> 
> I am running the "am/enable-include-all-networks" branch which has the following changeset:
> https://git.zx2c4.com/wireguard-apple/commit/?id=07bc66e7b181fb2068d457b31c1fdd05bdd2214a&id2=58e94f077329f6c7b96ec069243495d4e649fe36
> 
> Cheers,
> Andrej
> 
>> On 22 Sep 2021, at 15:26, Juraj Hilje <juraj.hilje at gmail.com> wrote:
>> 
>> Hi Andrej,
>> 
>> I've tested on iOS/iPadOS 15.1 Beta, and it looks like the issue is fixed there.
>> Let me know if you can confirm the same on your end.
>> 
>> Cheers,
>> Juraj H.
>> 
>>> On 22.09.2021., at 10:59, Andrej Mihajlov <and at mullvad.net> wrote:
>>> 
>>> Hi Juraj,
>>> 
>>> Installing iOS 15 right now. I am gonna test it today too. 
>>> 
>>> What stands out to me that, while you have multiple interfaces available, the network monitor still says that the network is unsatisfied. Very odd.
>>> 
>>> Cheers,
>>> Andrej
>>> 
>>>> On 22 Sep 2021, at 10:55, Juraj Hilje <juraj.hilje at gmail.com> wrote:
>>>> 
>>>> Hey Andrej, thanks for the response!
>>>> 
>>>> I've tested on iOS 14.8 and iOS 15.0 (public release), and even with the patch (b244febfdf3069dd4e8db2d31f0368d5474d7616) i still have the same issue on my end.
>>>> 
>>>> I will test the new iOS 15.1 Beta later today and let you know how it goes.
>>>> 
>>>> Juraj H.
>>>> 
>>>>> On 22.09.2021., at 10:08, Andrej Mihajlov <and at mullvad.net> wrote:
>>>>> 
>>>>> Have you tried on the most recent beta? I think it works over there, but requires some tweaks to the network monitor code in WireGuard. I had a patch somewhere here but haven?t spent much time testing it:
>>>>> 
>>>>> https://git.zx2c4.com/wireguard-apple/commit/?h=am/enable-include-all-networks&id=b244febfdf3069dd4e8db2d31f0368d5474d7616
>>>>> 
>>>>> Waiting for the final release of iOS 15.
>>>>> 
>>>>>> On 21 Sep 2021, at 12:55, Juraj Hilje <juraj.hilje at gmail.com> wrote:
>>>>>> 
>>>>>> If NETunnelProviderProtocol is configured with includeAllNetworks=true (Kill Switch), when network change is detected the device connectivity goes offline instead of routing VPN tunnel traffic through a new network.
>>>>>> 
>>>>>> Here are some logs from the moment of this event:
>>>>>> 2021-09-20 12:07:26.735453: [NET] Network change detected with unsatisfied route and interface order [en0, utun4, pdp_ip0]
>>>>>> 2021-09-20 12:07:26.736186: [NET] Connectivity offline, pausing backend.
>>>>>> 2021-09-20 12:07:26.736732: [NET] Device closing
>>>>>> 2021-09-20 12:07:26.737503: [NET] Routine: TUN reader - stopped
>>>>>> 2021-09-20 12:07:26.738970: [NET] Routine: event worker - stopped
>>>>>> 2021-09-20 12:07:26.739613: [NET] Routine: receive incoming v4 - stopped
>>>>>> 2021-09-20 12:07:26.742070: [NET] Routine: receive incoming v6 - stopped
>>>>>> 2021-09-20 12:07:26.746712: [NET] peer(eN1f?Oymc) - Stopping
>>>>>> 2021-09-20 12:07:26.751550: [NET] peer(eN1f?Oymc) - Routine: sequential receiver - stopped
>>>>>> 2021-09-20 12:07:26.751597: [NET] peer(eN1f?Oymc) - Routine: sequential sender - stopped
>>>>>> 2021-09-20 12:07:26.753433: [NET] Device closed
>>>>>> 2021-09-20 12:07:26.754097: [NET] Routine: decryption worker 5 - stopped
>>>>>> 
>>>>>> Tested on devices: iOS 14.8, iPadOS 15
>>>>>> WireGuardKit: 79aeb0be0d0aa3f6c8bd24309aaa8dcf03216fb4
>>>>>> 
>>>>>> More info on includeAllNetworks option:
>>>>>> https://developer.apple.com/documentation/networkextension/nevpnprotocol/3131931-includeallnetworks
>>>>>> 
>>>>>> Can someone confirm this issue or point to a possible workaround?
>>>>>> Thanks!
>>>>>> 
>>>>>> Juraj H.
>>>>> 
>>>> 
>>> 
>> 
> 


From faustin at fala.red  Tue Oct 19 11:01:33 2021
From: faustin at fala.red (Faustin Lammler)
Date: Tue, 19 Oct 2021 13:01:33 +0200
Subject: Build wireguard-linux-compat on SLES15 (s390x)
In-Reply-To: <YWFuczOf1dM3B4Pt@falared>
References: <YWFuczOf1dM3B4Pt@falared>
Message-ID: <YW6ljZ3yvGsv9Njq@falared>

Hi!

FYI,
I have managed to build wireguard by removing the #define.
https://git.zx2c4.com/wireguard-linux-compat/tree/src/compat/compat.h#n859

I am not a C dev, so I am not sure what the implications are but I
understand that it's a kind of debugging structure and it's also not
defined on CENTOS8 so, I guess it should be OK.

Cheers!

--
Faustin
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: not available
URL: <http://lists.zx2c4.com/pipermail/wireguard/attachments/20211019/dfec154e/attachment.sig>

From juraj.hilje at gmail.com  Tue Oct 19 12:22:37 2021
From: juraj.hilje at gmail.com (Juraj Hilje)
Date: Tue, 19 Oct 2021 14:22:37 +0200
Subject: [wireguard-apple] [iOS] Changing network fails with
 includeAllNetworks (Kill Switch)
In-Reply-To: <66E63699-8CC0-4382-ADD3-1AC878898D8C@mullvad.net>
References: <B3A4A27C-1FBF-4098-B38F-9F87F8C115B5@gmail.com>
 <5569D8DF-A7E8-4A13-BA6B-913785819E4E@mullvad.net>
 <CB84BD03-DB8A-4855-94CC-898100C0B97E@gmail.com>
 <9D47F45A-59C7-4FB0-A9F8-2F13A1D4AB1A@mullvad.net>
 <E65FFE7D-9BCC-4EEF-9CF0-FA46CE6B9096@gmail.com>
 <85F39D56-E800-4EAE-B159-85335D150A4A@mullvad.net>
 <66E63699-8CC0-4382-ADD3-1AC878898D8C@mullvad.net>
Message-ID: <FD3103A9-03E3-43E8-A398-D589D923B806@gmail.com>

Thanks for the follow-up!
I had the same result on my end, everything worked as expected on the iOS 15.1 Beta.

> On 19.10.2021., at 11:54, Andrej Mihajlov <and at mullvad.net> wrote:
> 
> Follow up on this. It looks like the VPN connection breaks on my iPad running iPadOS 15 after changing DNS settings via WireGuardKit. 
> 
> Tested on iPadOS 15.1 beta today and it seems to be stable.
> 
>> On 28 Sep 2021, at 13:03, Andrej Mihajlov <and at mullvad.net> wrote:
>> 
>> Hi,
>> 
>> I can confirm that it behaves correctly on iOS 15 (tested on iPhone 12) and iOS 15.1 beta (tested on iPhone 7). Tested by toggling cellular/wi-fi and airplane mode on both devices and network monitor seems to be functioning properly. 
>> 
>> I haven?t tested this patch on iOS 14.8, but I had previously tested it on iOS 14.5 (IIRC) and it didn?t work there, that's why this patch is scoped to iOS 15+.
>> 
>> I am running the "am/enable-include-all-networks" branch which has the following changeset:
>> https://git.zx2c4.com/wireguard-apple/commit/?id=07bc66e7b181fb2068d457b31c1fdd05bdd2214a&id2=58e94f077329f6c7b96ec069243495d4e649fe36
>> 
>> Cheers,
>> Andrej
>> 
>>> On 22 Sep 2021, at 15:26, Juraj Hilje <juraj.hilje at gmail.com> wrote:
>>> 
>>> Hi Andrej,
>>> 
>>> I've tested on iOS/iPadOS 15.1 Beta, and it looks like the issue is fixed there.
>>> Let me know if you can confirm the same on your end.
>>> 
>>> Cheers,
>>> Juraj H.
>>> 
>>>> On 22.09.2021., at 10:59, Andrej Mihajlov <and at mullvad.net> wrote:
>>>> 
>>>> Hi Juraj,
>>>> 
>>>> Installing iOS 15 right now. I am gonna test it today too. 
>>>> 
>>>> What stands out to me that, while you have multiple interfaces available, the network monitor still says that the network is unsatisfied. Very odd.
>>>> 
>>>> Cheers,
>>>> Andrej
>>>> 
>>>>> On 22 Sep 2021, at 10:55, Juraj Hilje <juraj.hilje at gmail.com> wrote:
>>>>> 
>>>>> Hey Andrej, thanks for the response!
>>>>> 
>>>>> I've tested on iOS 14.8 and iOS 15.0 (public release), and even with the patch (b244febfdf3069dd4e8db2d31f0368d5474d7616) i still have the same issue on my end.
>>>>> 
>>>>> I will test the new iOS 15.1 Beta later today and let you know how it goes.
>>>>> 
>>>>> Juraj H.
>>>>> 
>>>>>> On 22.09.2021., at 10:08, Andrej Mihajlov <and at mullvad.net> wrote:
>>>>>> 
>>>>>> Have you tried on the most recent beta? I think it works over there, but requires some tweaks to the network monitor code in WireGuard. I had a patch somewhere here but haven?t spent much time testing it:
>>>>>> 
>>>>>> https://git.zx2c4.com/wireguard-apple/commit/?h=am/enable-include-all-networks&id=b244febfdf3069dd4e8db2d31f0368d5474d7616
>>>>>> 
>>>>>> Waiting for the final release of iOS 15.
>>>>>> 
>>>>>>> On 21 Sep 2021, at 12:55, Juraj Hilje <juraj.hilje at gmail.com> wrote:
>>>>>>> 
>>>>>>> If NETunnelProviderProtocol is configured with includeAllNetworks=true (Kill Switch), when network change is detected the device connectivity goes offline instead of routing VPN tunnel traffic through a new network.
>>>>>>> 
>>>>>>> Here are some logs from the moment of this event:
>>>>>>> 2021-09-20 12:07:26.735453: [NET] Network change detected with unsatisfied route and interface order [en0, utun4, pdp_ip0]
>>>>>>> 2021-09-20 12:07:26.736186: [NET] Connectivity offline, pausing backend.
>>>>>>> 2021-09-20 12:07:26.736732: [NET] Device closing
>>>>>>> 2021-09-20 12:07:26.737503: [NET] Routine: TUN reader - stopped
>>>>>>> 2021-09-20 12:07:26.738970: [NET] Routine: event worker - stopped
>>>>>>> 2021-09-20 12:07:26.739613: [NET] Routine: receive incoming v4 - stopped
>>>>>>> 2021-09-20 12:07:26.742070: [NET] Routine: receive incoming v6 - stopped
>>>>>>> 2021-09-20 12:07:26.746712: [NET] peer(eN1f?Oymc) - Stopping
>>>>>>> 2021-09-20 12:07:26.751550: [NET] peer(eN1f?Oymc) - Routine: sequential receiver - stopped
>>>>>>> 2021-09-20 12:07:26.751597: [NET] peer(eN1f?Oymc) - Routine: sequential sender - stopped
>>>>>>> 2021-09-20 12:07:26.753433: [NET] Device closed
>>>>>>> 2021-09-20 12:07:26.754097: [NET] Routine: decryption worker 5 - stopped
>>>>>>> 
>>>>>>> Tested on devices: iOS 14.8, iPadOS 15
>>>>>>> WireGuardKit: 79aeb0be0d0aa3f6c8bd24309aaa8dcf03216fb4
>>>>>>> 
>>>>>>> More info on includeAllNetworks option:
>>>>>>> https://developer.apple.com/documentation/networkextension/nevpnprotocol/3131931-includeallnetworks
>>>>>>> 
>>>>>>> Can someone confirm this issue or point to a possible workaround?
>>>>>>> Thanks!
>>>>>>> 
>>>>>>> Juraj H.
>>>>>> 
>>>>> 
>>>> 
>>> 
>> 
> 


From Jason at zx2c4.com  Tue Oct 19 23:51:01 2021
From: Jason at zx2c4.com (Jason A. Donenfeld)
Date: Tue, 19 Oct 2021 17:51:01 -0600
Subject: Build wireguard-linux-compat on SLES15 (s390x)
In-Reply-To: <YW6ljZ3yvGsv9Njq@falared>
References: <YWFuczOf1dM3B4Pt@falared> <YW6ljZ3yvGsv9Njq@falared>
Message-ID: <CAHmME9off=sXrL0hjkGvOgJUtJx6oCvhdAycHT=bTQfUtQjO=w@mail.gmail.com>

Hi Faustin,

Isn't WireGuard already in the SLES kernel?

Jason

From optionmaker at gmail.com  Tue Oct 19 12:18:40 2021
From: optionmaker at gmail.com (CW)
Date: Tue, 19 Oct 2021 13:18:40 +0100
Subject: Build wireguard-linux-compat on SLES15 (s390x)
In-Reply-To: <YW6ljZ3yvGsv9Njq@falared>
References: <YW6ljZ3yvGsv9Njq@falared>
Message-ID: <F0E5BD71-BEA5-4DB5-A8F0-C51EF048A2CD@gmail.com>

Hi,
If I didn?t remember wrong, wg should be included in kernel with 15sp3.


Sent from my iPhone


> On 19 Oct 2021, at 12:05, Faustin Lammler <faustin at fala.red> wrote:
> 
> ?Hi!
> 
> FYI,
> I have managed to build wireguard by removing the #define.
> https://git.zx2c4.com/wireguard-linux-compat/tree/src/compat/compat.h#n859
> 
> I am not a C dev, so I am not sure what the implications are but I
> understand that it's a kind of debugging structure and it's also not
> defined on CENTOS8 so, I guess it should be OK.
> 
> Cheers!
> 
> --
> Faustin

From mindaugas.veblauskas at protonmail.com  Tue Oct 19 12:54:53 2021
From: mindaugas.veblauskas at protonmail.com (Mindaugas Veblauskas)
Date: Tue, 19 Oct 2021 12:54:53 +0000
Subject: Access violation exception on windows
In-Reply-To: <PUlmptu_AouCBnZfF_b_qSr6iE0vwNlZcQ_cB0u6vFmJ4i5zhrYhKm6LCU5z7-0VU36m-8CsjgP2pkdpymQ9B0oPyZIkvVlb3_4tjYAzN5o=@protonmail.com>
References: <PUlmptu_AouCBnZfF_b_qSr6iE0vwNlZcQ_cB0u6vFmJ4i5zhrYhKm6LCU5z7-0VU36m-8CsjgP2pkdpymQ9B0oPyZIkvVlb3_4tjYAzN5o=@protonmail.com>
Message-ID: <XcnYGCQFb8CjcIVfX4jdbb9vwXO0M2pth9Laor075LAPBtKtgQnb-D-NbmvnhEZL-BJfsTIR8V78CAVgJkcy7iN3WUecL78NssgR53axTEo=@protonmail.com>

Hi,

whenever WireGuardTunnel$xxxx service stops, it exits with the following exception:

Exception thrown at 0x0000000000E12A88 in wireguard.exe: 0xC0000005: Access violation reading location 0x0000000000000000.

The behavior can be reproduced by attaching visual studio debugger on the service process and then disconnecting/deactivating tunnel on the client.

I wonder where this exception is coming from and what's the impact of it to the system.

From Jason at zx2c4.com  Wed Oct 20 06:47:18 2021
From: Jason at zx2c4.com (Jason A. Donenfeld)
Date: Wed, 20 Oct 2021 00:47:18 -0600
Subject: Access violation exception on windows
In-Reply-To: <XcnYGCQFb8CjcIVfX4jdbb9vwXO0M2pth9Laor075LAPBtKtgQnb-D-NbmvnhEZL-BJfsTIR8V78CAVgJkcy7iN3WUecL78NssgR53axTEo=@protonmail.com>
References: <PUlmptu_AouCBnZfF_b_qSr6iE0vwNlZcQ_cB0u6vFmJ4i5zhrYhKm6LCU5z7-0VU36m-8CsjgP2pkdpymQ9B0oPyZIkvVlb3_4tjYAzN5o=@protonmail.com>
 <XcnYGCQFb8CjcIVfX4jdbb9vwXO0M2pth9Laor075LAPBtKtgQnb-D-NbmvnhEZL-BJfsTIR8V78CAVgJkcy7iN3WUecL78NssgR53axTEo=@protonmail.com>
Message-ID: <CAHmME9qXEj8zeSrv5nww4eavuCpRHoMqF-dm8i9Z-1aR3WwFSw@mail.gmail.com>

Thanks for the reminder.

Fixed here: https://github.com/golang/sys/commit/0ec99a608a1b8e24d6a2554b64f4208e8e5caae7
With a followup here waiting to be merged:
https://go-review.googlesource.com/c/sys/+/357250

The process now continues until NtTerminateProcess.

Jason

From faustin at fala.red  Wed Oct 20 09:00:48 2021
From: faustin at fala.red (Faustin Lammler)
Date: Wed, 20 Oct 2021 11:00:48 +0200
Subject: Build wireguard-linux-compat on SLES15 (s390x)
In-Reply-To: <CAHmME9off=sXrL0hjkGvOgJUtJx6oCvhdAycHT=bTQfUtQjO=w@mail.gmail.com>
References: <YWFuczOf1dM3B4Pt@falared> <YW6ljZ3yvGsv9Njq@falared>
 <CAHmME9off=sXrL0hjkGvOgJUtJx6oCvhdAycHT=bTQfUtQjO=w@mail.gmail.com>
Message-ID: <YW/awIN8jqUF0DPk@falared>

Hi Jason!

"Jason A. Donenfeld" <Jason at zx2c4.com>,
19/10/2021 ? 17:51:01 (-0600):

> Isn't WireGuard already in the SLES kernel?

My understanding is that it should, that's why I was surprised to have
to build it. But apparently not.

This may be related to a specific kernel from IBM LinuxONE cloud
infrastructure?

| $ uname -r
| 5.3.18-24.86-default

But again, my knowledge of SLES is very limited (and s390x architecture
even more) so I don't have any idea why this was needed.

I'll be happy to help you if you need me to do some tests on the
machine.

Cheers!

-- 
Faustin
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: not available
URL: <http://lists.zx2c4.com/pipermail/wireguard/attachments/20211020/2e24e61b/attachment.sig>

From wireguard at smar.fi  Wed Oct 20 16:25:49 2021
From: wireguard at smar.fi (Samu Voutilainen)
Date: Wed, 20 Oct 2021 19:25:49 +0300
Subject: Build wireguard-linux-compat on SLES15 (s390x)
In-Reply-To: <YW/awIN8jqUF0DPk@falared>
References: <YWFuczOf1dM3B4Pt@falared>
 <CAHmME9off=sXrL0hjkGvOgJUtJx6oCvhdAycHT=bTQfUtQjO=w@mail.gmail.com>
 <YW/awIN8jqUF0DPk@falared>
Message-ID: <2505087.Lt9SDvczpP@marie>

Faustin Lammler kirjoitti keskiviikkona 20. lokakuuta 2021 12.00.48 EEST:
> Hi Jason!
> 
> My understanding is that it should, that's why I was surprised to have
> to build it. But apparently not.
> 
> This may be related to a specific kernel from IBM LinuxONE cloud
> infrastructure?
> 

Hi,

At least on x86_64 SLE, the module is in kernel-*-extra package, for example 
in kernel-default-extra.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 488 bytes
Desc: This is a digitally signed message part.
URL: <http://lists.zx2c4.com/pipermail/wireguard/attachments/20211020/fcc909ea/attachment.sig>

From Jason at zx2c4.com  Wed Oct 20 17:56:49 2021
From: Jason at zx2c4.com (Jason A. Donenfeld)
Date: Wed, 20 Oct 2021 11:56:49 -0600
Subject: Build wireguard-linux-compat on SLES15 (s390x)
In-Reply-To: <YW/awIN8jqUF0DPk@falared>
References: <YWFuczOf1dM3B4Pt@falared> <YW6ljZ3yvGsv9Njq@falared>
 <CAHmME9off=sXrL0hjkGvOgJUtJx6oCvhdAycHT=bTQfUtQjO=w@mail.gmail.com>
 <YW/awIN8jqUF0DPk@falared>
Message-ID: <CAHmME9oL7UNWKbX3ZF-U_O+a9Kwn9gwsexzDQo9gngjhEdgc8Q@mail.gmail.com>

Hi Daniel,

Do you think you could chime in here regarding the apparent absence of
wireguard on s390x? The thread began as a discussion of building it
yourself there, but I'm wondering why it isn't already there, seeing
as it's on the other archs.

Jason

On Wed, Oct 20, 2021 at 3:01 AM Faustin Lammler <faustin at fala.red> wrote:
>
> Hi Jason!
>
> "Jason A. Donenfeld" <Jason at zx2c4.com>,
> 19/10/2021 ? 17:51:01 (-0600):
>
> > Isn't WireGuard already in the SLES kernel?
>
> My understanding is that it should, that's why I was surprised to have
> to build it. But apparently not.
>
> This may be related to a specific kernel from IBM LinuxONE cloud
> infrastructure?
>
> | $ uname -r
> | 5.3.18-24.86-default
>
> But again, my knowledge of SLES is very limited (and s390x architecture
> even more) so I don't have any idea why this was needed.
>
> I'll be happy to help you if you need me to do some tests on the
> machine.
>
> Cheers!
>
> --
> Faustin

From faustin at fala.red  Thu Oct 21 07:44:28 2021
From: faustin at fala.red (Faustin Lammler)
Date: Thu, 21 Oct 2021 09:44:28 +0200
Subject: Build wireguard-linux-compat on SLES15 (s390x)
In-Reply-To: <2505087.Lt9SDvczpP@marie>
References: <YWFuczOf1dM3B4Pt@falared>
 <CAHmME9off=sXrL0hjkGvOgJUtJx6oCvhdAycHT=bTQfUtQjO=w@mail.gmail.com>
 <YW/awIN8jqUF0DPk@falared> <2505087.Lt9SDvczpP@marie>
Message-ID: <YXEaXIkYfcSPO+tR@falared>

Samu Voutilainen <wireguard at smar.fi>,
20/10/2021 ? 19:25:49 (+0300):

> At least on x86_64 SLE, the module is in kernel-*-extra package, for example 
> in kernel-default-extra.

Zypper does show kernel-default-extra but:
| kernel-default-base | The Standard Kernel - base modules | package

That is not installed, maybe this is where the wireguard module is?

-- 
Faustin
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: not available
URL: <http://lists.zx2c4.com/pipermail/wireguard/attachments/20211021/786d4449/attachment.sig>

From wireguard at smar.fi  Thu Oct 21 08:07:44 2021
From: wireguard at smar.fi (Samu Voutilainen)
Date: Thu, 21 Oct 2021 11:07:44 +0300
Subject: Build wireguard-linux-compat on SLES15 (s390x)
In-Reply-To: <YXEaXIkYfcSPO+tR@falared>
References: <YWFuczOf1dM3B4Pt@falared> <2505087.Lt9SDvczpP@marie>
 <YXEaXIkYfcSPO+tR@falared>
Message-ID: <4688156.31r3eYUQgx@marie>

Faustin Lammler kirjoitti torstaina 21. lokakuuta 2021 10.44.28 EEST:
> Samu Voutilainen <wireguard at smar.fi>,
> 
> 20/10/2021 ? 19:25:49 (+0300):
> > At least on x86_64 SLE, the module is in kernel-*-extra package, for
> > example in kernel-default-extra.
> 
> Zypper does show kernel-default-extra but:
> | kernel-default-base | The Standard Kernel - base modules | package
> 
> That is not installed, maybe this is where the wireguard module is?

kernel-default-base is kind of minimal version of default kernel, usable in 
certain virtualization setups, for example. 

Actually it looks like in SLE-15-SP3 wireguard.ko is in kernel-default rather 
in -extra. That was case in SP2.

$ rpm -ql kernel-default | ag wireguard.ko
/lib/modules/5.3.18-59.27-default/kernel/drivers/net/wireguard/wireguard.ko.xz

If the module is not present, maybe it is not compiled for s390x. Maybe SUSE?s 
support could enlighten why such choice has been made?
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 488 bytes
Desc: This is a digitally signed message part.
URL: <http://lists.zx2c4.com/pipermail/wireguard/attachments/20211021/2fab3db7/attachment.sig>

From faustin at fala.red  Thu Oct 21 08:52:36 2021
From: faustin at fala.red (Faustin Lammler)
Date: Thu, 21 Oct 2021 10:52:36 +0200
Subject: Build wireguard-linux-compat on SLES15 (s390x)
In-Reply-To: <4688156.31r3eYUQgx@marie>
References: <YWFuczOf1dM3B4Pt@falared> <2505087.Lt9SDvczpP@marie>
 <YXEaXIkYfcSPO+tR@falared> <4688156.31r3eYUQgx@marie>
Message-ID: <YXEqVJkZ3rJiWHZU@falared>

Hi Samu!

Samu Voutilainen <wireguard at smar.fi>,
21/10/2021 ? 11:07:44 (+0300):

> kernel-default-base is kind of minimal version of default kernel, usable in 
> certain virtualization setups, for example. 
> 
> Actually it looks like in SLE-15-SP3 wireguard.ko is in kernel-default rather 
> in -extra. That was case in SP2.
> 
> $ rpm -ql kernel-default | ag wireguard.ko
> /lib/modules/5.3.18-59.27-default/kernel/drivers/net/wireguard/wireguard.ko.xz
Ok, thanks you.

> If the module is not present, maybe it is not compiled for s390x. Maybe SUSE?s 
> support could enlighten why such choice has been made?
I believe that Jason already asked Daniel to check for it.

-- 
Faustin
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: not available
URL: <http://lists.zx2c4.com/pipermail/wireguard/attachments/20211021/a355a420/attachment.sig>

From syzbot+fa3b49ed40f26375a8ee at syzkaller.appspotmail.com  Thu Oct 21 19:39:32 2021
From: syzbot+fa3b49ed40f26375a8ee at syzkaller.appspotmail.com (syzbot)
Date: Thu, 21 Oct 2021 12:39:32 -0700
Subject: [syzbot] INFO: task hung in wg_netns_pre_exit (2)
Message-ID: <000000000000c3110805cee20d4e@google.com>

Hello,

syzbot found the following issue on:

HEAD commit:    02d5e016800d Merge tag 'sound-5.15-rc4' of git://git.kerne..
git tree:       upstream
console output: https://syzkaller.appspot.com/x/log.txt?x=16e21b17300000
kernel config:  https://syzkaller.appspot.com/x/.config?x=c76f0f4ac6e9f8d2
dashboard link: https://syzkaller.appspot.com/bug?extid=fa3b49ed40f26375a8ee
compiler:       gcc (Debian 10.2.1-6) 10.2.1 20210110, GNU ld (GNU Binutils for Debian) 2.35.2

Unfortunately, I don't have any reproducer for this issue yet.

IMPORTANT: if you fix the issue, please add the following tag to the commit:
Reported-by: syzbot+fa3b49ed40f26375a8ee at syzkaller.appspotmail.com

INFO: task kworker/u4:3:254 blocked for more than 143 seconds.
      Not tainted 5.15.0-rc3-syzkaller #0
"echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
task:kworker/u4:3    state:D stack:24064 pid:  254 ppid:     2 flags:0x00004000
Workqueue: netns cleanup_net
Call Trace:
 context_switch kernel/sched/core.c:4940 [inline]
 __schedule+0x940/0x26f0 kernel/sched/core.c:6287
 schedule+0xd3/0x270 kernel/sched/core.c:6366
 schedule_preempt_disabled+0xf/0x20 kernel/sched/core.c:6425
 __mutex_lock_common kernel/locking/mutex.c:669 [inline]
 __mutex_lock+0xa34/0x12f0 kernel/locking/mutex.c:729
 wg_netns_pre_exit+0x15/0x190 drivers/net/wireguard/device.c:402
 ops_pre_exit_list net/core/net_namespace.c:158 [inline]
 cleanup_net+0x451/0xb00 net/core/net_namespace.c:579
 process_one_work+0x9bf/0x16b0 kernel/workqueue.c:2297
 worker_thread+0x658/0x11f0 kernel/workqueue.c:2444
 kthread+0x3e5/0x4d0 kernel/kthread.c:319
 ret_from_fork+0x1f/0x30 arch/x86/entry/entry_64.S:295
INFO: task syz-executor.5:13115 blocked for more than 143 seconds.
      Not tainted 5.15.0-rc3-syzkaller #0
"echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
task:syz-executor.5  state:D stack:28440 pid:13115 ppid:  6871 flags:0x00004004
Call Trace:
 context_switch kernel/sched/core.c:4940 [inline]
 __schedule+0x940/0x26f0 kernel/sched/core.c:6287
 schedule+0xd3/0x270 kernel/sched/core.c:6366
 schedule_preempt_disabled+0xf/0x20 kernel/sched/core.c:6425
 __mutex_lock_common kernel/locking/mutex.c:669 [inline]
 __mutex_lock+0xa34/0x12f0 kernel/locking/mutex.c:729
 smc_pnet_create_pnetids_list net/smc/smc_pnet.c:798 [inline]
 smc_pnet_net_init+0x1f9/0x410 net/smc/smc_pnet.c:867
 ops_init+0xaf/0x470 net/core/net_namespace.c:140
 setup_net+0x40f/0xa30 net/core/net_namespace.c:326
 copy_net_ns+0x319/0x760 net/core/net_namespace.c:470
 create_new_namespaces+0x3f6/0xb20 kernel/nsproxy.c:110
 unshare_nsproxy_namespaces+0xc1/0x1f0 kernel/nsproxy.c:226
 ksys_unshare+0x445/0x920 kernel/fork.c:3077
 __do_sys_unshare kernel/fork.c:3151 [inline]
 __se_sys_unshare kernel/fork.c:3149 [inline]
 __x64_sys_unshare+0x2d/0x40 kernel/fork.c:3149
 do_syscall_x64 arch/x86/entry/common.c:50 [inline]
 do_syscall_64+0x35/0xb0 arch/x86/entry/common.c:80
 entry_SYSCALL_64_after_hwframe+0x44/0xae
RIP: 0033:0x7f2348aa4709
RSP: 002b:00007f234601b188 EFLAGS: 00000246 ORIG_RAX: 0000000000000110
RAX: ffffffffffffffda RBX: 00007f2348ba8f60 RCX: 00007f2348aa4709
RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000048040200
RBP: 00007f2348afecb4 R08: 0000000000000000 R09: 0000000000000000
R10: 0000000000000000 R11: 0000000000000246 R12: 0000000000000000
R13: 00007ffe1dab5b5f R14: 00007f234601b300 R15: 0000000000022000
INFO: task syz-executor.5:13120 blocked for more than 144 seconds.
      Not tainted 5.15.0-rc3-syzkaller #0
"echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
task:syz-executor.5  state:D stack:26688 pid:13120 ppid:  6871 flags:0x00004004
Call Trace:
 context_switch kernel/sched/core.c:4940 [inline]
 __schedule+0x940/0x26f0 kernel/sched/core.c:6287
 schedule+0xd3/0x270 kernel/sched/core.c:6366
 schedule_preempt_disabled+0xf/0x20 kernel/sched/core.c:6425
 __mutex_lock_common kernel/locking/mutex.c:669 [inline]
 __mutex_lock+0xa34/0x12f0 kernel/locking/mutex.c:729
 rtnl_lock net/core/rtnetlink.c:72 [inline]
 rtnetlink_rcv_msg+0x3be/0xb80 net/core/rtnetlink.c:5569
 netlink_rcv_skb+0x153/0x420 net/netlink/af_netlink.c:2504
 netlink_unicast_kernel net/netlink/af_netlink.c:1314 [inline]
 netlink_unicast+0x533/0x7d0 net/netlink/af_netlink.c:1340
 netlink_sendmsg+0x86d/0xdb0 net/netlink/af_netlink.c:1929
 sock_sendmsg_nosec net/socket.c:704 [inline]
 sock_sendmsg+0xcf/0x120 net/socket.c:724
 ____sys_sendmsg+0x6e8/0x810 net/socket.c:2409
 ___sys_sendmsg+0xf3/0x170 net/socket.c:2463
 __sys_sendmsg+0xe5/0x1b0 net/socket.c:2492
 do_syscall_x64 arch/x86/entry/common.c:50 [inline]
 do_syscall_64+0x35/0xb0 arch/x86/entry/common.c:80
 entry_SYSCALL_64_after_hwframe+0x44/0xae
RIP: 0033:0x7f2348aa4709
RSP: 002b:00007f2345ffa188 EFLAGS: 00000246 ORIG_RAX: 000000000000002e
RAX: ffffffffffffffda RBX: 00007f2348ba9020 RCX: 00007f2348aa4709
RDX: 0000000000000000 RSI: 0000000020000100 RDI: 0000000000000003
RBP: 00007f2348afecb4 R08: 0000000000000000 R09: 0000000000000000
R10: 0000000000000000 R11: 0000000000000246 R12: 0000000000000000
R13: 00007ffe1dab5b5f R14: 00007f2345ffa300 R15: 0000000000022000
INFO: task syz-executor.5:13156 blocked for more than 144 seconds.
      Not tainted 5.15.0-rc3-syzkaller #0
"echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
task:syz-executor.5  state:D stack:28696 pid:13156 ppid:  6871 flags:0x00004004
Call Trace:
 context_switch kernel/sched/core.c:4940 [inline]
 __schedule+0x940/0x26f0 kernel/sched/core.c:6287
 schedule+0xd3/0x270 kernel/sched/core.c:6366
 schedule_preempt_disabled+0xf/0x20 kernel/sched/core.c:6425
 __mutex_lock_common kernel/locking/mutex.c:669 [inline]
 __mutex_lock+0xa34/0x12f0 kernel/locking/mutex.c:729
 smc_pnet_create_pnetids_list net/smc/smc_pnet.c:798 [inline]
 smc_pnet_net_init+0x1f9/0x410 net/smc/smc_pnet.c:867
 ops_init+0xaf/0x470 net/core/net_namespace.c:140
 setup_net+0x40f/0xa30 net/core/net_namespace.c:326
 copy_net_ns+0x319/0x760 net/core/net_namespace.c:470
 create_new_namespaces+0x3f6/0xb20 kernel/nsproxy.c:110
 unshare_nsproxy_namespaces+0xc1/0x1f0 kernel/nsproxy.c:226
 ksys_unshare+0x445/0x920 kernel/fork.c:3077
 __do_sys_unshare kernel/fork.c:3151 [inline]
 __se_sys_unshare kernel/fork.c:3149 [inline]
 __x64_sys_unshare+0x2d/0x40 kernel/fork.c:3149
 do_syscall_x64 arch/x86/entry/common.c:50 [inline]
 do_syscall_64+0x35/0xb0 arch/x86/entry/common.c:80
 entry_SYSCALL_64_after_hwframe+0x44/0xae
RIP: 0033:0x7f2348aa4709
RSP: 002b:00007f2345fd9188 EFLAGS: 00000246 ORIG_RAX: 0000000000000110
RAX: ffffffffffffffda RBX: 00007f2348ba90e0 RCX: 00007f2348aa4709
RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000048040200
RBP: 00007f2348afecb4 R08: 0000000000000000 R09: 0000000000000000
R10: 0000000000000000 R11: 0000000000000246 R12: 0000000000000000
R13: 00007ffe1dab5b5f R14: 00007f2345fd9300 R15: 0000000000022000
INFO: task syz-executor.5:13162 blocked for more than 145 seconds.
      Not tainted 5.15.0-rc3-syzkaller #0
"echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
task:syz-executor.5  state:D stack:28024 pid:13162 ppid:  6871 flags:0x00000004
Call Trace:
 context_switch kernel/sched/core.c:4940 [inline]
 __schedule+0x940/0x26f0 kernel/sched/core.c:6287
 schedule+0xd3/0x270 kernel/sched/core.c:6366
 schedule_preempt_disabled+0xf/0x20 kernel/sched/core.c:6425
 __mutex_lock_common kernel/locking/mutex.c:669 [inline]
 __mutex_lock+0xa34/0x12f0 kernel/locking/mutex.c:729
 rtnl_lock net/core/rtnetlink.c:72 [inline]
 rtnetlink_rcv_msg+0x3be/0xb80 net/core/rtnetlink.c:5569
 netlink_rcv_skb+0x153/0x420 net/netlink/af_netlink.c:2504
 netlink_unicast_kernel net/netlink/af_netlink.c:1314 [inline]
 netlink_unicast+0x533/0x7d0 net/netlink/af_netlink.c:1340
 netlink_sendmsg+0x86d/0xdb0 net/netlink/af_netlink.c:1929
 sock_sendmsg_nosec net/socket.c:704 [inline]
 sock_sendmsg+0xcf/0x120 net/socket.c:724
 ____sys_sendmsg+0x6e8/0x810 net/socket.c:2409
 ___sys_sendmsg+0xf3/0x170 net/socket.c:2463
 __sys_sendmsg+0xe5/0x1b0 net/socket.c:2492
 do_syscall_x64 arch/x86/entry/common.c:50 [inline]
 do_syscall_64+0x35/0xb0 arch/x86/entry/common.c:80
 entry_SYSCALL_64_after_hwframe+0x44/0xae
RIP: 0033:0x7f2348aa4709
RSP: 002b:00007f2345f97188 EFLAGS: 00000246 ORIG_RAX: 000000000000002e
RAX: ffffffffffffffda RBX: 00007f2348ba9260 RCX: 00007f2348aa4709
RDX: 0000000000000000 RSI: 0000000020000100 RDI: 0000000000000003
RBP: 00007f2348afecb4 R08: 0000000000000000 R09: 0000000000000000
R10: 0000000000000000 R11: 0000000000000246 R12: 0000000000000000
R13: 00007ffe1dab5b5f R14: 00007f2345f97300 R15: 0000000000022000
INFO: task syz-executor.2:13147 blocked for more than 145 seconds.
      Not tainted 5.15.0-rc3-syzkaller #0
"echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
task:syz-executor.2  state:D stack:28072 pid:13147 ppid:  6566 flags:0x00000004
Call Trace:
 context_switch kernel/sched/core.c:4940 [inline]
 __schedule+0x940/0x26f0 kernel/sched/core.c:6287
 schedule+0xd3/0x270 kernel/sched/core.c:6366
 schedule_preempt_disabled+0xf/0x20 kernel/sched/core.c:6425
 __mutex_lock_common kernel/locking/mutex.c:669 [inline]
 __mutex_lock+0xa34/0x12f0 kernel/locking/mutex.c:729
 rtnl_lock net/core/rtnetlink.c:72 [inline]
 rtnetlink_rcv_msg+0x3be/0xb80 net/core/rtnetlink.c:5569
 netlink_rcv_skb+0x153/0x420 net/netlink/af_netlink.c:2504
 netlink_unicast_kernel net/netlink/af_netlink.c:1314 [inline]
 netlink_unicast+0x533/0x7d0 net/netlink/af_netlink.c:1340
 netlink_sendmsg+0x86d/0xdb0 net/netlink/af_netlink.c:1929
 sock_sendmsg_nosec net/socket.c:704 [inline]
 sock_sendmsg+0xcf/0x120 net/socket.c:724
 ____sys_sendmsg+0x6e8/0x810 net/socket.c:2409
 ___sys_sendmsg+0xf3/0x170 net/socket.c:2463
 __sys_sendmsg+0xe5/0x1b0 net/socket.c:2492
 do_syscall_x64 arch/x86/entry/common.c:50 [inline]
 do_syscall_64+0x35/0xb0 arch/x86/entry/common.c:80
 entry_SYSCALL_64_after_hwframe+0x44/0xae
RIP: 0033:0x7fccb83ee709
RSP: 002b:00007fccb5965188 EFLAGS: 00000246 ORIG_RAX: 000000000000002e
RAX: ffffffffffffffda RBX: 00007fccb84f2f60 RCX: 00007fccb83ee709
RDX: 0000000000000000 RSI: 0000000020000040 RDI: 0000000000000006
RBP: 00007fccb8448cb4 R08: 0000000000000000 R09: 0000000000000000
R10: 0000000000000000 R11: 0000000000000246 R12: 0000000000000000
R13: 00007ffd586b757f R14: 00007fccb5965300 R15: 0000000000022000
INFO: task syz-executor.2:13150 blocked for more than 145 seconds.
      Not tainted 5.15.0-rc3-syzkaller #0
"echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
task:syz-executor.2  state:D stack:28656 pid:13150 ppid:  6566 flags:0x00000004
Call Trace:
 context_switch kernel/sched/core.c:4940 [inline]
 __schedule+0x940/0x26f0 kernel/sched/core.c:6287
 schedule+0xd3/0x270 kernel/sched/core.c:6366
 schedule_preempt_disabled+0xf/0x20 kernel/sched/core.c:6425
 __mutex_lock_common kernel/locking/mutex.c:669 [inline]
 __mutex_lock+0xa34/0x12f0 kernel/locking/mutex.c:729
 rtnl_lock net/core/rtnetlink.c:72 [inline]
 rtnetlink_rcv_msg+0x3be/0xb80 net/core/rtnetlink.c:5569
 netlink_rcv_skb+0x153/0x420 net/netlink/af_netlink.c:2504
 netlink_unicast_kernel net/netlink/af_netlink.c:1314 [inline]
 netlink_unicast+0x533/0x7d0 net/netlink/af_netlink.c:1340
 netlink_sendmsg+0x86d/0xdb0 net/netlink/af_netlink.c:1929
 sock_sendmsg_nosec net/socket.c:704 [inline]
 sock_sendmsg+0xcf/0x120 net/socket.c:724
 ____sys_sendmsg+0x6e8/0x810 net/socket.c:2409
 ___sys_sendmsg+0xf3/0x170 net/socket.c:2463
 __sys_sendmsg+0xe5/0x1b0 net/socket.c:2492
 do_syscall_x64 arch/x86/entry/common.c:50 [inline]
 do_syscall_64+0x35/0xb0 arch/x86/entry/common.c:80
 entry_SYSCALL_64_after_hwframe+0x44/0xae
RIP: 0033:0x7fccb83ee709
RSP: 002b:00007fccb5944188 EFLAGS: 00000246 ORIG_RAX: 000000000000002e
RAX: ffffffffffffffda RBX: 00007fccb84f3020 RCX: 00007fccb83ee709
RDX: 0000000000000000 RSI: 0000000020000180 RDI: 0000000000000005
RBP: 00007fccb8448cb4 R08: 0000000000000000 R09: 0000000000000000
R10: 0000000000000000 R11: 0000000000000246 R12: 0000000000000000
R13: 00007ffd586b757f R14: 00007fccb5944300 R15: 0000000000022000
INFO: task syz-executor.2:13153 blocked for more than 146 seconds.
      Not tainted 5.15.0-rc3-syzkaller #0
"echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
task:syz-executor.2  state:D stack:28656 pid:13153 ppid:  6566 flags:0x00004004
Call Trace:
 context_switch kernel/sched/core.c:4940 [inline]
 __schedule+0x940/0x26f0 kernel/sched/core.c:6287
 schedule+0xd3/0x270 kernel/sched/core.c:6366
 schedule_preempt_disabled+0xf/0x20 kernel/sched/core.c:6425
 __mutex_lock_common kernel/locking/mutex.c:669 [inline]
 __mutex_lock+0xa34/0x12f0 kernel/locking/mutex.c:729
 rtnl_lock net/core/rtnetlink.c:72 [inline]
 rtnetlink_rcv_msg+0x3be/0xb80 net/core/rtnetlink.c:5569
 netlink_rcv_skb+0x153/0x420 net/netlink/af_netlink.c:2504
 netlink_unicast_kernel net/netlink/af_netlink.c:1314 [inline]
 netlink_unicast+0x533/0x7d0 net/netlink/af_netlink.c:1340
 netlink_sendmsg+0x86d/0xdb0 net/netlink/af_netlink.c:1929
 sock_sendmsg_nosec net/socket.c:704 [inline]
 sock_sendmsg+0xcf/0x120 net/socket.c:724
 ____sys_sendmsg+0x6e8/0x810 net/socket.c:2409
 ___sys_sendmsg+0xf3/0x170 net/socket.c:2463
 __sys_sendmsg+0xe5/0x1b0 net/socket.c:2492
 do_syscall_x64 arch/x86/entry/common.c:50 [inline]
 do_syscall_64+0x35/0xb0 arch/x86/entry/common.c:80
 entry_SYSCALL_64_after_hwframe+0x44/0xae
RIP: 0033:0x7fccb83ee709
RSP: 002b:00007fccb5923188 EFLAGS: 00000246 ORIG_RAX: 000000000000002e
RAX: ffffffffffffffda RBX: 00007fccb84f30e0 RCX: 00007fccb83ee709
RDX: 0000000000000000 RSI: 0000000020000080 RDI: 0000000000000004
RBP: 00007fccb8448cb4 R08: 0000000000000000 R09: 0000000000000000
R10: 0000000000000000 R11: 0000000000000246 R12: 0000000000000000
R13: 00007ffd586b757f R14: 00007fccb5923300 R15: 0000000000022000
INFO: task syz-executor.2:13167 blocked for more than 146 seconds.
      Not tainted 5.15.0-rc3-syzkaller #0
"echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
task:syz-executor.2  state:D stack:28656 pid:13167 ppid:  6566 flags:0x00000004
Call Trace:
 context_switch kernel/sched/core.c:4940 [inline]
 __schedule+0x940/0x26f0 kernel/sched/core.c:6287
 schedule+0xd3/0x270 kernel/sched/core.c:6366
 schedule_preempt_disabled+0xf/0x20 kernel/sched/core.c:6425
 __mutex_lock_common kernel/locking/mutex.c:669 [inline]
 __mutex_lock+0xa34/0x12f0 kernel/locking/mutex.c:729
 rtnl_lock net/core/rtnetlink.c:72 [inline]
 rtnetlink_rcv_msg+0x3be/0xb80 net/core/rtnetlink.c:5569
 netlink_rcv_skb+0x153/0x420 net/netlink/af_netlink.c:2504
 netlink_unicast_kernel net/netlink/af_netlink.c:1314 [inline]
 netlink_unicast+0x533/0x7d0 net/netlink/af_netlink.c:1340
 netlink_sendmsg+0x86d/0xdb0 net/netlink/af_netlink.c:1929
 sock_sendmsg_nosec net/socket.c:704 [inline]
 sock_sendmsg+0xcf/0x120 net/socket.c:724
 ____sys_sendmsg+0x6e8/0x810 net/socket.c:2409
 ___sys_sendmsg+0xf3/0x170 net/socket.c:2463
 __sys_sendmsg+0xe5/0x1b0 net/socket.c:2492
 do_syscall_x64 arch/x86/entry/common.c:50 [inline]
 do_syscall_64+0x35/0xb0 arch/x86/entry/common.c:80
 entry_SYSCALL_64_after_hwframe+0x44/0xae
RIP: 0033:0x7fccb83ee709
RSP: 002b:00007fccb58e1188 EFLAGS: 00000246 ORIG_RAX: 000000000000002e
RAX: ffffffffffffffda RBX: 00007fccb84f3260 RCX: 00007fccb83ee709
RDX: 0000000000000000 RSI: 0000000020000040 RDI: 0000000000000006
RBP: 00007fccb8448cb4 R08: 0000000000000000 R09: 0000000000000000
R10: 0000000000000000 R11: 0000000000000246 R12: 0000000000000000
R13: 00007ffd586b757f R14: 00007fccb58e1300 R15: 0000000000022000
INFO: task syz-executor.2:13168 blocked for more than 147 seconds.
      Not tainted 5.15.0-rc3-syzkaller #0
"echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
task:syz-executor.2  state:D stack:28656 pid:13168 ppid:  6566 flags:0x00004004
Call Trace:
 context_switch kernel/sched/core.c:4940 [inline]
 __schedule+0x940/0x26f0 kernel/sched/core.c:6287
 schedule+0xd3/0x270 kernel/sched/core.c:6366
 schedule_preempt_disabled+0xf/0x20 kernel/sched/core.c:6425
 __mutex_lock_common kernel/locking/mutex.c:669 [inline]
 __mutex_lock+0xa34/0x12f0 kernel/locking/mutex.c:729
 rtnl_lock net/core/rtnetlink.c:72 [inline]
 rtnetlink_rcv_msg+0x3be/0xb80 net/core/rtnetlink.c:5569
 netlink_rcv_skb+0x153/0x420 net/netlink/af_netlink.c:2504
 netlink_unicast_kernel net/netlink/af_netlink.c:1314 [inline]
 netlink_unicast+0x533/0x7d0 net/netlink/af_netlink.c:1340
 netlink_sendmsg+0x86d/0xdb0 net/netlink/af_netlink.c:1929
 sock_sendmsg_nosec net/socket.c:704 [inline]
 sock_sendmsg+0xcf/0x120 net/socket.c:724
 ____sys_sendmsg+0x6e8/0x810 net/socket.c:2409
 ___sys_sendmsg+0xf3/0x170 net/socket.c:2463
 __sys_sendmsg+0xe5/0x1b0 net/socket.c:2492
 do_syscall_x64 arch/x86/entry/common.c:50 [inline]
 do_syscall_64+0x35/0xb0 arch/x86/entry/common.c:80
 entry_SYSCALL_64_after_hwframe+0x44/0xae
RIP: 0033:0x7fccb83ee709
RSP: 002b:00007fccb58c0188 EFLAGS: 00000246 ORIG_RAX: 000000000000002e
RAX: ffffffffffffffda RBX: 00007fccb84f3320 RCX: 00007fccb83ee709
RDX: 0000000000000000 RSI: 0000000020000180 RDI: 0000000000000005
RBP: 00007fccb8448cb4 R08: 0000000000000000 R09: 0000000000000000
R10: 0000000000000000 R11: 0000000000000246 R12: 0000000000000000
R13: 00007ffd586b757f R14: 00007fccb58c0300 R15: 0000000000022000

Showing all locks held in the system:
1 lock held by khungtaskd/27:
 #0: ffffffff8b97d420 (rcu_read_lock){....}-{1:2}, at: debug_show_all_locks+0x53/0x260 kernel/locking/lockdep.c:6446
4 locks held by kworker/u4:3/254:
 #0: ffff888140275938 ((wq_completion)netns){+.+.}-{0:0}, at: arch_atomic64_set arch/x86/include/asm/atomic64_64.h:34 [inline]
 #0: ffff888140275938 ((wq_completion)netns){+.+.}-{0:0}, at: arch_atomic_long_set include/linux/atomic/atomic-long.h:41 [inline]
 #0: ffff888140275938 ((wq_completion)netns){+.+.}-{0:0}, at: atomic_long_set include/linux/atomic/atomic-instrumented.h:1198 [inline]
 #0: ffff888140275938 ((wq_completion)netns){+.+.}-{0:0}, at: set_work_data kernel/workqueue.c:634 [inline]
 #0: ffff888140275938 ((wq_completion)netns){+.+.}-{0:0}, at: set_work_pool_and_clear_pending kernel/workqueue.c:661 [inline]
 #0: ffff888140275938 ((wq_completion)netns){+.+.}-{0:0}, at: process_one_work+0x8a3/0x16b0 kernel/workqueue.c:2268
 #1: ffffc90002107db0 (net_cleanup_work){+.+.}-{0:0}, at: process_one_work+0x8d7/0x16b0 kernel/workqueue.c:2272
 #2: ffffffff8d0cef50 (pernet_ops_rwsem){++++}-{3:3}, at: cleanup_net+0x9b/0xb00 net/core/net_namespace.c:553
 #3: ffffffff8d0e21a8 (rtnl_mutex){+.+.}-{3:3}, at: wg_netns_pre_exit+0x15/0x190 drivers/net/wireguard/device.c:402
3 locks held by kworker/1:3/2932:
 #0: ffff888027a77938 ((wq_completion)ipv6_addrconf){+.+.}-{0:0}, at: arch_atomic64_set arch/x86/include/asm/atomic64_64.h:34 [inline]
 #0: ffff888027a77938 ((wq_completion)ipv6_addrconf){+.+.}-{0:0}, at: arch_atomic_long_set include/linux/atomic/atomic-long.h:41 [inline]
 #0: ffff888027a77938 ((wq_completion)ipv6_addrconf){+.+.}-{0:0}, at: atomic_long_set include/linux/atomic/atomic-instrumented.h:1198 [inline]
 #0: ffff888027a77938 ((wq_completion)ipv6_addrconf){+.+.}-{0:0}, at: set_work_data kernel/workqueue.c:634 [inline]
 #0: ffff888027a77938 ((wq_completion)ipv6_addrconf){+.+.}-{0:0}, at: set_work_pool_and_clear_pending kernel/workqueue.c:661 [inline]
 #0: ffff888027a77938 ((wq_completion)ipv6_addrconf){+.+.}-{0:0}, at: process_one_work+0x8a3/0x16b0 kernel/workqueue.c:2268
 #1: ffffc9000c517db0 ((addr_chk_work).work){+.+.}-{0:0}, at: process_one_work+0x8d7/0x16b0 kernel/workqueue.c:2272
 #2: ffffffff8d0e21a8 (rtnl_mutex){+.+.}-{3:3}, at: addrconf_verify_work+0xa/0x20 net/ipv6/addrconf.c:4590
1 lock held by in:imklog/6229:
 #0: ffff88806fed6df0 (&f->f_pos_lock){+.+.}-{3:3}, at: __fdget_pos+0xe9/0x100 fs/file.c:990
3 locks held by kworker/0:4/8291:
 #0: ffff888010c67d38 ((wq_completion)events){+.+.}-{0:0}, at: arch_atomic64_set arch/x86/include/asm/atomic64_64.h:34 [inline]
 #0: ffff888010c67d38 ((wq_completion)events){+.+.}-{0:0}, at: arch_atomic_long_set include/linux/atomic/atomic-long.h:41 [inline]
 #0: ffff888010c67d38 ((wq_completion)events){+.+.}-{0:0}, at: atomic_long_set include/linux/atomic/atomic-instrumented.h:1198 [inline]
 #0: ffff888010c67d38 ((wq_completion)events){+.+.}-{0:0}, at: set_work_data kernel/workqueue.c:634 [inline]
 #0: ffff888010c67d38 ((wq_completion)events){+.+.}-{0:0}, at: set_work_pool_and_clear_pending kernel/workqueue.c:661 [inline]
 #0: ffff888010c67d38 ((wq_completion)events){+.+.}-{0:0}, at: process_one_work+0x8a3/0x16b0 kernel/workqueue.c:2268
 #1: ffffc9001698fdb0 ((linkwatch_work).work){+.+.}-{0:0}, at: process_one_work+0x8d7/0x16b0 kernel/workqueue.c:2272
 #2: ffffffff8d0e21a8 (rtnl_mutex){+.+.}-{3:3}, at: linkwatch_event+0xb/0x60 net/core/link_watch.c:251
3 locks held by kworker/0:5/8480:
 #0: ffff888010c67d38 ((wq_completion)events){+.+.}-{0:0}, at: arch_atomic64_set arch/x86/include/asm/atomic64_64.h:34 [inline]
 #0: ffff888010c67d38 ((wq_completion)events){+.+.}-{0:0}, at: arch_atomic_long_set include/linux/atomic/atomic-long.h:41 [inline]
 #0: ffff888010c67d38 ((wq_completion)events){+.+.}-{0:0}, at: atomic_long_set include/linux/atomic/atomic-instrumented.h:1198 [inline]
 #0: ffff888010c67d38 ((wq_completion)events){+.+.}-{0:0}, at: set_work_data kernel/workqueue.c:634 [inline]
 #0: ffff888010c67d38 ((wq_completion)events){+.+.}-{0:0}, at: set_work_pool_and_clear_pending kernel/workqueue.c:661 [inline]
 #0: ffff888010c67d38 ((wq_completion)events){+.+.}-{0:0}, at: process_one_work+0x8a3/0x16b0 kernel/workqueue.c:2268
 #1: ffffc900171cfdb0 (deferred_process_work){+.+.}-{0:0}, at: process_one_work+0x8d7/0x16b0 kernel/workqueue.c:2272
 #2: ffffffff8d0e21a8 (rtnl_mutex){+.+.}-{3:3}, at: switchdev_deferred_process_work+0xa/0x20 net/switchdev/switchdev.c:74
2 locks held by syz-executor.1/13097:
2 locks held by syz-executor.5/13115:
 #0: ffffffff8d0cef50 (pernet_ops_rwsem){++++}-{3:3}, at: copy_net_ns+0x2f5/0x760 net/core/net_namespace.c:466
 #1: ffffffff8d0e21a8 (rtnl_mutex){+.+.}-{3:3}, at: smc_pnet_create_pnetids_list net/smc/smc_pnet.c:798 [inline]
 #1: ffffffff8d0e21a8 (rtnl_mutex){+.+.}-{3:3}, at: smc_pnet_net_init+0x1f9/0x410 net/smc/smc_pnet.c:867
1 lock held by syz-executor.5/13120:
 #0: ffffffff8d0e21a8 (rtnl_mutex){+.+.}-{3:3}, at: rtnl_lock net/core/rtnetlink.c:72 [inline]
 #0: ffffffff8d0e21a8 (rtnl_mutex){+.+.}-{3:3}, at: rtnetlink_rcv_msg+0x3be/0xb80 net/core/rtnetlink.c:5569
2 locks held by syz-executor.5/13156:
 #0: ffffffff8d0cef50 (pernet_ops_rwsem){++++}-{3:3}, at: copy_net_ns+0x2f5/0x760 net/core/net_namespace.c:466
 #1: ffffffff8d0e21a8 (rtnl_mutex){+.+.}-{3:3}, at: smc_pnet_create_pnetids_list net/smc/smc_pnet.c:798 [inline]
 #1: ffffffff8d0e21a8 (rtnl_mutex){+.+.}-{3:3}, at: smc_pnet_net_init+0x1f9/0x410 net/smc/smc_pnet.c:867
1 lock held by syz-executor.5/13162:
 #0: ffffffff8d0e21a8 (rtnl_mutex){+.+.}-{3:3}, at: rtnl_lock net/core/rtnetlink.c:72 [inline]
 #0: ffffffff8d0e21a8 (rtnl_mutex){+.+.}-{3:3}, at: rtnetlink_rcv_msg+0x3be/0xb80 net/core/rtnetlink.c:5569
1 lock held by syz-executor.2/13147:
 #0: ffffffff8d0e21a8 (rtnl_mutex){+.+.}-{3:3}, at: rtnl_lock net/core/rtnetlink.c:72 [inline]
 #0: ffffffff8d0e21a8 (rtnl_mutex){+.+.}-{3:3}, at: rtnetlink_rcv_msg+0x3be/0xb80 net/core/rtnetlink.c:5569
1 lock held by syz-executor.2/13150:
 #0: ffffffff8d0e21a8 (rtnl_mutex){+.+.}-{3:3}, at: rtnl_lock net/core/rtnetlink.c:72 [inline]
 #0: ffffffff8d0e21a8 (rtnl_mutex){+.+.}-{3:3}, at: rtnetlink_rcv_msg+0x3be/0xb80 net/core/rtnetlink.c:5569
1 lock held by syz-executor.2/13153:
 #0: ffffffff8d0e21a8 (rtnl_mutex){+.+.}-{3:3}, at: rtnl_lock net/core/rtnetlink.c:72 [inline]
 #0: ffffffff8d0e21a8 (rtnl_mutex){+.+.}-{3:3}, at: rtnetlink_rcv_msg+0x3be/0xb80 net/core/rtnetlink.c:5569
1 lock held by syz-executor.2/13167:
 #0: ffffffff8d0e21a8 (rtnl_mutex){+.+.}-{3:3}, at: rtnl_lock net/core/rtnetlink.c:72 [inline]
 #0: ffffffff8d0e21a8 (rtnl_mutex){+.+.}-{3:3}, at: rtnetlink_rcv_msg+0x3be/0xb80 net/core/rtnetlink.c:5569
1 lock held by syz-executor.2/13168:
 #0: ffffffff8d0e21a8 (rtnl_mutex){+.+.}-{3:3}, at: rtnl_lock net/core/rtnetlink.c:72 [inline]
 #0: ffffffff8d0e21a8 (rtnl_mutex){+.+.}-{3:3}, at: rtnetlink_rcv_msg+0x3be/0xb80 net/core/rtnetlink.c:5569

=============================================

NMI backtrace for cpu 0
CPU: 0 PID: 27 Comm: khungtaskd Not tainted 5.15.0-rc3-syzkaller #0
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 01/01/2011
Call Trace:
 __dump_stack lib/dump_stack.c:88 [inline]
 dump_stack_lvl+0xcd/0x134 lib/dump_stack.c:106
 nmi_cpu_backtrace.cold+0x47/0x144 lib/nmi_backtrace.c:105
 nmi_trigger_cpumask_backtrace+0x1ae/0x220 lib/nmi_backtrace.c:62
 trigger_all_cpu_backtrace include/linux/nmi.h:146 [inline]
 check_hung_uninterruptible_tasks kernel/hung_task.c:210 [inline]
 watchdog+0xc1d/0xf50 kernel/hung_task.c:295
 kthread+0x3e5/0x4d0 kernel/kthread.c:319
 ret_from_fork+0x1f/0x30 arch/x86/entry/entry_64.S:295
Sending NMI from CPU 0 to CPUs 1:
NMI backtrace for cpu 1
CPU: 1 PID: 154 Comm: kworker/u4:2 Not tainted 5.15.0-rc3-syzkaller #0
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 01/01/2011
Workqueue: phy8 ieee80211_iface_work
RIP: 0010:lock_acquire kernel/locking/lockdep.c:5628 [inline]
RIP: 0010:lock_acquire+0x1d3/0x510 kernel/locking/lockdep.c:5590
Code: ff ff 48 c7 c7 c0 e5 8b 89 48 83 c4 20 e8 45 b4 da 07 b8 ff ff ff ff 65 0f c1 05 a8 70 a7 7e 83 f8 01 0f 85 b4 02 00 00 9c 58 <f6> c4 02 0f 85 9f 02 00 00 48 83 7c 24 08 00 74 01 fb 48 b8 00 00
RSP: 0018:ffffc900011e6e58 EFLAGS: 00000046
RAX: 0000000000000046 RBX: 1ffff9200023cdcd RCX: ffffffff815a36bf
RDX: 1ffff11002293d46 RSI: 0000000000000001 RDI: 0000000000000000
RBP: 0000000000000001 R08: 0000000000000000 R09: ffffffff8fd00927
R10: fffffbfff1fa0124 R11: 000000000000003f R12: 0000000000000000
R13: 0000000000000000 R14: ffffffff90455b70 R15: 0000000000000000
FS:  0000000000000000(0000) GS:ffff8880b9d00000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f056bfab000 CR3: 00000000273a9000 CR4: 00000000003506e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
 __raw_spin_lock_irqsave include/linux/spinlock_api_smp.h:110 [inline]
 _raw_spin_lock_irqsave+0x39/0x50 kernel/locking/spinlock.c:162
 __debug_check_no_obj_freed lib/debugobjects.c:980 [inline]
 debug_check_no_obj_freed+0xc7/0x420 lib/debugobjects.c:1023
 kfree+0xd1/0x2c0 mm/slab.c:3802
 ieee802_11_parse_elems_crc+0xac2/0xfe0 net/mac80211/util.c:1517
 ieee802_11_parse_elems net/mac80211/ieee80211_i.h:2207 [inline]
 ieee80211_bss_info_update+0x468/0xb60 net/mac80211/scan.c:212
 ieee80211_rx_bss_info net/mac80211/ibss.c:1119 [inline]
 ieee80211_rx_mgmt_probe_beacon+0xcce/0x17c0 net/mac80211/ibss.c:1608
 ieee80211_ibss_rx_queued_mgmt+0xd37/0x1610 net/mac80211/ibss.c:1635
 ieee80211_iface_process_skb net/mac80211/iface.c:1439 [inline]
 ieee80211_iface_work+0xa65/0xd00 net/mac80211/iface.c:1493
 process_one_work+0x9bf/0x16b0 kernel/workqueue.c:2297
 worker_thread+0x658/0x11f0 kernel/workqueue.c:2444
 kthread+0x3e5/0x4d0 kernel/kthread.c:319
 ret_from_fork+0x1f/0x30 arch/x86/entry/entry_64.S:295
----------------
Code disassembly (best guess), 1 bytes skipped:
   0:	ff 48 c7             	decl   -0x39(%rax)
   3:	c7 c0 e5 8b 89 48    	mov    $0x48898be5,%eax
   9:	83 c4 20             	add    $0x20,%esp
   c:	e8 45 b4 da 07       	callq  0x7dab456
  11:	b8 ff ff ff ff       	mov    $0xffffffff,%eax
  16:	65 0f c1 05 a8 70 a7 	xadd   %eax,%gs:0x7ea770a8(%rip)        # 0x7ea770c6
  1d:	7e
  1e:	83 f8 01             	cmp    $0x1,%eax
  21:	0f 85 b4 02 00 00    	jne    0x2db
  27:	9c                   	pushfq
  28:	58                   	pop    %rax
* 29:	f6 c4 02             	test   $0x2,%ah <-- trapping instruction
  2c:	0f 85 9f 02 00 00    	jne    0x2d1
  32:	48 83 7c 24 08 00    	cmpq   $0x0,0x8(%rsp)
  38:	74 01                	je     0x3b
  3a:	fb                   	sti
  3b:	48                   	rex.W
  3c:	b8                   	.byte 0xb8


---
This report is generated by a bot. It may contain errors.
See https://goo.gl/tpsmEJ for more information about syzbot.
syzbot engineers can be reached at syzkaller at googlegroups.com.

syzbot will keep track of this issue. See:
https://goo.gl/tpsmEJ#status for how to communicate with syzbot.

From eric.dumazet at gmail.com  Thu Oct 21 19:46:54 2021
From: eric.dumazet at gmail.com (Eric Dumazet)
Date: Thu, 21 Oct 2021 12:46:54 -0700
Subject: [syzbot] INFO: task hung in wg_netns_pre_exit (2)
In-Reply-To: <000000000000c3110805cee20d4e@google.com>
References: <000000000000c3110805cee20d4e@google.com>
Message-ID: <79b847f7-0cd2-5e43-7897-cac478f1d525@gmail.com>



On 10/21/21 12:39 PM, syzbot wrote:
> Hello,
> 
> syzbot found the following issue on:
> 
> HEAD commit:    02d5e016800d Merge tag 'sound-5.15-rc4' of git://git.kerne..
> git tree:       upstream
> console output: https://syzkaller.appspot.com/x/log.txt?x=16e21b17300000
> kernel config:  https://syzkaller.appspot.com/x/.config?x=c76f0f4ac6e9f8d2
> dashboard link: https://syzkaller.appspot.com/bug?extid=fa3b49ed40f26375a8ee
> compiler:       gcc (Debian 10.2.1-6) 10.2.1 20210110, GNU ld (GNU Binutils for Debian) 2.35.2
> 
> Unfortunately, I don't have any reproducer for this issue yet.
> 
> IMPORTANT: if you fix the issue, please add the following tag to the commit:
> Reported-by: syzbot+fa3b49ed40f26375a8ee at syzkaller.appspotmail.com
> 
> INFO: task kworker/u4:3:254 blocked for more than 143 seconds.
>       Not tainted 5.15.0-rc3-syzkaller #0
> "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
> task:kworker/u4:3    state:D stack:24064 pid:  254 ppid:     2 flags:0x00004000
> Workqueue: netns cleanup_net
> Call Trace:
>  context_switch kernel/sched/core.c:4940 [inline]
>  __schedule+0x940/0x26f0 kernel/sched/core.c:6287
>  schedule+0xd3/0x270 kernel/sched/core.c:6366
>  schedule_preempt_disabled+0xf/0x20 kernel/sched/core.c:6425
>  __mutex_lock_common kernel/locking/mutex.c:669 [inline]
>  __mutex_lock+0xa34/0x12f0 kernel/locking/mutex.c:729
>  wg_netns_pre_exit+0x15/0x190 drivers/net/wireguard/device.c:402
>  ops_pre_exit_list net/core/net_namespace.c:158 [inline]
>  cleanup_net+0x451/0xb00 net/core/net_namespace.c:579
>  process_one_work+0x9bf/0x16b0 kernel/workqueue.c:2297
>  worker_thread+0x658/0x11f0 kernel/workqueue.c:2444
>  kthread+0x3e5/0x4d0 kernel/kthread.c:319
>  ret_from_fork+0x1f/0x30 arch/x86/entry/entry_64.S:295
> INFO: task syz-executor.5:13115 blocked for more than 143 seconds.
>       Not tainted 5.15.0-rc3-syzkaller #0
> "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
> task:syz-executor.5  state:D stack:28440 pid:13115 ppid:  6871 flags:0x00004004
> Call Trace:
>  context_switch kernel/sched/core.c:4940 [inline]
>  __schedule+0x940/0x26f0 kernel/sched/core.c:6287
>  schedule+0xd3/0x270 kernel/sched/core.c:6366
>  schedule_preempt_disabled+0xf/0x20 kernel/sched/core.c:6425
>  __mutex_lock_common kernel/locking/mutex.c:669 [inline]
>  __mutex_lock+0xa34/0x12f0 kernel/locking/mutex.c:729
>  smc_pnet_create_pnetids_list net/smc/smc_pnet.c:798 [inline]
>  smc_pnet_net_init+0x1f9/0x410 net/smc/smc_pnet.c:867
>  ops_init+0xaf/0x470 net/core/net_namespace.c:140
>  setup_net+0x40f/0xa30 net/core/net_namespace.c:326
>  copy_net_ns+0x319/0x760 net/core/net_namespace.c:470
>  create_new_namespaces+0x3f6/0xb20 kernel/nsproxy.c:110
>  unshare_nsproxy_namespaces+0xc1/0x1f0 kernel/nsproxy.c:226
>  ksys_unshare+0x445/0x920 kernel/fork.c:3077
>  __do_sys_unshare kernel/fork.c:3151 [inline]
>  __se_sys_unshare kernel/fork.c:3149 [inline]
>  __x64_sys_unshare+0x2d/0x40 kernel/fork.c:3149
>  do_syscall_x64 arch/x86/entry/common.c:50 [inline]
>  do_syscall_64+0x35/0xb0 arch/x86/entry/common.c:80
>  entry_SYSCALL_64_after_hwframe+0x44/0xae
> RIP: 0033:0x7f2348aa4709
> RSP: 002b:00007f234601b188 EFLAGS: 00000246 ORIG_RAX: 0000000000000110
> RAX: ffffffffffffffda RBX: 00007f2348ba8f60 RCX: 00007f2348aa4709
> RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000048040200
> RBP: 00007f2348afecb4 R08: 0000000000000000 R09: 0000000000000000
> R10: 0000000000000000 R11: 0000000000000246 R12: 0000000000000000
> R13: 00007ffe1dab5b5f R14: 00007f234601b300 R15: 0000000000022000
> INFO: task syz-executor.5:13120 blocked for more than 144 seconds.
>       Not tainted 5.15.0-rc3-syzkaller #0
> "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
> task:syz-executor.5  state:D stack:26688 pid:13120 ppid:  6871 flags:0x00004004
> Call Trace:
>  context_switch kernel/sched/core.c:4940 [inline]
>  __schedule+0x940/0x26f0 kernel/sched/core.c:6287
>  schedule+0xd3/0x270 kernel/sched/core.c:6366
>  schedule_preempt_disabled+0xf/0x20 kernel/sched/core.c:6425
>  __mutex_lock_common kernel/locking/mutex.c:669 [inline]
>  __mutex_lock+0xa34/0x12f0 kernel/locking/mutex.c:729
>  rtnl_lock net/core/rtnetlink.c:72 [inline]
>  rtnetlink_rcv_msg+0x3be/0xb80 net/core/rtnetlink.c:5569
>  netlink_rcv_skb+0x153/0x420 net/netlink/af_netlink.c:2504
>  netlink_unicast_kernel net/netlink/af_netlink.c:1314 [inline]
>  netlink_unicast+0x533/0x7d0 net/netlink/af_netlink.c:1340
>  netlink_sendmsg+0x86d/0xdb0 net/netlink/af_netlink.c:1929
>  sock_sendmsg_nosec net/socket.c:704 [inline]
>  sock_sendmsg+0xcf/0x120 net/socket.c:724
>  ____sys_sendmsg+0x6e8/0x810 net/socket.c:2409
>  ___sys_sendmsg+0xf3/0x170 net/socket.c:2463
>  __sys_sendmsg+0xe5/0x1b0 net/socket.c:2492
>  do_syscall_x64 arch/x86/entry/common.c:50 [inline]
>  do_syscall_64+0x35/0xb0 arch/x86/entry/common.c:80
>  entry_SYSCALL_64_after_hwframe+0x44/0xae
> RIP: 0033:0x7f2348aa4709
> RSP: 002b:00007f2345ffa188 EFLAGS: 00000246 ORIG_RAX: 000000000000002e
> RAX: ffffffffffffffda RBX: 00007f2348ba9020 RCX: 00007f2348aa4709
> RDX: 0000000000000000 RSI: 0000000020000100 RDI: 0000000000000003
> RBP: 00007f2348afecb4 R08: 0000000000000000 R09: 0000000000000000
> R10: 0000000000000000 R11: 0000000000000246 R12: 0000000000000000
> R13: 00007ffe1dab5b5f R14: 00007f2345ffa300 R15: 0000000000022000
> INFO: task syz-executor.5:13156 blocked for more than 144 seconds.
>       Not tainted 5.15.0-rc3-syzkaller #0
> "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
> task:syz-executor.5  state:D stack:28696 pid:13156 ppid:  6871 flags:0x00004004
> Call Trace:
>  context_switch kernel/sched/core.c:4940 [inline]
>  __schedule+0x940/0x26f0 kernel/sched/core.c:6287
>  schedule+0xd3/0x270 kernel/sched/core.c:6366
>  schedule_preempt_disabled+0xf/0x20 kernel/sched/core.c:6425
>  __mutex_lock_common kernel/locking/mutex.c:669 [inline]
>  __mutex_lock+0xa34/0x12f0 kernel/locking/mutex.c:729
>  smc_pnet_create_pnetids_list net/smc/smc_pnet.c:798 [inline]
>  smc_pnet_net_init+0x1f9/0x410 net/smc/smc_pnet.c:867
>  ops_init+0xaf/0x470 net/core/net_namespace.c:140
>  setup_net+0x40f/0xa30 net/core/net_namespace.c:326
>  copy_net_ns+0x319/0x760 net/core/net_namespace.c:470
>  create_new_namespaces+0x3f6/0xb20 kernel/nsproxy.c:110
>  unshare_nsproxy_namespaces+0xc1/0x1f0 kernel/nsproxy.c:226
>  ksys_unshare+0x445/0x920 kernel/fork.c:3077
>  __do_sys_unshare kernel/fork.c:3151 [inline]
>  __se_sys_unshare kernel/fork.c:3149 [inline]
>  __x64_sys_unshare+0x2d/0x40 kernel/fork.c:3149
>  do_syscall_x64 arch/x86/entry/common.c:50 [inline]
>  do_syscall_64+0x35/0xb0 arch/x86/entry/common.c:80
>  entry_SYSCALL_64_after_hwframe+0x44/0xae
> RIP: 0033:0x7f2348aa4709
> RSP: 002b:00007f2345fd9188 EFLAGS: 00000246 ORIG_RAX: 0000000000000110
> RAX: ffffffffffffffda RBX: 00007f2348ba90e0 RCX: 00007f2348aa4709
> RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000048040200
> RBP: 00007f2348afecb4 R08: 0000000000000000 R09: 0000000000000000
> R10: 0000000000000000 R11: 0000000000000246 R12: 0000000000000000
> R13: 00007ffe1dab5b5f R14: 00007f2345fd9300 R15: 0000000000022000
> INFO: task syz-executor.5:13162 blocked for more than 145 seconds.
>       Not tainted 5.15.0-rc3-syzkaller #0
> "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
> task:syz-executor.5  state:D stack:28024 pid:13162 ppid:  6871 flags:0x00000004
> Call Trace:
>  context_switch kernel/sched/core.c:4940 [inline]
>  __schedule+0x940/0x26f0 kernel/sched/core.c:6287
>  schedule+0xd3/0x270 kernel/sched/core.c:6366
>  schedule_preempt_disabled+0xf/0x20 kernel/sched/core.c:6425
>  __mutex_lock_common kernel/locking/mutex.c:669 [inline]
>  __mutex_lock+0xa34/0x12f0 kernel/locking/mutex.c:729
>  rtnl_lock net/core/rtnetlink.c:72 [inline]
>  rtnetlink_rcv_msg+0x3be/0xb80 net/core/rtnetlink.c:5569
>  netlink_rcv_skb+0x153/0x420 net/netlink/af_netlink.c:2504
>  netlink_unicast_kernel net/netlink/af_netlink.c:1314 [inline]
>  netlink_unicast+0x533/0x7d0 net/netlink/af_netlink.c:1340
>  netlink_sendmsg+0x86d/0xdb0 net/netlink/af_netlink.c:1929
>  sock_sendmsg_nosec net/socket.c:704 [inline]
>  sock_sendmsg+0xcf/0x120 net/socket.c:724
>  ____sys_sendmsg+0x6e8/0x810 net/socket.c:2409
>  ___sys_sendmsg+0xf3/0x170 net/socket.c:2463
>  __sys_sendmsg+0xe5/0x1b0 net/socket.c:2492
>  do_syscall_x64 arch/x86/entry/common.c:50 [inline]
>  do_syscall_64+0x35/0xb0 arch/x86/entry/common.c:80
>  entry_SYSCALL_64_after_hwframe+0x44/0xae
> RIP: 0033:0x7f2348aa4709
> RSP: 002b:00007f2345f97188 EFLAGS: 00000246 ORIG_RAX: 000000000000002e
> RAX: ffffffffffffffda RBX: 00007f2348ba9260 RCX: 00007f2348aa4709
> RDX: 0000000000000000 RSI: 0000000020000100 RDI: 0000000000000003
> RBP: 00007f2348afecb4 R08: 0000000000000000 R09: 0000000000000000
> R10: 0000000000000000 R11: 0000000000000246 R12: 0000000000000000
> R13: 00007ffe1dab5b5f R14: 00007f2345f97300 R15: 0000000000022000
> INFO: task syz-executor.2:13147 blocked for more than 145 seconds.
>       Not tainted 5.15.0-rc3-syzkaller #0
> "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
> task:syz-executor.2  state:D stack:28072 pid:13147 ppid:  6566 flags:0x00000004
> Call Trace:
>  context_switch kernel/sched/core.c:4940 [inline]
>  __schedule+0x940/0x26f0 kernel/sched/core.c:6287
>  schedule+0xd3/0x270 kernel/sched/core.c:6366
>  schedule_preempt_disabled+0xf/0x20 kernel/sched/core.c:6425
>  __mutex_lock_common kernel/locking/mutex.c:669 [inline]
>  __mutex_lock+0xa34/0x12f0 kernel/locking/mutex.c:729
>  rtnl_lock net/core/rtnetlink.c:72 [inline]
>  rtnetlink_rcv_msg+0x3be/0xb80 net/core/rtnetlink.c:5569
>  netlink_rcv_skb+0x153/0x420 net/netlink/af_netlink.c:2504
>  netlink_unicast_kernel net/netlink/af_netlink.c:1314 [inline]
>  netlink_unicast+0x533/0x7d0 net/netlink/af_netlink.c:1340
>  netlink_sendmsg+0x86d/0xdb0 net/netlink/af_netlink.c:1929
>  sock_sendmsg_nosec net/socket.c:704 [inline]
>  sock_sendmsg+0xcf/0x120 net/socket.c:724
>  ____sys_sendmsg+0x6e8/0x810 net/socket.c:2409
>  ___sys_sendmsg+0xf3/0x170 net/socket.c:2463
>  __sys_sendmsg+0xe5/0x1b0 net/socket.c:2492
>  do_syscall_x64 arch/x86/entry/common.c:50 [inline]
>  do_syscall_64+0x35/0xb0 arch/x86/entry/common.c:80
>  entry_SYSCALL_64_after_hwframe+0x44/0xae
> RIP: 0033:0x7fccb83ee709
> RSP: 002b:00007fccb5965188 EFLAGS: 00000246 ORIG_RAX: 000000000000002e
> RAX: ffffffffffffffda RBX: 00007fccb84f2f60 RCX: 00007fccb83ee709
> RDX: 0000000000000000 RSI: 0000000020000040 RDI: 0000000000000006
> RBP: 00007fccb8448cb4 R08: 0000000000000000 R09: 0000000000000000
> R10: 0000000000000000 R11: 0000000000000246 R12: 0000000000000000
> R13: 00007ffd586b757f R14: 00007fccb5965300 R15: 0000000000022000
> INFO: task syz-executor.2:13150 blocked for more than 145 seconds.
>       Not tainted 5.15.0-rc3-syzkaller #0
> "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
> task:syz-executor.2  state:D stack:28656 pid:13150 ppid:  6566 flags:0x00000004
> Call Trace:
>  context_switch kernel/sched/core.c:4940 [inline]
>  __schedule+0x940/0x26f0 kernel/sched/core.c:6287
>  schedule+0xd3/0x270 kernel/sched/core.c:6366
>  schedule_preempt_disabled+0xf/0x20 kernel/sched/core.c:6425
>  __mutex_lock_common kernel/locking/mutex.c:669 [inline]
>  __mutex_lock+0xa34/0x12f0 kernel/locking/mutex.c:729
>  rtnl_lock net/core/rtnetlink.c:72 [inline]
>  rtnetlink_rcv_msg+0x3be/0xb80 net/core/rtnetlink.c:5569
>  netlink_rcv_skb+0x153/0x420 net/netlink/af_netlink.c:2504
>  netlink_unicast_kernel net/netlink/af_netlink.c:1314 [inline]
>  netlink_unicast+0x533/0x7d0 net/netlink/af_netlink.c:1340
>  netlink_sendmsg+0x86d/0xdb0 net/netlink/af_netlink.c:1929
>  sock_sendmsg_nosec net/socket.c:704 [inline]
>  sock_sendmsg+0xcf/0x120 net/socket.c:724
>  ____sys_sendmsg+0x6e8/0x810 net/socket.c:2409
>  ___sys_sendmsg+0xf3/0x170 net/socket.c:2463
>  __sys_sendmsg+0xe5/0x1b0 net/socket.c:2492
>  do_syscall_x64 arch/x86/entry/common.c:50 [inline]
>  do_syscall_64+0x35/0xb0 arch/x86/entry/common.c:80
>  entry_SYSCALL_64_after_hwframe+0x44/0xae
> RIP: 0033:0x7fccb83ee709
> RSP: 002b:00007fccb5944188 EFLAGS: 00000246 ORIG_RAX: 000000000000002e
> RAX: ffffffffffffffda RBX: 00007fccb84f3020 RCX: 00007fccb83ee709
> RDX: 0000000000000000 RSI: 0000000020000180 RDI: 0000000000000005
> RBP: 00007fccb8448cb4 R08: 0000000000000000 R09: 0000000000000000
> R10: 0000000000000000 R11: 0000000000000246 R12: 0000000000000000
> R13: 00007ffd586b757f R14: 00007fccb5944300 R15: 0000000000022000
> INFO: task syz-executor.2:13153 blocked for more than 146 seconds.
>       Not tainted 5.15.0-rc3-syzkaller #0
> "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
> task:syz-executor.2  state:D stack:28656 pid:13153 ppid:  6566 flags:0x00004004
> Call Trace:
>  context_switch kernel/sched/core.c:4940 [inline]
>  __schedule+0x940/0x26f0 kernel/sched/core.c:6287
>  schedule+0xd3/0x270 kernel/sched/core.c:6366
>  schedule_preempt_disabled+0xf/0x20 kernel/sched/core.c:6425
>  __mutex_lock_common kernel/locking/mutex.c:669 [inline]
>  __mutex_lock+0xa34/0x12f0 kernel/locking/mutex.c:729
>  rtnl_lock net/core/rtnetlink.c:72 [inline]
>  rtnetlink_rcv_msg+0x3be/0xb80 net/core/rtnetlink.c:5569
>  netlink_rcv_skb+0x153/0x420 net/netlink/af_netlink.c:2504
>  netlink_unicast_kernel net/netlink/af_netlink.c:1314 [inline]
>  netlink_unicast+0x533/0x7d0 net/netlink/af_netlink.c:1340
>  netlink_sendmsg+0x86d/0xdb0 net/netlink/af_netlink.c:1929
>  sock_sendmsg_nosec net/socket.c:704 [inline]
>  sock_sendmsg+0xcf/0x120 net/socket.c:724
>  ____sys_sendmsg+0x6e8/0x810 net/socket.c:2409
>  ___sys_sendmsg+0xf3/0x170 net/socket.c:2463
>  __sys_sendmsg+0xe5/0x1b0 net/socket.c:2492
>  do_syscall_x64 arch/x86/entry/common.c:50 [inline]
>  do_syscall_64+0x35/0xb0 arch/x86/entry/common.c:80
>  entry_SYSCALL_64_after_hwframe+0x44/0xae
> RIP: 0033:0x7fccb83ee709
> RSP: 002b:00007fccb5923188 EFLAGS: 00000246 ORIG_RAX: 000000000000002e
> RAX: ffffffffffffffda RBX: 00007fccb84f30e0 RCX: 00007fccb83ee709
> RDX: 0000000000000000 RSI: 0000000020000080 RDI: 0000000000000004
> RBP: 00007fccb8448cb4 R08: 0000000000000000 R09: 0000000000000000
> R10: 0000000000000000 R11: 0000000000000246 R12: 0000000000000000
> R13: 00007ffd586b757f R14: 00007fccb5923300 R15: 0000000000022000
> INFO: task syz-executor.2:13167 blocked for more than 146 seconds.
>       Not tainted 5.15.0-rc3-syzkaller #0
> "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
> task:syz-executor.2  state:D stack:28656 pid:13167 ppid:  6566 flags:0x00000004
> Call Trace:
>  context_switch kernel/sched/core.c:4940 [inline]
>  __schedule+0x940/0x26f0 kernel/sched/core.c:6287
>  schedule+0xd3/0x270 kernel/sched/core.c:6366
>  schedule_preempt_disabled+0xf/0x20 kernel/sched/core.c:6425
>  __mutex_lock_common kernel/locking/mutex.c:669 [inline]
>  __mutex_lock+0xa34/0x12f0 kernel/locking/mutex.c:729
>  rtnl_lock net/core/rtnetlink.c:72 [inline]
>  rtnetlink_rcv_msg+0x3be/0xb80 net/core/rtnetlink.c:5569
>  netlink_rcv_skb+0x153/0x420 net/netlink/af_netlink.c:2504
>  netlink_unicast_kernel net/netlink/af_netlink.c:1314 [inline]
>  netlink_unicast+0x533/0x7d0 net/netlink/af_netlink.c:1340
>  netlink_sendmsg+0x86d/0xdb0 net/netlink/af_netlink.c:1929
>  sock_sendmsg_nosec net/socket.c:704 [inline]
>  sock_sendmsg+0xcf/0x120 net/socket.c:724
>  ____sys_sendmsg+0x6e8/0x810 net/socket.c:2409
>  ___sys_sendmsg+0xf3/0x170 net/socket.c:2463
>  __sys_sendmsg+0xe5/0x1b0 net/socket.c:2492
>  do_syscall_x64 arch/x86/entry/common.c:50 [inline]
>  do_syscall_64+0x35/0xb0 arch/x86/entry/common.c:80
>  entry_SYSCALL_64_after_hwframe+0x44/0xae
> RIP: 0033:0x7fccb83ee709
> RSP: 002b:00007fccb58e1188 EFLAGS: 00000246 ORIG_RAX: 000000000000002e
> RAX: ffffffffffffffda RBX: 00007fccb84f3260 RCX: 00007fccb83ee709
> RDX: 0000000000000000 RSI: 0000000020000040 RDI: 0000000000000006
> RBP: 00007fccb8448cb4 R08: 0000000000000000 R09: 0000000000000000
> R10: 0000000000000000 R11: 0000000000000246 R12: 0000000000000000
> R13: 00007ffd586b757f R14: 00007fccb58e1300 R15: 0000000000022000
> INFO: task syz-executor.2:13168 blocked for more than 147 seconds.
>       Not tainted 5.15.0-rc3-syzkaller #0
> "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
> task:syz-executor.2  state:D stack:28656 pid:13168 ppid:  6566 flags:0x00004004
> Call Trace:
>  context_switch kernel/sched/core.c:4940 [inline]
>  __schedule+0x940/0x26f0 kernel/sched/core.c:6287
>  schedule+0xd3/0x270 kernel/sched/core.c:6366
>  schedule_preempt_disabled+0xf/0x20 kernel/sched/core.c:6425
>  __mutex_lock_common kernel/locking/mutex.c:669 [inline]
>  __mutex_lock+0xa34/0x12f0 kernel/locking/mutex.c:729
>  rtnl_lock net/core/rtnetlink.c:72 [inline]
>  rtnetlink_rcv_msg+0x3be/0xb80 net/core/rtnetlink.c:5569
>  netlink_rcv_skb+0x153/0x420 net/netlink/af_netlink.c:2504
>  netlink_unicast_kernel net/netlink/af_netlink.c:1314 [inline]
>  netlink_unicast+0x533/0x7d0 net/netlink/af_netlink.c:1340
>  netlink_sendmsg+0x86d/0xdb0 net/netlink/af_netlink.c:1929
>  sock_sendmsg_nosec net/socket.c:704 [inline]
>  sock_sendmsg+0xcf/0x120 net/socket.c:724
>  ____sys_sendmsg+0x6e8/0x810 net/socket.c:2409
>  ___sys_sendmsg+0xf3/0x170 net/socket.c:2463
>  __sys_sendmsg+0xe5/0x1b0 net/socket.c:2492
>  do_syscall_x64 arch/x86/entry/common.c:50 [inline]
>  do_syscall_64+0x35/0xb0 arch/x86/entry/common.c:80
>  entry_SYSCALL_64_after_hwframe+0x44/0xae
> RIP: 0033:0x7fccb83ee709
> RSP: 002b:00007fccb58c0188 EFLAGS: 00000246 ORIG_RAX: 000000000000002e
> RAX: ffffffffffffffda RBX: 00007fccb84f3320 RCX: 00007fccb83ee709
> RDX: 0000000000000000 RSI: 0000000020000180 RDI: 0000000000000005
> RBP: 00007fccb8448cb4 R08: 0000000000000000 R09: 0000000000000000
> R10: 0000000000000000 R11: 0000000000000246 R12: 0000000000000000
> R13: 00007ffd586b757f R14: 00007fccb58c0300 R15: 0000000000022000
> 
> Showing all locks held in the system:
> 1 lock held by khungtaskd/27:
>  #0: ffffffff8b97d420 (rcu_read_lock){....}-{1:2}, at: debug_show_all_locks+0x53/0x260 kernel/locking/lockdep.c:6446
> 4 locks held by kworker/u4:3/254:
>  #0: ffff888140275938 ((wq_completion)netns){+.+.}-{0:0}, at: arch_atomic64_set arch/x86/include/asm/atomic64_64.h:34 [inline]
>  #0: ffff888140275938 ((wq_completion)netns){+.+.}-{0:0}, at: arch_atomic_long_set include/linux/atomic/atomic-long.h:41 [inline]
>  #0: ffff888140275938 ((wq_completion)netns){+.+.}-{0:0}, at: atomic_long_set include/linux/atomic/atomic-instrumented.h:1198 [inline]
>  #0: ffff888140275938 ((wq_completion)netns){+.+.}-{0:0}, at: set_work_data kernel/workqueue.c:634 [inline]
>  #0: ffff888140275938 ((wq_completion)netns){+.+.}-{0:0}, at: set_work_pool_and_clear_pending kernel/workqueue.c:661 [inline]
>  #0: ffff888140275938 ((wq_completion)netns){+.+.}-{0:0}, at: process_one_work+0x8a3/0x16b0 kernel/workqueue.c:2268
>  #1: ffffc90002107db0 (net_cleanup_work){+.+.}-{0:0}, at: process_one_work+0x8d7/0x16b0 kernel/workqueue.c:2272
>  #2: ffffffff8d0cef50 (pernet_ops_rwsem){++++}-{3:3}, at: cleanup_net+0x9b/0xb00 net/core/net_namespace.c:553
>  #3: ffffffff8d0e21a8 (rtnl_mutex){+.+.}-{3:3}, at: wg_netns_pre_exit+0x15/0x190 drivers/net/wireguard/device.c:402
> 3 locks held by kworker/1:3/2932:
>  #0: ffff888027a77938 ((wq_completion)ipv6_addrconf){+.+.}-{0:0}, at: arch_atomic64_set arch/x86/include/asm/atomic64_64.h:34 [inline]
>  #0: ffff888027a77938 ((wq_completion)ipv6_addrconf){+.+.}-{0:0}, at: arch_atomic_long_set include/linux/atomic/atomic-long.h:41 [inline]
>  #0: ffff888027a77938 ((wq_completion)ipv6_addrconf){+.+.}-{0:0}, at: atomic_long_set include/linux/atomic/atomic-instrumented.h:1198 [inline]
>  #0: ffff888027a77938 ((wq_completion)ipv6_addrconf){+.+.}-{0:0}, at: set_work_data kernel/workqueue.c:634 [inline]
>  #0: ffff888027a77938 ((wq_completion)ipv6_addrconf){+.+.}-{0:0}, at: set_work_pool_and_clear_pending kernel/workqueue.c:661 [inline]
>  #0: ffff888027a77938 ((wq_completion)ipv6_addrconf){+.+.}-{0:0}, at: process_one_work+0x8a3/0x16b0 kernel/workqueue.c:2268
>  #1: ffffc9000c517db0 ((addr_chk_work).work){+.+.}-{0:0}, at: process_one_work+0x8d7/0x16b0 kernel/workqueue.c:2272
>  #2: ffffffff8d0e21a8 (rtnl_mutex){+.+.}-{3:3}, at: addrconf_verify_work+0xa/0x20 net/ipv6/addrconf.c:4590
> 1 lock held by in:imklog/6229:
>  #0: ffff88806fed6df0 (&f->f_pos_lock){+.+.}-{3:3}, at: __fdget_pos+0xe9/0x100 fs/file.c:990
> 3 locks held by kworker/0:4/8291:
>  #0: ffff888010c67d38 ((wq_completion)events){+.+.}-{0:0}, at: arch_atomic64_set arch/x86/include/asm/atomic64_64.h:34 [inline]
>  #0: ffff888010c67d38 ((wq_completion)events){+.+.}-{0:0}, at: arch_atomic_long_set include/linux/atomic/atomic-long.h:41 [inline]
>  #0: ffff888010c67d38 ((wq_completion)events){+.+.}-{0:0}, at: atomic_long_set include/linux/atomic/atomic-instrumented.h:1198 [inline]
>  #0: ffff888010c67d38 ((wq_completion)events){+.+.}-{0:0}, at: set_work_data kernel/workqueue.c:634 [inline]
>  #0: ffff888010c67d38 ((wq_completion)events){+.+.}-{0:0}, at: set_work_pool_and_clear_pending kernel/workqueue.c:661 [inline]
>  #0: ffff888010c67d38 ((wq_completion)events){+.+.}-{0:0}, at: process_one_work+0x8a3/0x16b0 kernel/workqueue.c:2268
>  #1: ffffc9001698fdb0 ((linkwatch_work).work){+.+.}-{0:0}, at: process_one_work+0x8d7/0x16b0 kernel/workqueue.c:2272
>  #2: ffffffff8d0e21a8 (rtnl_mutex){+.+.}-{3:3}, at: linkwatch_event+0xb/0x60 net/core/link_watch.c:251
> 3 locks held by kworker/0:5/8480:
>  #0: ffff888010c67d38 ((wq_completion)events){+.+.}-{0:0}, at: arch_atomic64_set arch/x86/include/asm/atomic64_64.h:34 [inline]
>  #0: ffff888010c67d38 ((wq_completion)events){+.+.}-{0:0}, at: arch_atomic_long_set include/linux/atomic/atomic-long.h:41 [inline]
>  #0: ffff888010c67d38 ((wq_completion)events){+.+.}-{0:0}, at: atomic_long_set include/linux/atomic/atomic-instrumented.h:1198 [inline]
>  #0: ffff888010c67d38 ((wq_completion)events){+.+.}-{0:0}, at: set_work_data kernel/workqueue.c:634 [inline]
>  #0: ffff888010c67d38 ((wq_completion)events){+.+.}-{0:0}, at: set_work_pool_and_clear_pending kernel/workqueue.c:661 [inline]
>  #0: ffff888010c67d38 ((wq_completion)events){+.+.}-{0:0}, at: process_one_work+0x8a3/0x16b0 kernel/workqueue.c:2268
>  #1: ffffc900171cfdb0 (deferred_process_work){+.+.}-{0:0}, at: process_one_work+0x8d7/0x16b0 kernel/workqueue.c:2272
>  #2: ffffffff8d0e21a8 (rtnl_mutex){+.+.}-{3:3}, at: switchdev_deferred_process_work+0xa/0x20 net/switchdev/switchdev.c:74
> 2 locks held by syz-executor.1/13097:
> 2 locks held by syz-executor.5/13115:
>  #0: ffffffff8d0cef50 (pernet_ops_rwsem){++++}-{3:3}, at: copy_net_ns+0x2f5/0x760 net/core/net_namespace.c:466
>  #1: ffffffff8d0e21a8 (rtnl_mutex){+.+.}-{3:3}, at: smc_pnet_create_pnetids_list net/smc/smc_pnet.c:798 [inline]
>  #1: ffffffff8d0e21a8 (rtnl_mutex){+.+.}-{3:3}, at: smc_pnet_net_init+0x1f9/0x410 net/smc/smc_pnet.c:867
> 1 lock held by syz-executor.5/13120:
>  #0: ffffffff8d0e21a8 (rtnl_mutex){+.+.}-{3:3}, at: rtnl_lock net/core/rtnetlink.c:72 [inline]
>  #0: ffffffff8d0e21a8 (rtnl_mutex){+.+.}-{3:3}, at: rtnetlink_rcv_msg+0x3be/0xb80 net/core/rtnetlink.c:5569
> 2 locks held by syz-executor.5/13156:
>  #0: ffffffff8d0cef50 (pernet_ops_rwsem){++++}-{3:3}, at: copy_net_ns+0x2f5/0x760 net/core/net_namespace.c:466
>  #1: ffffffff8d0e21a8 (rtnl_mutex){+.+.}-{3:3}, at: smc_pnet_create_pnetids_list net/smc/smc_pnet.c:798 [inline]
>  #1: ffffffff8d0e21a8 (rtnl_mutex){+.+.}-{3:3}, at: smc_pnet_net_init+0x1f9/0x410 net/smc/smc_pnet.c:867
> 1 lock held by syz-executor.5/13162:
>  #0: ffffffff8d0e21a8 (rtnl_mutex){+.+.}-{3:3}, at: rtnl_lock net/core/rtnetlink.c:72 [inline]
>  #0: ffffffff8d0e21a8 (rtnl_mutex){+.+.}-{3:3}, at: rtnetlink_rcv_msg+0x3be/0xb80 net/core/rtnetlink.c:5569
> 1 lock held by syz-executor.2/13147:
>  #0: ffffffff8d0e21a8 (rtnl_mutex){+.+.}-{3:3}, at: rtnl_lock net/core/rtnetlink.c:72 [inline]
>  #0: ffffffff8d0e21a8 (rtnl_mutex){+.+.}-{3:3}, at: rtnetlink_rcv_msg+0x3be/0xb80 net/core/rtnetlink.c:5569
> 1 lock held by syz-executor.2/13150:
>  #0: ffffffff8d0e21a8 (rtnl_mutex){+.+.}-{3:3}, at: rtnl_lock net/core/rtnetlink.c:72 [inline]
>  #0: ffffffff8d0e21a8 (rtnl_mutex){+.+.}-{3:3}, at: rtnetlink_rcv_msg+0x3be/0xb80 net/core/rtnetlink.c:5569
> 1 lock held by syz-executor.2/13153:
>  #0: ffffffff8d0e21a8 (rtnl_mutex){+.+.}-{3:3}, at: rtnl_lock net/core/rtnetlink.c:72 [inline]
>  #0: ffffffff8d0e21a8 (rtnl_mutex){+.+.}-{3:3}, at: rtnetlink_rcv_msg+0x3be/0xb80 net/core/rtnetlink.c:5569
> 1 lock held by syz-executor.2/13167:
>  #0: ffffffff8d0e21a8 (rtnl_mutex){+.+.}-{3:3}, at: rtnl_lock net/core/rtnetlink.c:72 [inline]
>  #0: ffffffff8d0e21a8 (rtnl_mutex){+.+.}-{3:3}, at: rtnetlink_rcv_msg+0x3be/0xb80 net/core/rtnetlink.c:5569
> 1 lock held by syz-executor.2/13168:
>  #0: ffffffff8d0e21a8 (rtnl_mutex){+.+.}-{3:3}, at: rtnl_lock net/core/rtnetlink.c:72 [inline]
>  #0: ffffffff8d0e21a8 (rtnl_mutex){+.+.}-{3:3}, at: rtnetlink_rcv_msg+0x3be/0xb80 net/core/rtnetlink.c:5569
> 
> =============================================
> 
> NMI backtrace for cpu 0
> CPU: 0 PID: 27 Comm: khungtaskd Not tainted 5.15.0-rc3-syzkaller #0
> Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 01/01/2011
> Call Trace:
>  __dump_stack lib/dump_stack.c:88 [inline]
>  dump_stack_lvl+0xcd/0x134 lib/dump_stack.c:106
>  nmi_cpu_backtrace.cold+0x47/0x144 lib/nmi_backtrace.c:105
>  nmi_trigger_cpumask_backtrace+0x1ae/0x220 lib/nmi_backtrace.c:62
>  trigger_all_cpu_backtrace include/linux/nmi.h:146 [inline]
>  check_hung_uninterruptible_tasks kernel/hung_task.c:210 [inline]
>  watchdog+0xc1d/0xf50 kernel/hung_task.c:295
>  kthread+0x3e5/0x4d0 kernel/kthread.c:319
>  ret_from_fork+0x1f/0x30 arch/x86/entry/entry_64.S:295
> Sending NMI from CPU 0 to CPUs 1:
> NMI backtrace for cpu 1
> CPU: 1 PID: 154 Comm: kworker/u4:2 Not tainted 5.15.0-rc3-syzkaller #0
> Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 01/01/2011
> Workqueue: phy8 ieee80211_iface_work
> RIP: 0010:lock_acquire kernel/locking/lockdep.c:5628 [inline]
> RIP: 0010:lock_acquire+0x1d3/0x510 kernel/locking/lockdep.c:5590
> Code: ff ff 48 c7 c7 c0 e5 8b 89 48 83 c4 20 e8 45 b4 da 07 b8 ff ff ff ff 65 0f c1 05 a8 70 a7 7e 83 f8 01 0f 85 b4 02 00 00 9c 58 <f6> c4 02 0f 85 9f 02 00 00 48 83 7c 24 08 00 74 01 fb 48 b8 00 00
> RSP: 0018:ffffc900011e6e58 EFLAGS: 00000046
> RAX: 0000000000000046 RBX: 1ffff9200023cdcd RCX: ffffffff815a36bf
> RDX: 1ffff11002293d46 RSI: 0000000000000001 RDI: 0000000000000000
> RBP: 0000000000000001 R08: 0000000000000000 R09: ffffffff8fd00927
> R10: fffffbfff1fa0124 R11: 000000000000003f R12: 0000000000000000
> R13: 0000000000000000 R14: ffffffff90455b70 R15: 0000000000000000
> FS:  0000000000000000(0000) GS:ffff8880b9d00000(0000) knlGS:0000000000000000
> CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
> CR2: 00007f056bfab000 CR3: 00000000273a9000 CR4: 00000000003506e0
> DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
> DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
> Call Trace:
>  __raw_spin_lock_irqsave include/linux/spinlock_api_smp.h:110 [inline]
>  _raw_spin_lock_irqsave+0x39/0x50 kernel/locking/spinlock.c:162
>  __debug_check_no_obj_freed lib/debugobjects.c:980 [inline]
>  debug_check_no_obj_freed+0xc7/0x420 lib/debugobjects.c:1023
>  kfree+0xd1/0x2c0 mm/slab.c:3802
>  ieee802_11_parse_elems_crc+0xac2/0xfe0 net/mac80211/util.c:1517
>  ieee802_11_parse_elems net/mac80211/ieee80211_i.h:2207 [inline]
>  ieee80211_bss_info_update+0x468/0xb60 net/mac80211/scan.c:212
>  ieee80211_rx_bss_info net/mac80211/ibss.c:1119 [inline]
>  ieee80211_rx_mgmt_probe_beacon+0xcce/0x17c0 net/mac80211/ibss.c:1608
>  ieee80211_ibss_rx_queued_mgmt+0xd37/0x1610 net/mac80211/ibss.c:1635
>  ieee80211_iface_process_skb net/mac80211/iface.c:1439 [inline]
>  ieee80211_iface_work+0xa65/0xd00 net/mac80211/iface.c:1493
>  process_one_work+0x9bf/0x16b0 kernel/workqueue.c:2297
>  worker_thread+0x658/0x11f0 kernel/workqueue.c:2444
>  kthread+0x3e5/0x4d0 kernel/kthread.c:319
>  ret_from_fork+0x1f/0x30 arch/x86/entry/entry_64.S:295
> ----------------
> Code disassembly (best guess), 1 bytes skipped:
>    0:	ff 48 c7             	decl   -0x39(%rax)
>    3:	c7 c0 e5 8b 89 48    	mov    $0x48898be5,%eax
>    9:	83 c4 20             	add    $0x20,%esp
>    c:	e8 45 b4 da 07       	callq  0x7dab456
>   11:	b8 ff ff ff ff       	mov    $0xffffffff,%eax
>   16:	65 0f c1 05 a8 70 a7 	xadd   %eax,%gs:0x7ea770a8(%rip)        # 0x7ea770c6
>   1d:	7e
>   1e:	83 f8 01             	cmp    $0x1,%eax
>   21:	0f 85 b4 02 00 00    	jne    0x2db
>   27:	9c                   	pushfq
>   28:	58                   	pop    %rax
> * 29:	f6 c4 02             	test   $0x2,%ah <-- trapping instruction
>   2c:	0f 85 9f 02 00 00    	jne    0x2d1
>   32:	48 83 7c 24 08 00    	cmpq   $0x0,0x8(%rsp)
>   38:	74 01                	je     0x3b
>   3a:	fb                   	sti
>   3b:	48                   	rex.W
>   3c:	b8                   	.byte 0xb8
> 
> 
> ---
> This report is generated by a bot. It may contain errors.
> See https://goo.gl/tpsmEJ for more information about syzbot.
> syzbot engineers can be reached at syzkaller at googlegroups.com.
> 
> syzbot will keep track of this issue. See:
> https://goo.gl/tpsmEJ#status for how to communicate with syzbot.
> 


Hmm.. ieee80211_iface_work() can apparently process unbounded lists.

Maybe this ?

diff --git a/net/mac80211/iface.c b/net/mac80211/iface.c
index 62c95597704b4031f6352d1b833cddbe22447db5..e9b0ddedb2f3660c4cfce4e58682dce907d00d91 100644
--- a/net/mac80211/iface.c
+++ b/net/mac80211/iface.c
@@ -1494,6 +1494,7 @@ static void ieee80211_iface_work(struct work_struct *work)
 
                kfree_skb(skb);
                kcov_remote_stop();
+               cond_resched();
        }
 
        /* process status queue */
@@ -1504,6 +1505,7 @@ static void ieee80211_iface_work(struct work_struct *work)
                kfree_skb(skb);
 
                kcov_remote_stop();
+               cond_resched();
        }
 
        /* then other type-dependent work */


From mikma.wg at lists.m7n.se  Thu Oct 21 22:04:20 2021
From: mikma.wg at lists.m7n.se (Mikael Magnusson)
Date: Fri, 22 Oct 2021 00:04:20 +0200
Subject: [PATCH wireguard-go] tun/netstack: update gvisor
Message-ID: <20211021220420.168820-1-mikma.wg@lists.m7n.se>

From: Mikael Magnusson <mikma at users.sourceforge.net>

Update gvisor to v0.0.0-20211020211948-f76a604701b6, which requires some
changes to tun.go:

WriteRawPacket: Add function with not implemented error.

CreateNetTUN: Replace stack.AddAddress with stack.AddProtocolAddress, and
fix IPv6 address in error message.

Signed-off-by: Mikael Magnusson <mikma at users.sourceforge.net>
---
 tun/netstack/go.mod |   2 +-
 tun/netstack/go.sum | 386 ++++++++++++++++++++++++++++++++++++++++++++
 tun/netstack/tun.go |  20 ++-
 3 files changed, 403 insertions(+), 5 deletions(-)

diff --git a/tun/netstack/go.mod b/tun/netstack/go.mod
index 3123eec..8db9f4b 100644
--- a/tun/netstack/go.mod
+++ b/tun/netstack/go.mod
@@ -7,5 +7,5 @@ require (
 	golang.org/x/sys v0.0.0-20210423185535-09eb48e85fd7 // indirect
 	golang.org/x/time v0.0.0-20210220033141-f8bda1e9f3ba // indirect
 	golang.zx2c4.com/wireguard v0.0.0-20210424170727-c9db4b7aaa22
-	gvisor.dev/gvisor v0.0.0-20210506004418-fbfeba3024f0
+	gvisor.dev/gvisor v0.0.0-20211020211948-f76a604701b6
 )
diff --git a/tun/netstack/go.sum b/tun/netstack/go.sum
index 8227b4c..33d0f4d 100644
--- a/tun/netstack/go.sum
+++ b/tun/netstack/go.sum
@@ -15,7 +15,10 @@ cloud.google.com/go v0.57.0/go.mod h1:oXiQ6Rzq3RAkkY7N6t3TcE6jE+CIBBbA36lwQ1JyzZ
 cloud.google.com/go v0.62.0/go.mod h1:jmCYTdRCQuc1PHIIJ/maLInMho30T/Y0M4hTdTShOYc=
 cloud.google.com/go v0.65.0/go.mod h1:O5N8zS7uWy9vkA9vayVHs65eM1ubvY4h553ofrNHObY=
 cloud.google.com/go v0.72.0/go.mod h1:M+5Vjvlc2wnp6tjzE102Dw08nGShTscUx2nZMufOKPI=
+cloud.google.com/go v0.74.0/go.mod h1:VV1xSbzvo+9QJOxLDaJfTjx5e+MePCpCWwvftOeQmWk=
 cloud.google.com/go v0.75.0/go.mod h1:VGuuCn7PG0dwsd5XPVm2Mm3wlh3EL55/79EKB6hlPTY=
+cloud.google.com/go v0.78.0/go.mod h1:QjdrLG0uq+YwhjoVOLsS1t7TW8fs36kLs4XO5R5ECHg=
+cloud.google.com/go v0.79.0/go.mod h1:3bzgcEeQlzbuEAYu4mrWhKqWjmpprinYgKJLgKHnbb8=
 cloud.google.com/go/bigquery v1.0.1/go.mod h1:i/xbL2UlR5RvWAURpBYZTtm/cXjCha9lbfbpx4poX+o=
 cloud.google.com/go/bigquery v1.3.0/go.mod h1:PjpwJnslEMmckchkHFfq+HTD2DmtT67aNFKH1/VBDHE=
 cloud.google.com/go/bigquery v1.4.0/go.mod h1:S8dzgnTigyfTmLBfrtrhyYhwRxG72rYxvftPBK2Dvzc=
@@ -34,90 +37,215 @@ cloud.google.com/go/storage v1.6.0/go.mod h1:N7U0C8pVQ/+NIKOBQyamJIeKQKkZ+mxpohl
 cloud.google.com/go/storage v1.8.0/go.mod h1:Wv1Oy7z6Yz3DshWRJFhqM/UCfaWIRTdp0RXyy7KQOVs=
 cloud.google.com/go/storage v1.10.0/go.mod h1:FLPqc6j+Ki4BU591ie1oL6qBQGu2Bl/tZ9ullr3+Kg0=
 dmitri.shuralyov.com/gpu/mtl v0.0.0-20190408044501-666a987793e9/go.mod h1:H6x//7gZCb22OMCxBHrMx7a5I7Hp++hsVxbQ4BYO7hU=
+github.com/Azure/azure-sdk-for-go v16.2.1+incompatible/go.mod h1:9XXNKU+eRnpl9moKnB4QOLf1HestfXbmab5FXxiDBjc=
+github.com/Azure/go-ansiterm v0.0.0-20170929234023-d6e3b3328b78/go.mod h1:LmzpDX56iTiv29bbRTIsUNlaFfuhWRQBWjQdVyAevI8=
+github.com/Azure/go-autorest v14.2.0+incompatible/go.mod h1:r+4oMnoxhatjLLJ6zxSWATqVooLgysK6ZNox3g/xq24=
 github.com/Azure/go-autorest/autorest v0.9.0/go.mod h1:xyHB1BMZT0cuDHU7I0+g046+BFDTQ8rEZB0s4Yfa6bI=
+github.com/Azure/go-autorest/autorest v0.11.1/go.mod h1:JFgpikqFJ/MleTTxwepExTKnFUKKszPS8UavbQYUMuw=
 github.com/Azure/go-autorest/autorest/adal v0.5.0/go.mod h1:8Z9fGy2MpX0PvDjB1pEgQTmVqjGhiHBW7RJJEciWzS0=
+github.com/Azure/go-autorest/autorest/adal v0.9.0/go.mod h1:/c022QCutn2P7uY+/oQWWNcK9YU+MH96NgK+jErpbcg=
+github.com/Azure/go-autorest/autorest/adal v0.9.5/go.mod h1:B7KF7jKIeC9Mct5spmyCB/A8CG/sEz1vwIRGv/bbw7A=
 github.com/Azure/go-autorest/autorest/date v0.1.0/go.mod h1:plvfp3oPSKwf2DNjlBjWF/7vwR+cUD/ELuzDCXwHUVA=
+github.com/Azure/go-autorest/autorest/date v0.3.0/go.mod h1:BI0uouVdmngYNUzGWeSYnokU+TrmwEsOqdt8Y6sso74=
 github.com/Azure/go-autorest/autorest/mocks v0.1.0/go.mod h1:OTyCOPRA2IgIlWxVYxBee2F5Gr4kF2zd2J5cFRaIDN0=
 github.com/Azure/go-autorest/autorest/mocks v0.2.0/go.mod h1:OTyCOPRA2IgIlWxVYxBee2F5Gr4kF2zd2J5cFRaIDN0=
+github.com/Azure/go-autorest/autorest/mocks v0.4.0/go.mod h1:LTp+uSrOhSkaKrUy935gNZuuIPPVsHlr9DSOxSayd+k=
+github.com/Azure/go-autorest/autorest/mocks v0.4.1/go.mod h1:LTp+uSrOhSkaKrUy935gNZuuIPPVsHlr9DSOxSayd+k=
 github.com/Azure/go-autorest/logger v0.1.0/go.mod h1:oExouG+K6PryycPJfVSxi/koC6LSNgds39diKLz7Vrc=
+github.com/Azure/go-autorest/logger v0.2.0/go.mod h1:T9E3cAhj2VqvPOtCYAvby9aBXkZmbF5NWuPV8+WeEW8=
 github.com/Azure/go-autorest/tracing v0.5.0/go.mod h1:r/s2XiOKccPW3HrqB+W0TQzfbtp2fGCgRFtBroKn4Dk=
+github.com/Azure/go-autorest/tracing v0.6.0/go.mod h1:+vhtPC754Xsa23ID7GlGsrdKBpUA79WCAKPPZVC2DeU=
 github.com/BurntSushi/toml v0.3.1/go.mod h1:xHWCNGjB5oqiDr8zfno3MHue2Ht5sIBksp03qcyfWMU=
 github.com/BurntSushi/xgb v0.0.0-20160522181843-27f122750802/go.mod h1:IVnqGOEym/WlBOVXweHU+Q+/VP0lqqI8lqeDx9IjBqo=
+github.com/Microsoft/go-winio v0.4.11/go.mod h1:VhR8bwka0BXejwEJY73c50VrPtXAaKcyvVC4A4RozmA=
+github.com/Microsoft/go-winio v0.4.14/go.mod h1:qXqCSQ3Xa7+6tgxaGTIe4Kpcdsi+P8jBhyzoq1bpyYA=
 github.com/Microsoft/go-winio v0.4.16-0.20201130162521-d1ffc52c7331/go.mod h1:XB6nPKklQyQ7GC9LdcBEcBl8PF76WugXOPRXwdLnMv0=
 github.com/Microsoft/go-winio v0.4.16/go.mod h1:XB6nPKklQyQ7GC9LdcBEcBl8PF76WugXOPRXwdLnMv0=
+github.com/Microsoft/go-winio v0.5.0/go.mod h1:JPGBdM1cNvN/6ISo+n8V5iA4v8pBzdOpzfwIujj1a84=
+github.com/Microsoft/hcsshim v0.8.6/go.mod h1:Op3hHsoHPAvb6lceZHDtd9OkTew38wNoXnJs8iY7rUg=
+github.com/Microsoft/hcsshim v0.8.7-0.20190325164909-8abdbb8205e4/go.mod h1:Op3hHsoHPAvb6lceZHDtd9OkTew38wNoXnJs8iY7rUg=
 github.com/Microsoft/hcsshim v0.8.14/go.mod h1:NtVKoYxQuTLx6gEq0L96c9Ju4JbRJ4nY2ow3VK6a9Lg=
 github.com/NYTimes/gziphandler v0.0.0-20170623195520-56545f4a5d46/go.mod h1:3wb06e3pkSAbeQ52E9H9iFoQsEEwGN64994WTCIhntQ=
+github.com/OneOfOne/xxhash v1.2.2/go.mod h1:HSdplMjZKSmBqAxg5vPj2TmRDmfkzw+cTzAElWljhcU=
 github.com/PuerkitoBio/purell v1.0.0/go.mod h1:c11w/QuzBsJSee3cPx9rAFu61PvFxuPbtSwDGJws/X0=
+github.com/PuerkitoBio/purell v1.1.0/go.mod h1:c11w/QuzBsJSee3cPx9rAFu61PvFxuPbtSwDGJws/X0=
+github.com/PuerkitoBio/purell v1.1.1/go.mod h1:c11w/QuzBsJSee3cPx9rAFu61PvFxuPbtSwDGJws/X0=
 github.com/PuerkitoBio/urlesc v0.0.0-20160726150825-5bd2802263f2/go.mod h1:uGdkoq3SwY9Y+13GIhn11/XLaGBb4BfwItxLd5jeuXE=
+github.com/PuerkitoBio/urlesc v0.0.0-20170810143723-de5bf2ad4578/go.mod h1:uGdkoq3SwY9Y+13GIhn11/XLaGBb4BfwItxLd5jeuXE=
+github.com/Shopify/logrus-bugsnag v0.0.0-20171204204709-577dee27f20d/go.mod h1:HI8ITrYtUY+O+ZhtlqUnD8+KwNPOyugEhfP9fdUIaEQ=
+github.com/alecthomas/template v0.0.0-20160405071501-a0175ee3bccc/go.mod h1:LOuyumcjzFXgccqObfd/Ljyb9UuFJ6TxHnclSeseNhc=
+github.com/alecthomas/template v0.0.0-20190718012654-fb15b899a751/go.mod h1:LOuyumcjzFXgccqObfd/Ljyb9UuFJ6TxHnclSeseNhc=
+github.com/alecthomas/units v0.0.0-20151022065526-2efee857e7cf/go.mod h1:ybxpYRFXyAe+OPACYpWeL0wqObRcbAqCMya13uyzqw0=
+github.com/alecthomas/units v0.0.0-20190717042225-c3de453c63f4/go.mod h1:ybxpYRFXyAe+OPACYpWeL0wqObRcbAqCMya13uyzqw0=
+github.com/alexflint/go-filemutex v0.0.0-20171022225611-72bdc8eae2ae/go.mod h1:CgnQgUtFrFz9mxFNtED3jI5tLDjKlOM+oUF/sTk6ps0=
+github.com/antihax/optional v1.0.0/go.mod h1:uupD/76wgC+ih3iEmQUL+0Ugr19nfwCT1kdvxnR2qWY=
+github.com/armon/consul-api v0.0.0-20180202201655-eb2c6b5be1b6/go.mod h1:grANhF5doyWs3UAsr3K4I6qtAmlQcZDesFNEHPZAzj8=
+github.com/asaskevich/govalidator v0.0.0-20190424111038-f61b66f89f4a/go.mod h1:lB+ZfQJz7igIIfQNfa7Ml4HSf2uFQQRzpGGRXenZAgY=
+github.com/aws/aws-sdk-go v1.15.11/go.mod h1:mFuSZ37Z9YOHbQEwBWztmVzqXrEkub65tZoCYDt7FT0=
+github.com/bazelbuild/rules_go v0.27.0/go.mod h1:MC23Dc/wkXEyk3Wpq6lCqz0ZAYOZDw2DR5y3N1q2i7M=
+github.com/beorn7/perks v0.0.0-20180321164747-3a771d992973/go.mod h1:Dwedo/Wpr24TaqPxmxbtue+5NUziq4I4S80YR8gNf3Q=
+github.com/beorn7/perks v1.0.0/go.mod h1:KWe93zE9D1o94FZ5RNwFwVgaQK1VOXiVxmqh+CedLV8=
+github.com/beorn7/perks v1.0.1/go.mod h1:G2ZrVWU2WbWT9wwq4/hrbKbnv/1ERSJQ0ibhJ6rlkpw=
+github.com/bgentry/speakeasy v0.1.0/go.mod h1:+zsyZBPWlz7T6j88CTgSN5bM796AkVf0kBD4zp0CCIs=
+github.com/bitly/go-simplejson v0.5.0/go.mod h1:cXHtHw4XUPsvGaxgjIAn8PhEWG9NfngEKAMDJEczWVA=
+github.com/blang/semver v3.5.0+incompatible/go.mod h1:kRBLl5iJ+tD4TcOOxsy/0fnwebNt5EWlYSAyrTnjyyk=
+github.com/blang/semver v3.5.1+incompatible/go.mod h1:kRBLl5iJ+tD4TcOOxsy/0fnwebNt5EWlYSAyrTnjyyk=
+github.com/bmizerany/assert v0.0.0-20160611221934-b7ed37b82869/go.mod h1:Ekp36dRnpXw/yCqJaO+ZrUyxD+3VXMFFr56k5XYrpB4=
+github.com/bshuster-repo/logrus-logstash-hook v0.4.1/go.mod h1:zsTqEiSzDgAa/8GZR7E1qaXrhYNDKBYy5/dWPTIflbk=
+github.com/buger/jsonparser v0.0.0-20180808090653-f4dd9f5a6b44/go.mod h1:bbYlZJ7hK1yFx9hf58LP0zeX7UjIGs20ufpu3evjr+s=
+github.com/bugsnag/bugsnag-go v0.0.0-20141110184014-b1d153021fcd/go.mod h1:2oa8nejYd4cQ/b0hMIopN0lCRxU0bueqREvZLWFrtK8=
+github.com/bugsnag/osext v0.0.0-20130617224835-0dd3f918b21b/go.mod h1:obH5gd0BsqsP2LwDJ9aOkm/6J86V6lyAXCoQWGw3K50=
+github.com/bugsnag/panicwrap v0.0.0-20151223152923-e2c28503fcd0/go.mod h1:D/8v3kj0zr8ZAKg1AQ6crr+5VwKN5eIywRkfhyM/+dE=
 github.com/cenkalti/backoff v1.1.1-0.20190506075156-2146c9339422/go.mod h1:b6Nc7NRH5C4aCISLry0tLnTjcuTEvoiqcWDdsU0sOGM=
 github.com/census-instrumentation/opencensus-proto v0.2.1/go.mod h1:f6KPmirojxKA12rnyqOA5BBL4O983OfeGPqjHWSTneU=
+github.com/cespare/xxhash v1.1.0/go.mod h1:XrSqR1VqqWfGrhpAt58auRo0WTKS1nRRg3ghfAqPWnc=
+github.com/cespare/xxhash/v2 v2.1.1/go.mod h1:VGX0DQ3Q6kWi7AoAeZDth3/j3BFtOZR5XLFGgcrjCOs=
+github.com/checkpoint-restore/go-criu/v4 v4.1.0/go.mod h1:xUQBLp4RLc5zJtWY++yjOoMoB5lihDt7fai+75m+rGw=
 github.com/chzyer/logex v1.1.10/go.mod h1:+Ywpsq7O8HXn0nuIou7OrIPyXbp3wmkHB+jjWRnGsAI=
 github.com/chzyer/readline v0.0.0-20180603132655-2972be24d48e/go.mod h1:nSuG5e5PlCu98SY8svDHJxuZscDgtXS6KTTbou5AhLI=
 github.com/chzyer/test v0.0.0-20180213035817-a1ea475d72b1/go.mod h1:Q3SI9o4m/ZMnBNeIyt5eFwwo7qiLfzFZmjNmxjkiQlU=
 github.com/cilium/ebpf v0.0.0-20200110133405-4032b1d8aae3/go.mod h1:MA5e5Lr8slmEg9bt0VpxxWqJlO4iwu3FBdHUzV7wQVg=
 github.com/cilium/ebpf v0.2.0/go.mod h1:To2CFviqOWL/M0gIMsvSMlqe7em/l1ALkX1PyjrX2Qs=
+github.com/cilium/ebpf v0.4.0/go.mod h1:4tRaxcgiL706VnOzHOdBlY8IEAIdxINsQBcU4xJJXRs=
 github.com/client9/misspell v0.3.4/go.mod h1:qj6jICC3Q7zFZvVWo7KLAzC3yx5G7kyvSDkc90ppPyw=
 github.com/cncf/udpa/go v0.0.0-20191209042840-269d4d468f6f/go.mod h1:M8M6+tZqaGXZJjfX53e64911xZQV5JYwmTeXPW+k8Sc=
 github.com/cncf/udpa/go v0.0.0-20200629203442-efcf912fb354/go.mod h1:WmhPx2Nbnhtbo57+VJT5O0JRkEi1Wbu0z5j0R8u5Hbk=
 github.com/cncf/udpa/go v0.0.0-20201120205902-5459f2c99403/go.mod h1:WmhPx2Nbnhtbo57+VJT5O0JRkEi1Wbu0z5j0R8u5Hbk=
+github.com/cncf/xds/go v0.0.0-20210312221358-fbca930ec8ed/go.mod h1:eXthEFrGJvWHgFFCl3hGmgk+/aYT6PnTQLykKQRLhEs=
+github.com/cockroachdb/datadriven v0.0.0-20190809214429-80d97fb3cbaa/go.mod h1:zn76sxSg3SzpJ0PPJaLDCu+Bu0Lg3sKTORVIj19EIF8=
+github.com/containerd/btrfs v1.0.0/go.mod h1:zMcX3qkXTAi9GI50+0HOeuV8LU2ryCE/V2vG/ZBiTss=
+github.com/containerd/cgroups v0.0.0-20190717030353-c4b9ac5c7601/go.mod h1:X9rLEHIqSf/wfK8NsPqxJmeZgW4pcfzdXITDrUSJ6uI=
 github.com/containerd/cgroups v0.0.0-20200531161412-0dbf7f05ba59/go.mod h1:pA0z1pT8KYB3TCXK/ocprsh7MAkoW8bZVzPdih9snmM=
 github.com/containerd/cgroups v0.0.0-20201119153540-4cbc285b3327/go.mod h1:ZJeTFisyysqgcCdecO57Dj79RfL0LNeGiFUqLYQRYLE=
 github.com/containerd/console v0.0.0-20180822173158-c12b1e7919c1/go.mod h1:Tj/on1eG8kiEhd0+fhSDzsPAFESxzBBvdyEgyryXffw=
+github.com/containerd/console v0.0.0-20181022165439-0650fd9eeb50/go.mod h1:Tj/on1eG8kiEhd0+fhSDzsPAFESxzBBvdyEgyryXffw=
 github.com/containerd/console v0.0.0-20191206165004-02ecf6a7291e/go.mod h1:8Pf4gM6VEbTNRIT26AyyU7hxdQU3MvAvxVI0sc00XBE=
 github.com/containerd/console v1.0.1/go.mod h1:XUsP6YE/mKtz6bxc+I8UiKKTP04qjQL4qcS3XoQ5xkw=
+github.com/containerd/containerd v1.3.0/go.mod h1:bC6axHOhabU15QhwfG7w5PipXdVtMXFTttgp+kVtyUA=
 github.com/containerd/containerd v1.3.2/go.mod h1:bC6axHOhabU15QhwfG7w5PipXdVtMXFTttgp+kVtyUA=
 github.com/containerd/containerd v1.3.9/go.mod h1:bC6axHOhabU15QhwfG7w5PipXdVtMXFTttgp+kVtyUA=
 github.com/containerd/continuity v0.0.0-20190426062206-aaeac12a7ffc/go.mod h1:GL3xCUCBDV3CZiTSEKksMWbLE66hEyuu9qyDOOqM47Y=
+github.com/containerd/continuity v0.0.0-20190815185530-f2a389ac0a02/go.mod h1:GL3xCUCBDV3CZiTSEKksMWbLE66hEyuu9qyDOOqM47Y=
 github.com/containerd/continuity v0.0.0-20210208174643-50096c924a4e/go.mod h1:EXlVlkqNba9rJe3j7w3Xa924itAMLgZH4UD/Q4PExuQ=
+github.com/containerd/continuity v0.1.0/go.mod h1:ICJu0PwR54nI0yPEnJ6jcS+J7CZAUXrLh8lPo2knzsM=
+github.com/containerd/fifo v0.0.0-20180307165137-3d5202aec260/go.mod h1:ODA38xgv3Kuk8dQz2ZQXpnv/UZZUHUCL7pnLehbXgQI=
 github.com/containerd/fifo v0.0.0-20190226154929-a9fb20d87448/go.mod h1:ODA38xgv3Kuk8dQz2ZQXpnv/UZZUHUCL7pnLehbXgQI=
 github.com/containerd/fifo v0.0.0-20191213151349-ff969a566b00/go.mod h1:jPQ2IAeZRCYxpS/Cm1495vGFww6ecHmMk1YJH2Q5ln0=
+github.com/containerd/go-cni v1.0.2/go.mod h1:nrNABBHzu0ZwCug9Ije8hL2xBCYh/pjfMb1aZGrrohk=
 github.com/containerd/go-runc v0.0.0-20180907222934-5a6d9f37cfa3/go.mod h1:IV7qH3hrUgRmyYrtgEeGWJfWbgcHL9CSRruz2Vqcph0=
+github.com/containerd/go-runc v0.0.0-20190911050354-e029b79d8cda/go.mod h1:IV7qH3hrUgRmyYrtgEeGWJfWbgcHL9CSRruz2Vqcph0=
 github.com/containerd/go-runc v0.0.0-20200220073739-7016d3ce2328/go.mod h1:PpyHrqVs8FTi9vpyHwPwiNEGaACDxT/N/pLcvMSRA9g=
+github.com/containerd/imgcrypt v1.0.3/go.mod h1:v4X3p/H0lzcvVE0r7whbRYjYuK9Y2KEJnL08tXT63Is=
 github.com/containerd/ttrpc v0.0.0-20190828154514-0e0f228740de/go.mod h1:PvCDdDGpgqzQIzDW1TphrGLssLDZp2GuS+X5DkEJB8o=
+github.com/containerd/ttrpc v0.0.0-20190828172938-92c8520ef9f8/go.mod h1:PvCDdDGpgqzQIzDW1TphrGLssLDZp2GuS+X5DkEJB8o=
 github.com/containerd/ttrpc v1.0.2/go.mod h1:UAxOpgT9ziI0gJrmKvgcZivgxOp8iFPSk8httJEt98Y=
 github.com/containerd/typeurl v0.0.0-20180627222232-a93fcdb778cd/go.mod h1:Cm3kwCdlkCfMSHURc+r6fwoGH6/F1hH3S4sg0rLFWPc=
 github.com/containerd/typeurl v0.0.0-20200205145503-b45ef1f1f737/go.mod h1:TB1hUtrpaiO88KEK56ijojHS1+NeF0izUACaJW2mdXg=
+github.com/containernetworking/cni v0.8.0/go.mod h1:LGwApLUm2FpoOfxTDEeq8T9ipbpZ61X79hmU3w8FmsY=
+github.com/containernetworking/cni v0.8.1/go.mod h1:LGwApLUm2FpoOfxTDEeq8T9ipbpZ61X79hmU3w8FmsY=
+github.com/containernetworking/plugins v0.8.7/go.mod h1:R7lXeZaBzpfqapcAbHRW8/CYwm0dHzbz0XEjofx0uB0=
+github.com/containers/ocicrypt v1.0.3/go.mod h1:CUBa+8MRNL/VkpxYIpaMtgn1WgXGyvPQj8jcy0EVG6g=
+github.com/coreos/bbolt v1.3.2/go.mod h1:iRUV2dpdMOn7Bo10OQBFzIJO9kkE559Wcmn+qkEiiKk=
+github.com/coreos/etcd v3.3.10+incompatible/go.mod h1:uF7uidLiAD3TWHmW31ZFd/JWoc32PjwdhPthX9715RE=
+github.com/coreos/go-iptables v0.4.5/go.mod h1:/mVI274lEDI2ns62jHCDnCyBF9Iwsmekav8Dbxlm1MU=
+github.com/coreos/go-iptables v0.5.0/go.mod h1:/mVI274lEDI2ns62jHCDnCyBF9Iwsmekav8Dbxlm1MU=
+github.com/coreos/go-oidc v2.1.0+incompatible/go.mod h1:CgnwVTmzoESiwO9qyAFEMiHoZ1nMCKZlZ9V6mm3/LKc=
+github.com/coreos/go-semver v0.2.0/go.mod h1:nnelYz7RCh+5ahJtPPxZlU+153eP4D4r3EedlOD2RNk=
+github.com/coreos/go-semver v0.3.0/go.mod h1:nnelYz7RCh+5ahJtPPxZlU+153eP4D4r3EedlOD2RNk=
+github.com/coreos/go-systemd v0.0.0-20161114122254-48702e0da86b/go.mod h1:F5haX7vjVVG0kc13fIWeqUViNPyEJxv/OmvnBo0Yme4=
+github.com/coreos/go-systemd v0.0.0-20180511133405-39ca1b05acc7/go.mod h1:F5haX7vjVVG0kc13fIWeqUViNPyEJxv/OmvnBo0Yme4=
+github.com/coreos/go-systemd v0.0.0-20190321100706-95778dfbb74e/go.mod h1:F5haX7vjVVG0kc13fIWeqUViNPyEJxv/OmvnBo0Yme4=
 github.com/coreos/go-systemd/v22 v22.0.0/go.mod h1:xO0FLkIi5MaZafQlIrOotqXZ90ih+1atmu1JpKERPPk=
 github.com/coreos/go-systemd/v22 v22.1.0/go.mod h1:xO0FLkIi5MaZafQlIrOotqXZ90ih+1atmu1JpKERPPk=
+github.com/coreos/pkg v0.0.0-20160727233714-3ac0863d7acf/go.mod h1:E3G3o1h8I7cfcXa63jLwjI0eiQQMgzzUDFVpN/nH/eA=
+github.com/coreos/pkg v0.0.0-20180928190104-399ea9e2e55f/go.mod h1:E3G3o1h8I7cfcXa63jLwjI0eiQQMgzzUDFVpN/nH/eA=
 github.com/cpuguy83/go-md2man/v2 v2.0.0-20190314233015-f79a8a8ca69d/go.mod h1:maD7wRr/U5Z6m/iR4s+kqSMx2CaBsrgA7czyZG/E6dU=
 github.com/cpuguy83/go-md2man/v2 v2.0.0/go.mod h1:maD7wRr/U5Z6m/iR4s+kqSMx2CaBsrgA7czyZG/E6dU=
+github.com/creack/pty v1.1.7/go.mod h1:lj5s0c3V2DBrqTV7llrYr5NG6My20zk30Fl46Y7DoTY=
+github.com/cyphar/filepath-securejoin v0.2.2/go.mod h1:FpkQEhXnPnOthhzymB7CGsFk2G9VLXONKD9G7QGMM+4=
+github.com/d2g/dhcp4 v0.0.0-20170904100407-a1d1b6c41b1c/go.mod h1:Ct2BUK8SB0YC1SMSibvLzxjeJLnrYEVLULFNiHY9YfQ=
+github.com/d2g/dhcp4client v1.0.0/go.mod h1:j0hNfjhrt2SxUOw55nL0ATM/z4Yt3t2Kd1mW34z5W5s=
+github.com/d2g/dhcp4server v0.0.0-20181031114812-7d4a0a7f59a5/go.mod h1:Eo87+Kg/IX2hfWJfwxMzLyuSZyxSoAug2nGa1G2QAi8=
+github.com/d2g/hardwareaddr v0.0.0-20190221164911-e7d9fbe030e4/go.mod h1:bMl4RjIciD2oAxI7DmWRx6gbeqrkoLqv3MV0vzNad+I=
 github.com/davecgh/go-spew v0.0.0-20151105211317-5215b55f46b2/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
 github.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
 github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
+github.com/denverdino/aliyungo v0.0.0-20190125010748-a747050bb1ba/go.mod h1:dV8lFg6daOBZbT6/BDGIz6Y3WFGn8juu6G+CQ6LHtl0=
 github.com/dgrijalva/jwt-go v3.2.0+incompatible/go.mod h1:E3ru+11k8xSBh+hMPgOLZmtrrCbhqsmaPHjLKYnJCaQ=
+github.com/dgryski/go-sip13 v0.0.0-20181026042036-e10d5fee7954/go.mod h1:vAd38F8PWV+bWy6jNmig1y/TA+kYO4g3RSRF0IAv0no=
+github.com/dnaeon/go-vcr v1.0.1/go.mod h1:aBB1+wY4s93YsC3HHjMBMrwTj2R9FHDzUr9KyGc8n1E=
 github.com/docker/distribution v2.7.1-0.20190205005809-0d3efadf0154+incompatible/go.mod h1:J2gT2udsDAN96Uj4KfcMRqY0/ypR+oyYUYmja8H+y+w=
 github.com/docker/docker v1.4.2-0.20191028175130-9e7d5ac5ea55/go.mod h1:eEKB0N0r5NX/I1kEveEz05bcu8tLC/8azJZsviup8Sk=
 github.com/docker/go-connections v0.3.0/go.mod h1:Gbd7IOopHjR8Iph03tsViu4nIes5XhDvyHbTtUxmeec=
+github.com/docker/go-events v0.0.0-20170721190031-9461782956ad/go.mod h1:Uw6UezgYA44ePAFQYUehOuCzmy5zmg/+nl2ZfMWGkpA=
 github.com/docker/go-events v0.0.0-20190806004212-e31b211e4f1c/go.mod h1:Uw6UezgYA44ePAFQYUehOuCzmy5zmg/+nl2ZfMWGkpA=
+github.com/docker/go-metrics v0.0.1/go.mod h1:cG1hvH2utMXtqgqqYE9plW6lDxS3/5ayHzueweSI3Vw=
 github.com/docker/go-units v0.4.0/go.mod h1:fgPhTUdO+D/Jk86RDLlptpiXQzgHJF7gydDDbaIK4Dk=
+github.com/docker/libtrust v0.0.0-20150114040149-fa567046d9b1/go.mod h1:cyGadeNEkKy96OOhEzfZl+yxihPEzKnqJwvfuSUqbZE=
 github.com/docker/spdystream v0.0.0-20160310174837-449fdfce4d96/go.mod h1:Qh8CwZgvJUkLughtfhJv5dyTYa91l1fOUCrgjqmcifM=
+github.com/docopt/docopt-go v0.0.0-20180111231733-ee0de3bc6815/go.mod h1:WwZ+bS3ebgob9U8Nd0kOddGdZWjyMGR8Wziv+TBNwSE=
+github.com/dustin/go-humanize v0.0.0-20171111073723-bb3d318650d4/go.mod h1:HtrtbFcZ19U5GC7JDqmcUSB87Iq5E25KnS6fMYU6eOk=
 github.com/dustin/go-humanize v1.0.0/go.mod h1:HtrtbFcZ19U5GC7JDqmcUSB87Iq5E25KnS6fMYU6eOk=
 github.com/elazarl/goproxy v0.0.0-20170405201442-c4fc26588b6e/go.mod h1:/Zj4wYkgs4iZTTu3o/KG3Itv/qCCa8VVMlb3i9OVuzc=
+github.com/elazarl/goproxy v0.0.0-20180725130230-947c36da3153/go.mod h1:/Zj4wYkgs4iZTTu3o/KG3Itv/qCCa8VVMlb3i9OVuzc=
 github.com/emicklei/go-restful v0.0.0-20170410110728-ff4f55a20633/go.mod h1:otzb+WCGbkyDHkqmQmT5YD2WR4BBwUdeQoFo8l/7tVs=
+github.com/emicklei/go-restful v2.9.5+incompatible/go.mod h1:otzb+WCGbkyDHkqmQmT5YD2WR4BBwUdeQoFo8l/7tVs=
 github.com/envoyproxy/go-control-plane v0.9.0/go.mod h1:YTl/9mNaCwkRvm6d1a2C3ymFceY/DCBVvsKhRF0iEA4=
 github.com/envoyproxy/go-control-plane v0.9.1-0.20191026205805-5f8ba28d4473/go.mod h1:YTl/9mNaCwkRvm6d1a2C3ymFceY/DCBVvsKhRF0iEA4=
 github.com/envoyproxy/go-control-plane v0.9.4/go.mod h1:6rpuAdCZL397s3pYoYcLgu1mIlRU8Am5FuJP05cCM98=
 github.com/envoyproxy/go-control-plane v0.9.7/go.mod h1:cwu0lG7PUMfa9snN8LXBig5ynNVH9qI8YYLbd1fK2po=
 github.com/envoyproxy/go-control-plane v0.9.9-0.20201210154907-fd9021fe5dad/go.mod h1:cXg6YxExXjJnVBQHBLXeUAgxn2UodCpnH306RInaBQk=
+github.com/envoyproxy/go-control-plane v0.9.9-0.20210512163311-63b5d3c536b0/go.mod h1:hliV/p42l8fGbc6Y9bQ70uLwIvmJyVE5k4iMKlh8wCQ=
 github.com/envoyproxy/protoc-gen-validate v0.1.0/go.mod h1:iSmxcyjqTsJpI2R4NaDN7+kN2VEUnK/pcBlmesArF7c=
 github.com/evanphx/json-patch v4.2.0+incompatible/go.mod h1:50XU6AFN0ol/bzJsmQLiYLvXMP4fmwYFNcr97nuDLSk=
+github.com/evanphx/json-patch v4.9.0+incompatible/go.mod h1:50XU6AFN0ol/bzJsmQLiYLvXMP4fmwYFNcr97nuDLSk=
+github.com/fatih/color v1.7.0/go.mod h1:Zm6kSWBoL9eyXnKyktHP6abPY2pDugNf5KwzbycvMj4=
+github.com/form3tech-oss/jwt-go v3.2.2+incompatible/go.mod h1:pbq4aXjuKjdthFRnoDwaVPLA+WlJuPGy+QneDUgJi2k=
+github.com/frankban/quicktest v1.11.3/go.mod h1:wRf/ReqHper53s+kmmSZizM8NamnL3IM0I9ntUbOk+k=
 github.com/fsnotify/fsnotify v1.4.7/go.mod h1:jwhsz4b93w/PPRr/qN1Yymfu8t87LnFCMoQvtojpjFo=
+github.com/fsnotify/fsnotify v1.4.9/go.mod h1:znqG4EE+3YCdAaPaxE2ZRY/06pZUdp0tY4IgpuI1SZQ=
+github.com/fullsailor/pkcs7 v0.0.0-20190404230743-d7302db945fa/go.mod h1:KnogPXtdwXqoenmZCw6S+25EAm2MkxbG0deNDu4cbSA=
+github.com/garyburd/redigo v0.0.0-20150301180006-535138d7bcd7/go.mod h1:NR3MbYisc3/PwhQ00EMzDiPmrwpPxAn5GI05/YaO1SY=
 github.com/ghodss/yaml v0.0.0-20150909031657-73d445a93680/go.mod h1:4dBDuWmgqj2HViK6kFavaiC9ZROes6MMH2rRYeMEF04=
+github.com/ghodss/yaml v1.0.0/go.mod h1:4dBDuWmgqj2HViK6kFavaiC9ZROes6MMH2rRYeMEF04=
 github.com/go-gl/glfw v0.0.0-20190409004039-e6da0acd62b1/go.mod h1:vR7hzQXu2zJy9AVAgeJqvqgH9Q5CA+iKCZ2gyEVpxRU=
 github.com/go-gl/glfw/v3.3/glfw v0.0.0-20191125211704-12ad95a8df72/go.mod h1:tQ2UAYgL5IevRw8kRxooKSPJfGvJ9fJQFa0TUsXzTg8=
 github.com/go-gl/glfw/v3.3/glfw v0.0.0-20200222043503-6f7a984d4dc4/go.mod h1:tQ2UAYgL5IevRw8kRxooKSPJfGvJ9fJQFa0TUsXzTg8=
+github.com/go-ini/ini v1.25.4/go.mod h1:ByCAeIL28uOIIG0E3PJtZPDL8WnHpFKFOtgjp+3Ies8=
+github.com/go-kit/kit v0.8.0/go.mod h1:xBxKIO96dXMWWy0MnWVtmwkA9/13aqxPnvrjFYMA2as=
+github.com/go-kit/kit v0.9.0/go.mod h1:xBxKIO96dXMWWy0MnWVtmwkA9/13aqxPnvrjFYMA2as=
+github.com/go-logfmt/logfmt v0.3.0/go.mod h1:Qt1PoO58o5twSAckw1HlFXLmHsOX5/0LbT9GBnD5lWE=
+github.com/go-logfmt/logfmt v0.4.0/go.mod h1:3RMwSq7FuexP4Kalkev3ejPJsZTpXXBr9+V4qmtdjCk=
 github.com/go-logr/logr v0.1.0/go.mod h1:ixOQHD9gLJUVQQ2ZOR7zLEifBX6tGkNJF4QyIY7sIas=
+github.com/go-logr/logr v0.2.0/go.mod h1:z6/tIYblkpsD+a4lm/fGIIU9mZ+XfAiaFtq7xTgseGU=
 github.com/go-openapi/jsonpointer v0.0.0-20160704185906-46af16f9f7b1/go.mod h1:+35s3my2LFTysnkMfxsJBAMHj/DoqoB9knIWoYG/Vk0=
+github.com/go-openapi/jsonpointer v0.17.0/go.mod h1:cOnomiV+CVVwFLk0A/MExoFMjwdsUdVpsRhURCKh+3M=
+github.com/go-openapi/jsonpointer v0.19.3/go.mod h1:Pl9vOtqEWErmShwVjC8pYs9cog34VGT37dQOVbmoatg=
 github.com/go-openapi/jsonreference v0.0.0-20160704190145-13c6e3589ad9/go.mod h1:W3Z9FmVs9qj+KR4zFKmDPGiLdk1D9Rlm7cyMvf57TTg=
+github.com/go-openapi/jsonreference v0.17.0/go.mod h1:g4xxGn04lDIRh0GJb5QlpE3HfopLOL6uZrK/VgnsK9I=
+github.com/go-openapi/jsonreference v0.19.3/go.mod h1:rjx6GuL8TTa9VaixXglHmQmIL98+wF9xc8zWvFonSJ8=
 github.com/go-openapi/spec v0.0.0-20160808142527-6aced65f8501/go.mod h1:J8+jY1nAiCcj+friV/PDoE1/3eeccG9LYBs0tYvLOWc=
+github.com/go-openapi/spec v0.19.0/go.mod h1:XkF/MOi14NmjsfZ8VtAKf8pIlbZzyoTvZsdfssdxcBI=
 github.com/go-openapi/swag v0.0.0-20160704191624-1d0bd113de87/go.mod h1:DXUve3Dpr1UfpPtxFw+EFuQ41HhCWZfha5jSVRG7C7I=
+github.com/go-openapi/swag v0.17.0/go.mod h1:AByQ+nYG6gQg71GINrmuDXCPWdL640yX49/kXLo40Tg=
+github.com/go-openapi/swag v0.19.5/go.mod h1:POnQmlKehdgb5mhVOsnJFsivZCEZ/vjK9gh66Z9tfKk=
+github.com/go-stack/stack v1.8.0/go.mod h1:v0f6uXyyMGvRgIKkXu+yp6POWl0qKG85gN/melR3HDY=
+github.com/godbus/dbus v0.0.0-20151105175453-c7fdd8b5cd55/go.mod h1:/YcGZj5zSblfDWMMoOzV4fas9FZnQYTkDnsGvmh2Grw=
+github.com/godbus/dbus v0.0.0-20180201030542-885f9cc04c9c/go.mod h1:/YcGZj5zSblfDWMMoOzV4fas9FZnQYTkDnsGvmh2Grw=
+github.com/godbus/dbus v0.0.0-20190422162347-ade71ed3457e/go.mod h1:bBOAhwG1umN6/6ZUMtDFBMQR8jRg9O75tm9K00oMsK4=
 github.com/godbus/dbus/v5 v5.0.3/go.mod h1:xhWf0FNVPg57R7Z0UbKHbJfkEywrmjJnf7w5xrFpKfA=
 github.com/gofrs/flock v0.8.0/go.mod h1:F1TvTiK9OcQqauNUHlbJvyl9Qa1QvF/gOUDKA14jxHU=
+github.com/gogo/googleapis v1.2.0/go.mod h1:Njal3psf3qN6dwBtQfUmBZh2ybovJ0tlu3o/AC7HYjU=
 github.com/gogo/googleapis v1.4.0/go.mod h1:5YRNX2z1oM5gXdAkurHa942MDgEJyk02w4OecKY87+c=
+github.com/gogo/googleapis v1.4.1/go.mod h1:2lpHqI5OcWCtVElxXnPt+s8oJvMpySlOyM6xDCrzib4=
+github.com/gogo/protobuf v1.1.1/go.mod h1:r8qH/GZQm5c6nD/R0oafs1akxWv10x8SbQlK7atdtwQ=
+github.com/gogo/protobuf v1.2.1/go.mod h1:hp+jE20tsWTFYpLwKvXlhS1hjn+gTNwPg2I6zVXpSg4=
 github.com/gogo/protobuf v1.2.2-0.20190723190241-65acae22fc9d/go.mod h1:SlYgWuQ5SjCEi6WLHjHCa1yvBfUnHcTbrrZtXPKa29o=
 github.com/gogo/protobuf v1.3.1/go.mod h1:SlYgWuQ5SjCEi6WLHjHCa1yvBfUnHcTbrrZtXPKa29o=
+github.com/gogo/protobuf v1.3.2/go.mod h1:P1XiOD3dCwIKUDQYPy72D8LYyHL2YPYrpS2s69NZV8Q=
 github.com/golang/glog v0.0.0-20160126235308-23def4e6c14b/go.mod h1:SBH7ygxi8pfUlaOkMMuAQtPIUF8ecWP5IEl/CR7VP2Q=
 github.com/golang/groupcache v0.0.0-20160516000752-02826c3e7903/go.mod h1:cIg4eruTrX1D+g88fzRXU5OdNfaM+9IcxsU14FzY7Hc=
+github.com/golang/groupcache v0.0.0-20190129154638-5b532d6fd5ef/go.mod h1:cIg4eruTrX1D+g88fzRXU5OdNfaM+9IcxsU14FzY7Hc=
 github.com/golang/groupcache v0.0.0-20190702054246-869f871628b6/go.mod h1:cIg4eruTrX1D+g88fzRXU5OdNfaM+9IcxsU14FzY7Hc=
 github.com/golang/groupcache v0.0.0-20191227052852-215e87163ea7/go.mod h1:cIg4eruTrX1D+g88fzRXU5OdNfaM+9IcxsU14FzY7Hc=
 github.com/golang/groupcache v0.0.0-20200121045136-8c9f03a8e57e/go.mod h1:cIg4eruTrX1D+g88fzRXU5OdNfaM+9IcxsU14FzY7Hc=
@@ -128,6 +256,7 @@ github.com/golang/mock v1.4.0/go.mod h1:UOMv5ysSaYNkG+OFQykRIcU/QvvxJf3p21QfJ2Bt
 github.com/golang/mock v1.4.1/go.mod h1:UOMv5ysSaYNkG+OFQykRIcU/QvvxJf3p21QfJ2Bt3cw=
 github.com/golang/mock v1.4.3/go.mod h1:UOMv5ysSaYNkG+OFQykRIcU/QvvxJf3p21QfJ2Bt3cw=
 github.com/golang/mock v1.4.4/go.mod h1:l3mdAwkq5BuhzHwde/uurv3sEJeZMXNpwsxVWU71h+4=
+github.com/golang/mock v1.5.0/go.mod h1:CWnOUgYIOo4TcNZ0wHX3YZCqsaM1I1Jvs6v3mP3KVu8=
 github.com/golang/protobuf v0.0.0-20161109072736-4bd1920723d7/go.mod h1:6lQm79b+lXiMfvg/cZm0SGofjICqVBUtrP5yJMmIC1U=
 github.com/golang/protobuf v1.2.0/go.mod h1:6lQm79b+lXiMfvg/cZm0SGofjICqVBUtrP5yJMmIC1U=
 github.com/golang/protobuf v1.3.1/go.mod h1:6lQm79b+lXiMfvg/cZm0SGofjICqVBUtrP5yJMmIC1U=
@@ -143,9 +272,12 @@ github.com/golang/protobuf v1.4.0/go.mod h1:jodUvKwWbYaEsadDk5Fwe5c77LiNKVO9IDvq
 github.com/golang/protobuf v1.4.1/go.mod h1:U8fpvMrcmy5pZrNK1lt4xCsGvpyWQ/VVv6QDs8UjoX8=
 github.com/golang/protobuf v1.4.2/go.mod h1:oDoupMAO8OvCJWAcko0GGGIgR6R6ocIYbsSw735rRwI=
 github.com/golang/protobuf v1.4.3/go.mod h1:oDoupMAO8OvCJWAcko0GGGIgR6R6ocIYbsSw735rRwI=
+github.com/golang/protobuf v1.5.0/go.mod h1:FsONVRAS9T7sI+LIUmWTfcYkHO4aIWwzhcaSAoJOfIk=
 github.com/google/btree v0.0.0-20180813153112-4030bb1f1f0c/go.mod h1:lNA+9X1NB3Zf8V7Ke586lFgjr2dZNuvo3lPJSGZ5JPQ=
 github.com/google/btree v1.0.0 h1:0udJVsspx3VBr5FwtLhQQtuAsVc79tTq0ocGIPAU6qo=
 github.com/google/btree v1.0.0/go.mod h1:lNA+9X1NB3Zf8V7Ke586lFgjr2dZNuvo3lPJSGZ5JPQ=
+github.com/google/btree v1.0.1 h1:gK4Kx5IaGY9CD5sPJ36FHiBJ6ZXl0kilRiiCj+jdYp4=
+github.com/google/btree v1.0.1/go.mod h1:xXMiIv4Fb/0kKde4SpL7qlzvu5cMJDRkFDxJfI9uaxA=
 github.com/google/go-cmp v0.2.0/go.mod h1:oXzfMopK8JAjlY9xF4vHSVASa0yLyX7SntLO5aqRK0M=
 github.com/google/go-cmp v0.3.0/go.mod h1:8QqcDgzrUqlUb/G2PQTWiueGozuR1884gddMywk6iLU=
 github.com/google/go-cmp v0.3.1/go.mod h1:8QqcDgzrUqlUb/G2PQTWiueGozuR1884gddMywk6iLU=
@@ -154,11 +286,14 @@ github.com/google/go-cmp v0.4.1/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/
 github.com/google/go-cmp v0.5.0/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE=
 github.com/google/go-cmp v0.5.1/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE=
 github.com/google/go-cmp v0.5.2/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE=
+github.com/google/go-cmp v0.5.3/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE=
 github.com/google/go-cmp v0.5.4/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE=
+github.com/google/go-cmp v0.5.5/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE=
 github.com/google/go-github/v32 v32.1.0/go.mod h1:rIEpZD9CTDQwDK9GDrtMTycQNA4JU3qBsCizh3q2WCI=
 github.com/google/go-querystring v1.0.0/go.mod h1:odCYkC5MyYFN7vkCjXpyrEuKhc/BUO6wN/zVPAxq5ck=
 github.com/google/gofuzz v0.0.0-20161122191042-44d81051d367/go.mod h1:HP5RmnzzSNb993RKQDq4+1A4ia9nllfqcQFTQJedwGI=
 github.com/google/gofuzz v1.0.0/go.mod h1:dBl0BpW6vV/+mYPU4Po3pmUjxk6FQPldtuIdl/M65Eg=
+github.com/google/gofuzz v1.1.0/go.mod h1:dBl0BpW6vV/+mYPU4Po3pmUjxk6FQPldtuIdl/M65Eg=
 github.com/google/martian v2.1.0+incompatible/go.mod h1:9I4somxYTbIHy5NJKHRl3wXiIaQGbYVAs8BPL6v8lEs=
 github.com/google/martian/v3 v3.0.0/go.mod h1:y5Zk1BBys9G+gd6Jrk0W3cC1+ELVxBWuIGO+w/tUAp0=
 github.com/google/martian/v3 v3.1.0/go.mod h1:y5Zk1BBys9G+gd6Jrk0W3cC1+ELVxBWuIGO+w/tUAp0=
@@ -170,114 +305,270 @@ github.com/google/pprof v0.0.0-20200229191704-1ebb73c60ed3/go.mod h1:ZgVRPoUq/hf
 github.com/google/pprof v0.0.0-20200430221834-fc25d7d30c6d/go.mod h1:ZgVRPoUq/hfqzAqh7sHMqb3I9Rq5C59dIz2SbBwJ4eM=
 github.com/google/pprof v0.0.0-20200708004538-1a94d8640e99/go.mod h1:ZgVRPoUq/hfqzAqh7sHMqb3I9Rq5C59dIz2SbBwJ4eM=
 github.com/google/pprof v0.0.0-20201023163331-3e6fc7fc9c4c/go.mod h1:kpwsk12EmLew5upagYY7GY0pfYCcupk39gWOCRROcvE=
+github.com/google/pprof v0.0.0-20201203190320-1bf35d6f28c2/go.mod h1:kpwsk12EmLew5upagYY7GY0pfYCcupk39gWOCRROcvE=
 github.com/google/pprof v0.0.0-20201218002935-b9804c9f04c2/go.mod h1:kpwsk12EmLew5upagYY7GY0pfYCcupk39gWOCRROcvE=
 github.com/google/pprof v0.0.0-20210115211752-39141e76b647/go.mod h1:kpwsk12EmLew5upagYY7GY0pfYCcupk39gWOCRROcvE=
+github.com/google/pprof v0.0.0-20210122040257-d980be63207e/go.mod h1:kpwsk12EmLew5upagYY7GY0pfYCcupk39gWOCRROcvE=
+github.com/google/pprof v0.0.0-20210226084205-cbba55b83ad5/go.mod h1:kpwsk12EmLew5upagYY7GY0pfYCcupk39gWOCRROcvE=
+github.com/google/pprof v0.0.0-20210423192551-a2663126120b/go.mod h1:kpwsk12EmLew5upagYY7GY0pfYCcupk39gWOCRROcvE=
 github.com/google/renameio v0.1.0/go.mod h1:KWCgfxg9yswjAJkECMjeO8J8rahYeXnNhOm40UhjYkI=
 github.com/google/subcommands v1.0.2-0.20190508160503-636abe8753b8/go.mod h1:ZjhPrFU+Olkh9WazFPsl27BQ4UPiG37m3yTrtFlrHVk=
 github.com/google/uuid v1.0.0/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo=
 github.com/google/uuid v1.1.1/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo=
 github.com/google/uuid v1.1.2/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo=
+github.com/google/uuid v1.2.0/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo=
 github.com/googleapis/gax-go/v2 v2.0.4/go.mod h1:0Wqv26UfaUD9n4G6kQubkQ+KchISgw+vpHVxEJEs9eg=
 github.com/googleapis/gax-go/v2 v2.0.5/go.mod h1:DWXyrwAJ9X0FpwwEdw+IPEYBICEFu5mhpdKc/us6bOk=
 github.com/googleapis/gnostic v0.0.0-20170729233727-0c5108395e2d/go.mod h1:sJBsCZ4ayReDTBIg8b9dl28c5xFWyhBTVRp3pOg5EKY=
+github.com/googleapis/gnostic v0.4.0/go.mod h1:on+2t9HRStVgn95RSsFWFz+6Q0Snyqv1awfrALZdbtU=
 github.com/gophercloud/gophercloud v0.1.0/go.mod h1:vxM41WHh5uqHVBMZHzuwNOHh8XEoIEcSTewFxm1c5g8=
+github.com/gopherjs/gopherjs v0.0.0-20181017120253-0766667cb4d1/go.mod h1:wJfORRmW1u3UXTncJ5qlYoELFm8eSnnEO6hX4iZ3EWY=
+github.com/gorilla/handlers v0.0.0-20150720190736-60c7bfde3e33/go.mod h1:Qkdc/uu4tH4g6mTK6auzZ766c4CA0Ng8+o/OAirnOIQ=
+github.com/gorilla/mux v1.7.2/go.mod h1:1lud6UwP+6orDFRuTfBEV8e9/aOM/c4fVVCaMa2zaAs=
+github.com/gorilla/websocket v0.0.0-20170926233335-4201258b820c/go.mod h1:E7qHFY5m1UJ88s3WnNqhKjPHQ0heANvMoAMk2YaljkQ=
+github.com/gorilla/websocket v1.4.0/go.mod h1:E7qHFY5m1UJ88s3WnNqhKjPHQ0heANvMoAMk2YaljkQ=
+github.com/gorilla/websocket v1.4.2/go.mod h1:YR8l580nyteQvAITg2hZ9XVh4b55+EU/adAjf1fMHhE=
 github.com/gregjones/httpcache v0.0.0-20180305231024-9cad4c3443a7/go.mod h1:FecbI9+v66THATjSRHfNgh1IVFe/9kFxbXtjV0ctIMA=
+github.com/grpc-ecosystem/go-grpc-middleware v1.0.0/go.mod h1:FiyG127CGDf3tlThmgyCl78X/SZQqEOJBCDaAfeWzPs=
+github.com/grpc-ecosystem/go-grpc-middleware v1.0.1-0.20190118093823-f849b5445de4/go.mod h1:FiyG127CGDf3tlThmgyCl78X/SZQqEOJBCDaAfeWzPs=
+github.com/grpc-ecosystem/go-grpc-prometheus v1.2.0/go.mod h1:8NvIoxWQoOIhqOTXgfV/d3M/q6VIi02HzZEHgUlZvzk=
+github.com/grpc-ecosystem/grpc-gateway v1.9.0/go.mod h1:vNeuVxBJEsws4ogUvrchl83t/GYV9WGTSLVdBhOQFDY=
+github.com/grpc-ecosystem/grpc-gateway v1.9.5/go.mod h1:vNeuVxBJEsws4ogUvrchl83t/GYV9WGTSLVdBhOQFDY=
+github.com/grpc-ecosystem/grpc-gateway v1.16.0/go.mod h1:BDjrQk3hbvj6Nolgz8mAMFbcEtjT1g+wF4CSlocrBnw=
 github.com/hashicorp/errwrap v1.0.0/go.mod h1:YH+1FKiLXxHSkmPseP+kNlulaMuP3n2brvKWEqk/Jc4=
 github.com/hashicorp/go-multierror v1.1.0/go.mod h1:spPvp8C1qA32ftKqdAHm4hHTbPw+vmowP0z+KUhOZdA=
 github.com/hashicorp/golang-lru v0.5.0/go.mod h1:/m3WP610KZHVQ1SGc6re/UDhFvYD7pJ4Ao+sR/qLZy8=
 github.com/hashicorp/golang-lru v0.5.1/go.mod h1:/m3WP610KZHVQ1SGc6re/UDhFvYD7pJ4Ao+sR/qLZy8=
+github.com/hashicorp/hcl v1.0.0/go.mod h1:E5yfLk+7swimpb2L/Alb/PJmXilQ/rhwaUYs4T20WEQ=
 github.com/hpcloud/tail v1.0.0/go.mod h1:ab1qPbhIpdTxEkNHXyeSf5vhxWSCs/tWer42PpOxQnU=
 github.com/ianlancetaylor/demangle v0.0.0-20181102032728-5e5cf60278f6/go.mod h1:aSSvb/t6k1mPoxDqO4vJh6VOCGPwU4O0C2/Eqndh1Sc=
 github.com/ianlancetaylor/demangle v0.0.0-20200824232613-28f6c0f3b639/go.mod h1:aSSvb/t6k1mPoxDqO4vJh6VOCGPwU4O0C2/Eqndh1Sc=
 github.com/imdario/mergo v0.3.5/go.mod h1:2EnlNZ0deacrJVfApfmtdGgDfMuh/nq6Ok1EcJh5FfA=
+github.com/imdario/mergo v0.3.8/go.mod h1:2EnlNZ0deacrJVfApfmtdGgDfMuh/nq6Ok1EcJh5FfA=
+github.com/imdario/mergo v0.3.9/go.mod h1:2EnlNZ0deacrJVfApfmtdGgDfMuh/nq6Ok1EcJh5FfA=
 github.com/inconshreveable/mousetrap v1.0.0/go.mod h1:PxqpIevigyE2G7u3NXJIT2ANytuPF1OarO4DADm73n8=
+github.com/j-keck/arping v0.0.0-20160618110441-2cf9dc699c56/go.mod h1:ymszkNOg6tORTn+6F6j+Jc8TOr5osrynvN6ivFWZ2GA=
+github.com/jmespath/go-jmespath v0.0.0-20160202185014-0b12d6b521d8/go.mod h1:Nht3zPeWKUH0NzdCt2Blrr5ys8VGpn0CEB0cQHVjt7k=
+github.com/jmespath/go-jmespath v0.0.0-20160803190731-bd40a432e4c7/go.mod h1:Nht3zPeWKUH0NzdCt2Blrr5ys8VGpn0CEB0cQHVjt7k=
+github.com/jonboulle/clockwork v0.1.0/go.mod h1:Ii8DK3G1RaLaWxj9trq07+26W01tbo22gdxWY5EU2bo=
 github.com/json-iterator/go v0.0.0-20180612202835-f2b4162afba3/go.mod h1:+SdeFBvtyEkXs7REEP0seUULqWtbJapLOCVDaaPEHmU=
+github.com/json-iterator/go v1.1.6/go.mod h1:+SdeFBvtyEkXs7REEP0seUULqWtbJapLOCVDaaPEHmU=
 github.com/json-iterator/go v1.1.7/go.mod h1:KdQUCv79m/52Kvf8AW2vK1V8akMuk1QjK/uOdHXbAo4=
+github.com/json-iterator/go v1.1.10/go.mod h1:KdQUCv79m/52Kvf8AW2vK1V8akMuk1QjK/uOdHXbAo4=
 github.com/jstemmer/go-junit-report v0.0.0-20190106144839-af01ea7f8024/go.mod h1:6v2b51hI/fHJwM22ozAgKL4VKDeJcHhJFhtBdhmNjmU=
 github.com/jstemmer/go-junit-report v0.9.1/go.mod h1:Brl9GWCQeLvo8nXZwPNNblvFj/XSXhF0NWZEnDohbsk=
+github.com/jtolds/gls v4.20.0+incompatible/go.mod h1:QJZ7F/aHp+rZTRtaJ1ow/lLfFfVYBRgL+9YlvaHOwJU=
+github.com/julienschmidt/httprouter v1.2.0/go.mod h1:SYymIcj16QtmaHHD7aYtjjsJG7VTCxuUUipMqKk8s4w=
+github.com/kisielk/errcheck v1.1.0/go.mod h1:EZBBE59ingxPouuu3KfxchcWSUPOHkagtvWXihfKN4Q=
 github.com/kisielk/errcheck v1.2.0/go.mod h1:/BMXB+zMLi60iA8Vv6Ksmxu/1UDYcXs4uQLJ+jE2L00=
+github.com/kisielk/errcheck v1.5.0/go.mod h1:pFxgyoBC7bSaBwPgfKdkLd5X25qrDl4LWUI2bnpBCr8=
 github.com/kisielk/gotool v1.0.0/go.mod h1:XhKaO+MFFWcvkIS/tQcRk01m1F5IRFswLeQ+oQHNcck=
+github.com/klauspost/compress v1.11.13/go.mod h1:aoV0uJVorq1K+umq18yTdKaF57EivdYsUV+/s2qKfXs=
 github.com/konsorten/go-windows-terminal-sequences v1.0.1/go.mod h1:T0+1ngSBFLxvqU3pZ+m/2kptfBszLMUkC4ZK/EgS/cQ=
 github.com/konsorten/go-windows-terminal-sequences v1.0.2/go.mod h1:T0+1ngSBFLxvqU3pZ+m/2kptfBszLMUkC4ZK/EgS/cQ=
+github.com/konsorten/go-windows-terminal-sequences v1.0.3/go.mod h1:T0+1ngSBFLxvqU3pZ+m/2kptfBszLMUkC4ZK/EgS/cQ=
+github.com/kr/logfmt v0.0.0-20140226030751-b84e30acd515/go.mod h1:+0opPa2QZZtGFBFZlji/RkVcI2GknAs/DXo4wKdlNEc=
 github.com/kr/pretty v0.1.0/go.mod h1:dAy3ld7l9f0ibDNOQOHHMYYIIbhfbHSm3C4ZsoJORNo=
+github.com/kr/pretty v0.2.0/go.mod h1:ipq/a2n7PKx3OHsz4KJII5eveXtPO4qwEXGdVfWzfnI=
+github.com/kr/pretty v0.2.1/go.mod h1:ipq/a2n7PKx3OHsz4KJII5eveXtPO4qwEXGdVfWzfnI=
 github.com/kr/pty v1.1.1/go.mod h1:pFQYn66WHrOpPYNljwOMqo10TkYh1fy3cYio2l3bCsQ=
 github.com/kr/pty v1.1.4-0.20190131011033-7dc38fb350b1/go.mod h1:pFQYn66WHrOpPYNljwOMqo10TkYh1fy3cYio2l3bCsQ=
 github.com/kr/text v0.1.0/go.mod h1:4Jbv+DJW3UT/LiOwJeYQe1efqtUx/iVham/4vfdArNI=
+github.com/magiconair/properties v1.8.0/go.mod h1:PppfXfuXeibc/6YijjN8zIbojt8czPbwD3XqdrwzmxQ=
 github.com/mailru/easyjson v0.0.0-20160728113105-d5b7844b561a/go.mod h1:C1wdFJiN94OJF2b5HbByQZoLdCWB1Yqtg26g4irojpc=
+github.com/mailru/easyjson v0.0.0-20180823135443-60711f1a8329/go.mod h1:C1wdFJiN94OJF2b5HbByQZoLdCWB1Yqtg26g4irojpc=
+github.com/mailru/easyjson v0.0.0-20190614124828-94de47d64c63/go.mod h1:C1wdFJiN94OJF2b5HbByQZoLdCWB1Yqtg26g4irojpc=
+github.com/mailru/easyjson v0.0.0-20190626092158-b2ccc519800e/go.mod h1:C1wdFJiN94OJF2b5HbByQZoLdCWB1Yqtg26g4irojpc=
+github.com/mailru/easyjson v0.7.0/go.mod h1:KAzv3t3aY1NaHWoQz1+4F1ccyAH66Jk7yos7ldAVICs=
+github.com/marstr/guid v1.1.0/go.mod h1:74gB1z2wpxxInTG6yaqA7KrtM0NZ+RbrcqDvYHefzho=
 github.com/mattbaird/jsonpatch v0.0.0-20171005235357-81af80346b1a/go.mod h1:M1qoD/MqPgTZIk0EWKB38wE28ACRfVcn+cU08jyArI0=
+github.com/mattn/go-colorable v0.0.9/go.mod h1:9vuHe8Xs5qXnSaW/c/ABM9alt+Vo+STaOChaDxuIBZU=
+github.com/mattn/go-isatty v0.0.4/go.mod h1:M+lRXTBqGeGNdLjl/ufCoiOlB5xdOkqRJdNxMWT7Zi4=
+github.com/mattn/go-runewidth v0.0.2/go.mod h1:LwmH8dsx7+W8Uxz3IHJYH5QSwggIsqBzpuz5H//U1FU=
+github.com/mattn/go-shellwords v1.0.3/go.mod h1:3xCvwCdWdlDJUrvuMn7Wuy9eWs4pE8vqg+NOMyg4B2o=
+github.com/matttproud/golang_protobuf_extensions v1.0.1/go.mod h1:D8He9yQNgCq6Z5Ld7szi9bcBfOoFv/3dc6xSMkL2PC0=
+github.com/matttproud/golang_protobuf_extensions v1.0.2-0.20181231171920-c182affec369/go.mod h1:BSXmuO+STAnVfrANrmjBb36TMTDstsz7MSK+HVaYKv4=
+github.com/miekg/pkcs11 v1.0.3/go.mod h1:XsNlhZGX73bx86s2hdc/FuaLm2CPZJemRLMA+WTFxgs=
+github.com/mistifyio/go-zfs v2.1.2-0.20190413222219-f784269be439+incompatible/go.mod h1:8AuVvqP/mXw1px98n46wfvcGfQ4ci2FwoAjKYxuo3Z4=
+github.com/mitchellh/go-homedir v1.1.0/go.mod h1:SfyaCUpYCn1Vlf4IUYiD9fPX4A5wJrkLzIz1N1q0pr0=
+github.com/mitchellh/mapstructure v1.1.2/go.mod h1:FVVH3fgwuzCH5S8UJGiWEs2h04kUh9fWfEaFds41c1Y=
+github.com/mitchellh/osext v0.0.0-20151018003038-5e2d6d41470f/go.mod h1:OkQIRizQZAeMln+1tSwduZz7+Af5oFlKirV/MSYes2A=
+github.com/moby/locker v1.0.1/go.mod h1:S7SDdo5zpBK84bzzVlKr2V0hz+7x9hWbYC/kq7oQppc=
+github.com/moby/sys/mountinfo v0.4.1/go.mod h1:rEr8tzG/lsIZHBtN/JjGG+LMYx9eXgW2JI+6q0qou+A=
+github.com/moby/sys/symlink v0.1.0/go.mod h1:GGDODQmbFOjFsXvfLVn3+ZRxkch54RkSiGqsZeMYowQ=
+github.com/moby/term v0.0.0-20200312100748-672ec06f55cd/go.mod h1:DdlQx2hp0Ss5/fLikoLlEeIYiATotOjgB//nb973jeo=
 github.com/modern-go/concurrent v0.0.0-20180228061459-e0a39a4cb421/go.mod h1:6dJC0mAP4ikYIbvyc7fijjWJddQyLn8Ig3JB5CqoB9Q=
 github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd/go.mod h1:6dJC0mAP4ikYIbvyc7fijjWJddQyLn8Ig3JB5CqoB9Q=
 github.com/modern-go/reflect2 v0.0.0-20180320133207-05fbef0ca5da/go.mod h1:bx2lNnkwVCuqBIxFjflWJWanXIb3RllmbCylyMrvgv0=
 github.com/modern-go/reflect2 v0.0.0-20180701023420-4b7aa43c6742/go.mod h1:bx2lNnkwVCuqBIxFjflWJWanXIb3RllmbCylyMrvgv0=
 github.com/modern-go/reflect2 v1.0.1/go.mod h1:bx2lNnkwVCuqBIxFjflWJWanXIb3RllmbCylyMrvgv0=
 github.com/mohae/deepcopy v0.0.0-20170308212314-bb9b5e7adda9/go.mod h1:TaXosZuwdSHYgviHp1DAtfrULt5eUgsSMsZf+YrPgl8=
+github.com/mrunalp/fileutils v0.5.0/go.mod h1:M1WthSahJixYnrXQl/DFQuteStB1weuxD2QJNHXfbSQ=
 github.com/munnerz/goautoneg v0.0.0-20120707110453-a547fc61f48d/go.mod h1:+n7T8mK8HuQTcFwEeznm/DIxMOiR9yIdICNftLE1DvQ=
+github.com/munnerz/goautoneg v0.0.0-20191010083416-a7dc8b61c822/go.mod h1:+n7T8mK8HuQTcFwEeznm/DIxMOiR9yIdICNftLE1DvQ=
+github.com/mwitkow/go-conntrack v0.0.0-20161129095857-cc309e4a2223/go.mod h1:qRWi+5nqEBWmkhHvq77mSJWrCKwh8bxhgT7d/eI7P4U=
 github.com/mxk/go-flowrate v0.0.0-20140419014527-cca7078d478f/go.mod h1:ZdcZmHo+o7JKHSa8/e818NopupXU1YMK5fe1lsApnBw=
+github.com/ncw/swift v1.0.47/go.mod h1:23YIA4yWVnGwv2dQlN4bB7egfYX6YLn0Yo/S6zZO/ZM=
+github.com/nxadm/tail v1.4.4/go.mod h1:kenIhsEOeOJmVchQTgglprH7qJGnHDVpk1VPCcaMI8A=
+github.com/oklog/ulid v1.3.1/go.mod h1:CirwcVhetQ6Lv90oh/F+FBtV6XMibvdAFo93nm5qn4U=
+github.com/olekukonko/tablewriter v0.0.0-20170122224234-a0225b3f23b5/go.mod h1:vsDQFd/mU46D+Z4whnwzcISnGGzXWMclvtLoiIKAKIo=
+github.com/onsi/ginkgo v0.0.0-20151202141238-7f8ab55aaf3b/go.mod h1:lLunBs/Ym6LB5Z9jYTR76FiuTmxDTDusOGeTQH+WWjE=
 github.com/onsi/ginkgo v0.0.0-20170829012221-11459a886d9c/go.mod h1:lLunBs/Ym6LB5Z9jYTR76FiuTmxDTDusOGeTQH+WWjE=
 github.com/onsi/ginkgo v1.6.0/go.mod h1:lLunBs/Ym6LB5Z9jYTR76FiuTmxDTDusOGeTQH+WWjE=
 github.com/onsi/ginkgo v1.8.0/go.mod h1:lLunBs/Ym6LB5Z9jYTR76FiuTmxDTDusOGeTQH+WWjE=
+github.com/onsi/ginkgo v1.10.3/go.mod h1:lLunBs/Ym6LB5Z9jYTR76FiuTmxDTDusOGeTQH+WWjE=
+github.com/onsi/ginkgo v1.12.1/go.mod h1:zj2OWP4+oCPe1qIXoGWkgMRwljMUYCdkwsT2108oapk=
+github.com/onsi/gomega v0.0.0-20151007035656-2152b45fa28a/go.mod h1:C1qb7wdrVGGVU+Z6iS04AVkA3Q65CEZX59MT0QO5uiA=
 github.com/onsi/gomega v0.0.0-20170829124025-dcabb60a477c/go.mod h1:C1qb7wdrVGGVU+Z6iS04AVkA3Q65CEZX59MT0QO5uiA=
 github.com/onsi/gomega v1.5.0/go.mod h1:ex+gbHU/CVuBBDIJjb2X0qEXbFg53c61hWP/1CpauHY=
+github.com/onsi/gomega v1.7.1/go.mod h1:XdKZgCCFLUoM/7CFJVPcG8C1xQ1AJ0vpAezJrB7JYyY=
+github.com/onsi/gomega v1.10.0/go.mod h1:Ho0h+IUsWyvy1OpqCwxlQ/21gkhVunqlU8fDGcoTdcA=
 github.com/opencontainers/go-digest v0.0.0-20180430190053-c9281466c8b2/go.mod h1:cMLVZDEM3+U2I4VmLI6N8jQYUd2OVphdqWwCJHrFt2s=
+github.com/opencontainers/go-digest v1.0.0-rc1/go.mod h1:cMLVZDEM3+U2I4VmLI6N8jQYUd2OVphdqWwCJHrFt2s=
+github.com/opencontainers/go-digest v1.0.0-rc1.0.20180430190053-c9281466c8b2/go.mod h1:cMLVZDEM3+U2I4VmLI6N8jQYUd2OVphdqWwCJHrFt2s=
 github.com/opencontainers/go-digest v1.0.0/go.mod h1:0JzlMkj0TRzQZfJkVvzbP0HBR3IKzErnv2BNG4W4MAM=
 github.com/opencontainers/image-spec v1.0.1/go.mod h1:BtxoFyWECRxE4U/7sNtV5W15zMzWCbyJoFRP3s7yZA0=
 github.com/opencontainers/runc v0.0.0-20190115041553-12f6a991201f/go.mod h1:qT5XzbpPznkRYVz/mWwUaVBUv2rmF59PVA73FjuZG0U=
 github.com/opencontainers/runc v0.1.1/go.mod h1:qT5XzbpPznkRYVz/mWwUaVBUv2rmF59PVA73FjuZG0U=
+github.com/opencontainers/runc v1.0.0-rc8.0.20190926000215-3e425f80a8c9/go.mod h1:qT5XzbpPznkRYVz/mWwUaVBUv2rmF59PVA73FjuZG0U=
+github.com/opencontainers/runc v1.0.0-rc90/go.mod h1:qT5XzbpPznkRYVz/mWwUaVBUv2rmF59PVA73FjuZG0U=
 github.com/opencontainers/runtime-spec v1.0.1/go.mod h1:jwyrGlmzljRJv/Fgzds9SsS/C5hL+LL3ko9hs6T5lQ0=
+github.com/opencontainers/runtime-spec v1.0.2-0.20190207185410-29686dbc5559/go.mod h1:jwyrGlmzljRJv/Fgzds9SsS/C5hL+LL3ko9hs6T5lQ0=
 github.com/opencontainers/runtime-spec v1.0.2/go.mod h1:jwyrGlmzljRJv/Fgzds9SsS/C5hL+LL3ko9hs6T5lQ0=
+github.com/opencontainers/runtime-tools v0.0.0-20181011054405-1d69bd0f9c39/go.mod h1:r3f7wjNzSs2extwzU3Y+6pKfobzPh+kKFJ3ofN+3nfs=
+github.com/opencontainers/selinux v1.8.0/go.mod h1:RScLhm78qiWa2gbVCcGkC7tCGdgk3ogry1nUQF8Evvo=
 github.com/pborman/uuid v1.2.0/go.mod h1:X/NO0urCmaxf9VXbdlT7C2Yzkj2IKimNn4k+gtPdI/k=
+github.com/pelletier/go-toml v1.2.0/go.mod h1:5z9KED0ma1S8pY6P1sdut58dfprrGBbd/94hg7ilaic=
+github.com/pelletier/go-toml v1.8.1/go.mod h1:T2/BmBdy8dvIRq1a/8aqjN41wvWlN4lrapLU/GW4pbc=
 github.com/peterbourgon/diskv v2.0.1+incompatible/go.mod h1:uqqh8zWWbv1HBMNONnaR/tNboyR3/BZd58JJSHlUSCU=
+github.com/pkg/errors v0.8.0/go.mod h1:bwawxfHBFNV+L2hUp1rHADufV3IMtnDRdf1r5NINEl0=
 github.com/pkg/errors v0.8.1/go.mod h1:bwawxfHBFNV+L2hUp1rHADufV3IMtnDRdf1r5NINEl0=
 github.com/pkg/errors v0.9.1/go.mod h1:bwawxfHBFNV+L2hUp1rHADufV3IMtnDRdf1r5NINEl0=
 github.com/pmezard/go-difflib v0.0.0-20151028094244-d8ed2627bdf0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=
 github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=
+github.com/pquerna/cachecontrol v0.0.0-20171018203845-0dec1b30a021/go.mod h1:prYjPmNq4d1NPVmpShWobRqXY3q7Vp+80DqgxxUrUIA=
+github.com/prometheus/client_golang v0.9.1/go.mod h1:7SWBe2y4D6OKWSNQJUaRYU/AaXPKyh/dDVn+NZz0KFw=
+github.com/prometheus/client_golang v0.9.2/go.mod h1:OsXs2jCmiKlQ1lTBmv21f2mNfw4xf/QclQDMrYNZzcM=
+github.com/prometheus/client_golang v0.9.3/go.mod h1:/TN21ttK/J9q6uSwhBd54HahCDft0ttaMvbicHlPoso=
+github.com/prometheus/client_golang v1.0.0/go.mod h1:db9x61etRT2tGnBNRi70OPL5FsnadC4Ky3P0J6CfImo=
+github.com/prometheus/client_golang v1.1.0/go.mod h1:I1FGZT9+L76gKKOs5djB6ezCbFQP1xR9D75/vuwEF3g=
+github.com/prometheus/client_golang v1.7.1/go.mod h1:PY5Wy2awLA44sXw4AOSfFBetzPP4j5+D6mVACh+pe2M=
+github.com/prometheus/client_model v0.0.0-20180712105110-5c3871d89910/go.mod h1:MbSGuTsp3dbXC40dX6PRTWyKYBIrTGTE9sqQNg2J8bo=
+github.com/prometheus/client_model v0.0.0-20190129233127-fd36f4220a90/go.mod h1:xMI15A0UPsDsEKsMN9yxemIoYk6Tm2C1GtYGdfGttqA=
 github.com/prometheus/client_model v0.0.0-20190812154241-14fe0d1b01d4/go.mod h1:xMI15A0UPsDsEKsMN9yxemIoYk6Tm2C1GtYGdfGttqA=
+github.com/prometheus/client_model v0.2.0/go.mod h1:xMI15A0UPsDsEKsMN9yxemIoYk6Tm2C1GtYGdfGttqA=
+github.com/prometheus/common v0.0.0-20181113130724-41aa239b4cce/go.mod h1:daVV7qP5qjZbuso7PdcryaAu0sAZbrN9i7WWcTMWvro=
+github.com/prometheus/common v0.0.0-20181126121408-4724e9255275/go.mod h1:daVV7qP5qjZbuso7PdcryaAu0sAZbrN9i7WWcTMWvro=
+github.com/prometheus/common v0.4.0/go.mod h1:TNfzLD0ON7rHzMJeJkieUDPYmFC7Snx/y86RQel1bk4=
+github.com/prometheus/common v0.4.1/go.mod h1:TNfzLD0ON7rHzMJeJkieUDPYmFC7Snx/y86RQel1bk4=
+github.com/prometheus/common v0.6.0/go.mod h1:eBmuwkDJBwy6iBfxCBob6t6dR6ENT/y+J+Zk0j9GMYc=
+github.com/prometheus/common v0.10.0/go.mod h1:Tlit/dnDKsSWFlCLTWaA1cyBgKHSMdTB80sz/V91rCo=
 github.com/prometheus/procfs v0.0.0-20180125133057-cb4147076ac7/go.mod h1:c3At6R/oaqEKCNdg8wHV1ftS6bRYblBhIjjI8uT2IGk=
+github.com/prometheus/procfs v0.0.0-20181005140218-185b4288413d/go.mod h1:c3At6R/oaqEKCNdg8wHV1ftS6bRYblBhIjjI8uT2IGk=
+github.com/prometheus/procfs v0.0.0-20181204211112-1dc9a6cbc91a/go.mod h1:c3At6R/oaqEKCNdg8wHV1ftS6bRYblBhIjjI8uT2IGk=
+github.com/prometheus/procfs v0.0.0-20190507164030-5867b95ac084/go.mod h1:TjEm7ze935MbeOT/UhFTIMYKhuLP4wbCsTZCD3I8kEA=
 github.com/prometheus/procfs v0.0.0-20190522114515-bc1a522cf7b1/go.mod h1:TjEm7ze935MbeOT/UhFTIMYKhuLP4wbCsTZCD3I8kEA=
+github.com/prometheus/procfs v0.0.2/go.mod h1:TjEm7ze935MbeOT/UhFTIMYKhuLP4wbCsTZCD3I8kEA=
+github.com/prometheus/procfs v0.0.3/go.mod h1:4A/X28fw3Fc593LaREMrKMqOKvUAntwMDaekg4FpcdQ=
+github.com/prometheus/procfs v0.0.8/go.mod h1:7Qr8sr6344vo1JqZ6HhLceV9o3AJ1Ff+GxbHq6oeK9A=
+github.com/prometheus/procfs v0.1.3/go.mod h1:lV6e/gmhEcM9IjHGsFOCxxuZ+z1YqCvr4OA4YeYWdaU=
+github.com/prometheus/procfs v0.6.0/go.mod h1:cz+aTbrPOrUb4q7XlbU9ygM+/jj0fzG6c1xBZuNvfVA=
+github.com/prometheus/tsdb v0.7.1/go.mod h1:qhTCs0VvXwvX/y3TZrWD7rabWM+ijKTux40TwIPHuXU=
+github.com/rogpeppe/fastuuid v0.0.0-20150106093220-6724a57986af/go.mod h1:XWv6SoW27p1b0cqNHllgS5HIMJraePCO15w5zCzIWYg=
+github.com/rogpeppe/fastuuid v1.2.0/go.mod h1:jVj6XXZzXRy/MSR5jhDC/2q6DgLz+nrA6LYCDYWNEvQ=
 github.com/rogpeppe/go-internal v1.3.0/go.mod h1:M8bDsm7K2OlrFYOpmOWEs/qY81heoFRclV5y23lUDJ4=
 github.com/russross/blackfriday/v2 v2.0.1/go.mod h1:+Rmxgy9KzJVeS9/2gXHxylqXiyQDYRxCVz55jmeOWTM=
+github.com/safchain/ethtool v0.0.0-20190326074333-42ed695e3de8/go.mod h1:Z0q5wiBQGYcxhMZ6gUqHn6pYNLypFAvaL3UvgZLR0U4=
+github.com/satori/go.uuid v1.2.0/go.mod h1:dA0hQrYB0VpLJoorglMZABFdXlWrHn1NEOzdhQKdks0=
+github.com/seccomp/libseccomp-golang v0.9.1/go.mod h1:GbW5+tmTXfcxTToHLXlScSlAvWlF4P2Ca7zGrPiEpWo=
 github.com/shurcooL/sanitized_anchor_name v1.0.0/go.mod h1:1NzhyTcUVG4SuEtjjoZeVRXNmyL/1OwPU0+IJeTBvfc=
+github.com/sirupsen/logrus v1.0.6/go.mod h1:pMByvHTf9Beacp5x1UXfOR9xyW/9antXMhjMPG0dEzc=
+github.com/sirupsen/logrus v1.2.0/go.mod h1:LxeOpSwHxABJmUn/MG1IvRgCAasNZTLOkJPxbbu5VWo=
 github.com/sirupsen/logrus v1.4.1/go.mod h1:ni0Sbl8bgC9z8RoU9G6nDWqqs/fq4eDPysMBDgk/93Q=
 github.com/sirupsen/logrus v1.4.2/go.mod h1:tLMulIdttU9McNUspp0xgXVQah82FyeX6MwdIuYE2rE=
 github.com/sirupsen/logrus v1.7.0/go.mod h1:yWOB1SBYBC5VeMP7gHvWumXLIWorT60ONWic61uBYv0=
+github.com/smartystreets/assertions v0.0.0-20180927180507-b2de0cb4f26d/go.mod h1:OnSkiWE9lh6wB0YB77sQom3nweQdgAjqCqsofrRNTgc=
+github.com/smartystreets/goconvey v0.0.0-20190330032615-68dc04aab96a/go.mod h1:syvi0/a8iFYH4r/RixwvyeAJjdLS9QV7WQ/tjFTllLA=
+github.com/soheilhy/cmux v0.1.4/go.mod h1:IM3LyeVVIOuxMH7sFAkER9+bJ4dT7Ms6E4xg4kGIyLM=
+github.com/spaolacci/murmur3 v0.0.0-20180118202830-f09979ecbc72/go.mod h1:JwIasOWyU6f++ZhiEuf87xNszmSA2myDM2Kzu9HwQUA=
+github.com/spf13/afero v1.1.2/go.mod h1:j4pytiNVoe2o6bmDsKpLACNPDBIoEAkihy7loJ1B0CQ=
 github.com/spf13/afero v1.2.2/go.mod h1:9ZxEEn6pIJ8Rxe320qSDBk6AsU0r9pR7Q4OcevTdifk=
+github.com/spf13/cast v1.3.0/go.mod h1:Qx5cxh0v+4UWYiBimWS+eyWzqEqokIECu5etghLkUJE=
 github.com/spf13/cobra v0.0.2-0.20171109065643-2da4a54c5cee/go.mod h1:1l0Ry5zgKvJasoi3XT1TypsSe7PqH0Sj9dhYf7v3XqQ=
+github.com/spf13/cobra v0.0.3/go.mod h1:1l0Ry5zgKvJasoi3XT1TypsSe7PqH0Sj9dhYf7v3XqQ=
+github.com/spf13/cobra v1.0.0/go.mod h1:/6GTrnGXV9HjY+aR4k0oJ5tcvakLuG6EuKReYlHNrgE=
+github.com/spf13/jwalterweatherman v1.0.0/go.mod h1:cQK4TGJAtQXfYWX+Ddv3mKDzgVb68N+wFjFa4jdeBTo=
 github.com/spf13/pflag v0.0.0-20170130214245-9ff6c6923cff/go.mod h1:DYY7MBk1bdzusC3SYhjObp+wFpr4gzcvqqNjLnInEg4=
 github.com/spf13/pflag v1.0.1-0.20171106142849-4c012f6dcd95/go.mod h1:DYY7MBk1bdzusC3SYhjObp+wFpr4gzcvqqNjLnInEg4=
+github.com/spf13/pflag v1.0.1/go.mod h1:DYY7MBk1bdzusC3SYhjObp+wFpr4gzcvqqNjLnInEg4=
+github.com/spf13/pflag v1.0.3/go.mod h1:DYY7MBk1bdzusC3SYhjObp+wFpr4gzcvqqNjLnInEg4=
 github.com/spf13/pflag v1.0.5/go.mod h1:McXfInJRrz4CZXVZOBLb0bTZqETkiAhM9Iw0y3An2Bg=
+github.com/spf13/viper v1.4.0/go.mod h1:PTJ7Z/lr49W6bUbkmS1V3by4uWynFiR9p7+dSq/yZzE=
+github.com/stefanberger/go-pkcs11uri v0.0.0-20201008174630-78d3cae3a980/go.mod h1:AO3tvPzVZ/ayst6UlUKUv6rcPQInYe3IknH3jYhAKu8=
 github.com/stretchr/objx v0.1.0/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME=
 github.com/stretchr/objx v0.1.1/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME=
+github.com/stretchr/objx v0.2.0/go.mod h1:qt09Ya8vawLte6SNmTgCsAVtYtaKzEcn8ATUoHMkEqE=
 github.com/stretchr/testify v0.0.0-20151208002404-e3a8ff8ce365/go.mod h1:a8OnRcib4nhh0OaRAV+Yts87kKdq0PP7pXfy6kDkUVs=
 github.com/stretchr/testify v1.2.2/go.mod h1:a8OnRcib4nhh0OaRAV+Yts87kKdq0PP7pXfy6kDkUVs=
 github.com/stretchr/testify v1.3.0/go.mod h1:M5WIy9Dh21IEIfnGCwXGc5bZfKNJtfHm1UVUgZn+9EI=
 github.com/stretchr/testify v1.4.0/go.mod h1:j7eGeouHqKxXV5pUuKE4zz7dFj8WfuZ+81PSLYec5m4=
 github.com/stretchr/testify v1.5.1/go.mod h1:5W2xD1RspED5o8YsWQXVCued0rvSQ+mT+I5cxcmMvtA=
+github.com/stretchr/testify v1.6.1/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg=
 github.com/syndtr/gocapability v0.0.0-20180916011248-d98352740cb2/go.mod h1:hkRG7XYTFWNJGYcbNJQlaLq0fg1yr4J4t/NcTQtrfww=
+github.com/tchap/go-patricia v2.2.6+incompatible/go.mod h1:bmLyhP68RS6kStMGxByiQ23RP/odRBOTVjwp2cDyi6I=
+github.com/tmc/grpc-websocket-proxy v0.0.0-20170815181823-89b8d40f7ca8/go.mod h1:ncp9v5uamzpCO7NfCPTXjqaC+bZgJeR0sMTm6dMHP7U=
+github.com/tmc/grpc-websocket-proxy v0.0.0-20190109142713-0ad062ec5ee5/go.mod h1:ncp9v5uamzpCO7NfCPTXjqaC+bZgJeR0sMTm6dMHP7U=
+github.com/ugorji/go v1.1.4/go.mod h1:uQMGLiO92mf5W77hV/PUCpI3pbzQx3CRekS0kk+RGrc=
+github.com/urfave/cli v1.20.0/go.mod h1:70zkFmudgCuE/ngEzBv17Jvp/497gISqfk5gWijbERA=
+github.com/urfave/cli v1.22.1/go.mod h1:Gos4lmkARVdJ6EkW0WaNv/tZAAMe9V7XWyB60NtXRu0=
 github.com/urfave/cli v1.22.2/go.mod h1:Gos4lmkARVdJ6EkW0WaNv/tZAAMe9V7XWyB60NtXRu0=
+github.com/vishvananda/netlink v0.0.0-20181108222139-023a6dafdcdf/go.mod h1:+SR5DhBJrl6ZM7CoCKvpw5BKroDKQ+PJqOg65H/2ktk=
 github.com/vishvananda/netlink v1.0.1-0.20190930145447-2ec5bdc52b86/go.mod h1:+SR5DhBJrl6ZM7CoCKvpw5BKroDKQ+PJqOg65H/2ktk=
+github.com/vishvananda/netns v0.0.0-20180720170159-13995c7128cc/go.mod h1:ZjcWmFBXmLKZu9Nxj3WKYEafiSqer2rnvPr0en9UNpI=
 github.com/vishvananda/netns v0.0.0-20210104183010-2eb08e3e575f/go.mod h1:DD4vA1DwXk04H54A1oHXtwZmA0grkVMdPxx/VGLCah0=
+github.com/willf/bitset v1.1.11/go.mod h1:83CECat5yLh5zVOf4P1ErAgKA5UDvKtgyUABdr3+MjI=
 github.com/xeipuuv/gojsonpointer v0.0.0-20180127040702-4e3ac2762d5f/go.mod h1:N2zxlSyiKSe5eX1tZViRH5QA0qijqEDrYZiPEAiq3wU=
 github.com/xeipuuv/gojsonreference v0.0.0-20180127040603-bd5ef7bd5415/go.mod h1:GwrjFmJcFw6At/Gs6z4yjiIwzuJ1/+UwLxMQDVQXShQ=
 github.com/xeipuuv/gojsonschema v1.2.0/go.mod h1:anYRn/JVcOK2ZgGU+IjEV4nwlhoK5sQluxsYJ78Id3Y=
+github.com/xiang90/probing v0.0.0-20190116061207-43a291ad63a2/go.mod h1:UETIi67q53MR2AWcXfiuqkDkRtnGDLqkBTpCHuJHxtU=
+github.com/xordataexchange/crypt v0.0.3-0.20170626215501-b2862e3d0a77/go.mod h1:aYKd//L2LvnjZzWKhF00oedf4jCCReLcmhLdhm1A27Q=
 github.com/yuin/goldmark v1.1.25/go.mod h1:3hX8gzYuyVAZsxl0MRgGTJEmQBFcNTphYh9decYSb74=
 github.com/yuin/goldmark v1.1.27/go.mod h1:3hX8gzYuyVAZsxl0MRgGTJEmQBFcNTphYh9decYSb74=
 github.com/yuin/goldmark v1.1.32/go.mod h1:3hX8gzYuyVAZsxl0MRgGTJEmQBFcNTphYh9decYSb74=
 github.com/yuin/goldmark v1.2.1/go.mod h1:3hX8gzYuyVAZsxl0MRgGTJEmQBFcNTphYh9decYSb74=
+github.com/yvasiyarov/go-metrics v0.0.0-20140926110328-57bccd1ccd43/go.mod h1:aX5oPXxHm3bOH+xeAttToC8pqch2ScQN/JoXYupl6xs=
+github.com/yvasiyarov/gorelic v0.0.0-20141212073537-a9bba5b9ab50/go.mod h1:NUSPSUX/bi6SeDMUh6brw0nXpxHnc96TguQh0+r/ssA=
+github.com/yvasiyarov/newrelic_platform_go v0.0.0-20140908184405-b21fdbd4370f/go.mod h1:GlGEuHIJweS1mbCqG+7vt2nvWLzLLnRHbXz5JKd/Qbg=
+go.etcd.io/bbolt v1.3.2/go.mod h1:IbVyRI1SCnLcuJnV2u8VeU0CEYM7e686BmAb1XKL+uU=
+go.etcd.io/bbolt v1.3.3/go.mod h1:IbVyRI1SCnLcuJnV2u8VeU0CEYM7e686BmAb1XKL+uU=
+go.etcd.io/bbolt v1.3.5/go.mod h1:G5EMThwa9y8QZGBClrRx5EY+Yw9kAhnjy3bSjsnlVTQ=
+go.etcd.io/etcd v0.5.0-alpha.5.0.20200910180754-dd1b699fc489/go.mod h1:yVHk9ub3CSBatqGNg7GRmsnfLWtoW60w4eDYfh7vHDg=
+go.mozilla.org/pkcs7 v0.0.0-20200128120323-432b2356ecb1/go.mod h1:SNgMg+EgDFwmvSmLRTNKC5fegJjB7v23qTQ0XLGUNHk=
 go.opencensus.io v0.21.0/go.mod h1:mSImk1erAIZhrmZN+AvHh14ztQfjbGwt4TtuofqLduU=
 go.opencensus.io v0.22.0/go.mod h1:+kGneAE2xo2IficOXnaByMWTGM9T73dGwxeWcUqIpI8=
 go.opencensus.io v0.22.2/go.mod h1:yxeiOL68Rb0Xd1ddK5vPZ/oVn4vY4Ynel7k9FzqtOIw=
 go.opencensus.io v0.22.3/go.mod h1:yxeiOL68Rb0Xd1ddK5vPZ/oVn4vY4Ynel7k9FzqtOIw=
 go.opencensus.io v0.22.4/go.mod h1:yxeiOL68Rb0Xd1ddK5vPZ/oVn4vY4Ynel7k9FzqtOIw=
 go.opencensus.io v0.22.5/go.mod h1:5pWMHQbX5EPX2/62yrJeAkowc+lfs/XD7Uxpq3pI6kk=
+go.opencensus.io v0.23.0/go.mod h1:XItmlyltB5F7CS4xOC1DcqMoFqwtC6OG2xF7mCv7P7E=
+go.opentelemetry.io/proto/otlp v0.7.0/go.mod h1:PqfVotwruBrMGOCsRd/89rSnXhoiJIqeYNgFYFoEGnI=
+go.uber.org/atomic v1.3.2/go.mod h1:gD2HeocX3+yG+ygLZcrzQJaqmWj9AIm7n08wl/qW/PE=
+go.uber.org/atomic v1.4.0/go.mod h1:gD2HeocX3+yG+ygLZcrzQJaqmWj9AIm7n08wl/qW/PE=
 go.uber.org/atomic v1.7.0/go.mod h1:fEN4uk6kAWBTFdckzkM89CLk9XfWZrxpCo0nPH17wJc=
+go.uber.org/multierr v1.1.0/go.mod h1:wR5kodmAFQ0UK8QlbwjlSNy0Z68gJhDJUG5sjR94q/0=
 go.uber.org/multierr v1.6.0/go.mod h1:cdWPpRnG4AhwMwsgIHip0KRBQjJy5kYEpYjJxpXp9iU=
+go.uber.org/zap v1.10.0/go.mod h1:vwi/ZaCAaUcBkycHslxD9B2zi4UTXhF60s6SWpuDF0Q=
+golang.org/x/crypto v0.0.0-20180904163835-0709b304e793/go.mod h1:6SG95UA2DQfeDnfUPMdvaQW0Q7yPrPDi9nlGo2tz2b4=
+golang.org/x/crypto v0.0.0-20181009213950-7c1a557ab941/go.mod h1:6SG95UA2DQfeDnfUPMdvaQW0Q7yPrPDi9nlGo2tz2b4=
 golang.org/x/crypto v0.0.0-20190211182817-74369b46fc67/go.mod h1:6SG95UA2DQfeDnfUPMdvaQW0Q7yPrPDi9nlGo2tz2b4=
 golang.org/x/crypto v0.0.0-20190308221718-c2843e01d9a2/go.mod h1:djNgcEr1/C05ACkg1iLfiJU5Ep61QUkGW8qpdssI0+w=
 golang.org/x/crypto v0.0.0-20190510104115-cbcb75029529/go.mod h1:yigFU9vqHzYiE8UmvKecakEJjdnWj3jj499lnFckfCI=
 golang.org/x/crypto v0.0.0-20190605123033-f99c8df09eb5/go.mod h1:yigFU9vqHzYiE8UmvKecakEJjdnWj3jj499lnFckfCI=
+golang.org/x/crypto v0.0.0-20190701094942-4def268fd1a4/go.mod h1:yigFU9vqHzYiE8UmvKecakEJjdnWj3jj499lnFckfCI=
 golang.org/x/crypto v0.0.0-20191011191535-87dc89f01550/go.mod h1:yigFU9vqHzYiE8UmvKecakEJjdnWj3jj499lnFckfCI=
 golang.org/x/crypto v0.0.0-20200220183623-bac4c82f6975/go.mod h1:LzIPMQfyMNhhGPhUkYOs5KpL4U8rLKemX1yGLhDgUto=
 golang.org/x/crypto v0.0.0-20200622213623-75b288015ac9/go.mod h1:LzIPMQfyMNhhGPhUkYOs5KpL4U8rLKemX1yGLhDgUto=
+golang.org/x/crypto v0.0.0-20201002170205-7f63de1d35b0/go.mod h1:LzIPMQfyMNhhGPhUkYOs5KpL4U8rLKemX1yGLhDgUto=
 golang.org/x/crypto v0.0.0-20210220033148-5ea612d1eb83 h1:/ZScEX8SfEmUGRHs0gxpqteO5nfNW6axyZbBdw9A12g=
 golang.org/x/crypto v0.0.0-20210220033148-5ea612d1eb83/go.mod h1:jdWPYTVW3xRLrWPugEBEK3UY2ZEsg3UU495nc5E+M+I=
 golang.org/x/exp v0.0.0-20190121172915-509febef88a4/go.mod h1:CJ0aWSM057203Lf6IL+f9T1iT9GByDxfZKAQTCR3kQA=
@@ -312,20 +603,30 @@ golang.org/x/mod v0.1.1-0.20191107180719-034126e5016b/go.mod h1:QqPTAvyqsEbceGzB
 golang.org/x/mod v0.2.0/go.mod h1:s0Qsj1ACt9ePp/hMypM3fl4fZqREWJwdYDEqhRiZZUA=
 golang.org/x/mod v0.3.0/go.mod h1:s0Qsj1ACt9ePp/hMypM3fl4fZqREWJwdYDEqhRiZZUA=
 golang.org/x/mod v0.4.0/go.mod h1:s0Qsj1ACt9ePp/hMypM3fl4fZqREWJwdYDEqhRiZZUA=
+golang.org/x/mod v0.4.1/go.mod h1:s0Qsj1ACt9ePp/hMypM3fl4fZqREWJwdYDEqhRiZZUA=
 golang.org/x/net v0.0.0-20170114055629-f2499483f923/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=
 golang.org/x/net v0.0.0-20180724234803-3673e40ba225/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=
 golang.org/x/net v0.0.0-20180826012351-8a410e7b638d/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=
 golang.org/x/net v0.0.0-20180906233101-161cd47e91fd/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=
+golang.org/x/net v0.0.0-20181005035420-146acd28ed58/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=
+golang.org/x/net v0.0.0-20181011144130-49bb7cea24b1/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=
+golang.org/x/net v0.0.0-20181114220301-adae6a3d119a/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=
+golang.org/x/net v0.0.0-20181201002055-351d144fa1fc/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=
+golang.org/x/net v0.0.0-20181220203305-927f97764cc3/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=
 golang.org/x/net v0.0.0-20190108225652-1e06a53dbb7e/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=
 golang.org/x/net v0.0.0-20190213061140-3a22650c66bd/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=
 golang.org/x/net v0.0.0-20190311183353-d8887717615a/go.mod h1:t9HGtf8HONx5eT2rtn7q6eTqICYqUVnKs3thJo3Qplg=
 golang.org/x/net v0.0.0-20190404232315-eb5bcb51f2a3/go.mod h1:t9HGtf8HONx5eT2rtn7q6eTqICYqUVnKs3thJo3Qplg=
 golang.org/x/net v0.0.0-20190501004415-9ce7a6920f09/go.mod h1:t9HGtf8HONx5eT2rtn7q6eTqICYqUVnKs3thJo3Qplg=
 golang.org/x/net v0.0.0-20190503192946-f4e77d36d62c/go.mod h1:t9HGtf8HONx5eT2rtn7q6eTqICYqUVnKs3thJo3Qplg=
+golang.org/x/net v0.0.0-20190522155817-f3200d17e092/go.mod h1:HSz+uSET+XFnRR8LxR5pz3Of3rY3CfYBVs4xY44aLks=
 golang.org/x/net v0.0.0-20190603091049-60506f45cf65/go.mod h1:HSz+uSET+XFnRR8LxR5pz3Of3rY3CfYBVs4xY44aLks=
+golang.org/x/net v0.0.0-20190613194153-d28f0bde5980/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=
 golang.org/x/net v0.0.0-20190620200207-3b0461eec859/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=
 golang.org/x/net v0.0.0-20190628185345-da137c7871d7/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=
 golang.org/x/net v0.0.0-20190724013045-ca1201d0de80/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=
+golang.org/x/net v0.0.0-20190813141303-74dc4d7220e7/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=
+golang.org/x/net v0.0.0-20190827160401-ba9fcec4b297/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=
 golang.org/x/net v0.0.0-20191004110552-13f9640d40b9/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=
 golang.org/x/net v0.0.0-20191209160850-c0dbc17a3553/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=
 golang.org/x/net v0.0.0-20200114155413-6afb5195e5aa/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=
@@ -343,7 +644,10 @@ golang.org/x/net v0.0.0-20200707034311-ab3426394381/go.mod h1:/O7V0waA8r7cgGh81R
 golang.org/x/net v0.0.0-20200822124328-c89045814202/go.mod h1:/O7V0waA8r7cgGh81Ro3o1hOxt32SMVPicZroKQ2sZA=
 golang.org/x/net v0.0.0-20201021035429-f5854403a974/go.mod h1:sp8m0HH+o8qH0wwXwYZr8TS3Oi6o0r6Gce1SSxlDquU=
 golang.org/x/net v0.0.0-20201031054903-ff519b6c9102/go.mod h1:sp8m0HH+o8qH0wwXwYZr8TS3Oi6o0r6Gce1SSxlDquU=
+golang.org/x/net v0.0.0-20201110031124-69a78807bb2b/go.mod h1:sp8m0HH+o8qH0wwXwYZr8TS3Oi6o0r6Gce1SSxlDquU=
+golang.org/x/net v0.0.0-20201209123823-ac852fbbde11/go.mod h1:m0MpNAwzfU5UDzcl9v0D8zg8gWTRqZa9RBIspLL5mdg=
 golang.org/x/net v0.0.0-20201224014010-6772e930b67b/go.mod h1:m0MpNAwzfU5UDzcl9v0D8zg8gWTRqZa9RBIspLL5mdg=
+golang.org/x/net v0.0.0-20210119194325-5f4716e94777/go.mod h1:m0MpNAwzfU5UDzcl9v0D8zg8gWTRqZa9RBIspLL5mdg=
 golang.org/x/net v0.0.0-20210226172049-e18ecbb05110/go.mod h1:m0MpNAwzfU5UDzcl9v0D8zg8gWTRqZa9RBIspLL5mdg=
 golang.org/x/net v0.0.0-20210423184538-5f58ad60dda6 h1:0PC75Fz/kyMGhL0e1QnypqK2kQMqKt9csD1GnMJR+Zk=
 golang.org/x/net v0.0.0-20210423184538-5f58ad60dda6/go.mod h1:OJAsFXCWl8Ukc7SiCT/9KSuxbyM7479/AVlXFRxuMCk=
@@ -355,6 +659,10 @@ golang.org/x/oauth2 v0.0.0-20200107190931-bf48bf16ab8d/go.mod h1:gOpvHmFTYa4Iltr
 golang.org/x/oauth2 v0.0.0-20200902213428-5d25da1a8d43/go.mod h1:KelEdhl1UZF7XfJ4dDtk6s++YSgaE7mD/BuKKDLBl4A=
 golang.org/x/oauth2 v0.0.0-20201109201403-9fd604954f58/go.mod h1:KelEdhl1UZF7XfJ4dDtk6s++YSgaE7mD/BuKKDLBl4A=
 golang.org/x/oauth2 v0.0.0-20201208152858-08078c50e5b5/go.mod h1:KelEdhl1UZF7XfJ4dDtk6s++YSgaE7mD/BuKKDLBl4A=
+golang.org/x/oauth2 v0.0.0-20210218202405-ba52d332ba99/go.mod h1:KelEdhl1UZF7XfJ4dDtk6s++YSgaE7mD/BuKKDLBl4A=
+golang.org/x/oauth2 v0.0.0-20210220000619-9bb904979d93/go.mod h1:KelEdhl1UZF7XfJ4dDtk6s++YSgaE7mD/BuKKDLBl4A=
+golang.org/x/oauth2 v0.0.0-20210313182246-cd4f82c27b84/go.mod h1:KelEdhl1UZF7XfJ4dDtk6s++YSgaE7mD/BuKKDLBl4A=
+golang.org/x/oauth2 v0.0.0-20210402161424-2e8d93401602/go.mod h1:KelEdhl1UZF7XfJ4dDtk6s++YSgaE7mD/BuKKDLBl4A=
 golang.org/x/sync v0.0.0-20180314180146-1d60e4601c6f/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
 golang.org/x/sync v0.0.0-20181108010431-42b317875d0f/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
 golang.org/x/sync v0.0.0-20181221193216-37e7f081c4d4/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
@@ -364,10 +672,14 @@ golang.org/x/sync v0.0.0-20190911185100-cd5d95a43a6e/go.mod h1:RxMgew5VJxzue5/jJ
 golang.org/x/sync v0.0.0-20200317015054-43a5402ce75a/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
 golang.org/x/sync v0.0.0-20200625203802-6e8e738ad208/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
 golang.org/x/sync v0.0.0-20201020160332-67f06af15bc9/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
+golang.org/x/sync v0.0.0-20201207232520-09787c993a3a/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
+golang.org/x/sync v0.0.0-20210220032951-036812b2e83c/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
 golang.org/x/sys v0.0.0-20170830134202-bb24a47a89ea/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
 golang.org/x/sys v0.0.0-20180830151530-49385e6e1522/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
 golang.org/x/sys v0.0.0-20180905080454-ebe1bf3edb33/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
 golang.org/x/sys v0.0.0-20180909124046-d0be0721c37e/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
+golang.org/x/sys v0.0.0-20181107165924-66b7b1311ac8/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
+golang.org/x/sys v0.0.0-20181116152217-5ac8a444bdc5/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
 golang.org/x/sys v0.0.0-20190209173611-3b5209105503/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
 golang.org/x/sys v0.0.0-20190215142949-d0b11bdaac8a/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
 golang.org/x/sys v0.0.0-20190312061237-fead79001313/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
@@ -376,17 +688,25 @@ golang.org/x/sys v0.0.0-20190422165155-953cdadca894/go.mod h1:h1NjWce9XRLGQEsW7w
 golang.org/x/sys v0.0.0-20190502145724-3ef323f4f1fd/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
 golang.org/x/sys v0.0.0-20190507160741-ecd444e8653b/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
 golang.org/x/sys v0.0.0-20190606165138-5da285871e9c/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
+golang.org/x/sys v0.0.0-20190616124812-15dcb6c0061f/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
 golang.org/x/sys v0.0.0-20190624142023-c5567b49c5d0/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
 golang.org/x/sys v0.0.0-20190726091711-fc99dfbffb4e/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
+golang.org/x/sys v0.0.0-20190801041406-cbf593c0f2f3/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
+golang.org/x/sys v0.0.0-20190812073006-9eafafc0a87e/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
 golang.org/x/sys v0.0.0-20190813064441-fde4db37ae7a/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
+golang.org/x/sys v0.0.0-20190826190057-c7b8b68b1456/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
+golang.org/x/sys v0.0.0-20190904154756-749cb33beabd/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
 golang.org/x/sys v0.0.0-20190916202348-b4ddaad3f8a3/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
 golang.org/x/sys v0.0.0-20191001151750-bb3f8db39f24/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
+golang.org/x/sys v0.0.0-20191005200804-aed5e4c7ecf9/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
 golang.org/x/sys v0.0.0-20191022100944-742c48ecaeb7/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
 golang.org/x/sys v0.0.0-20191026070338-33540a1f6037/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
+golang.org/x/sys v0.0.0-20191115151921-52ab43148777/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
 golang.org/x/sys v0.0.0-20191120155948-bd437916bb0e/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
 golang.org/x/sys v0.0.0-20191204072324-ce4227a45e2e/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
 golang.org/x/sys v0.0.0-20191210023423-ac6580df4449/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
 golang.org/x/sys v0.0.0-20191228213918-04cbcbbfeed8/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
+golang.org/x/sys v0.0.0-20200106162015-b016eb3dc98e/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
 golang.org/x/sys v0.0.0-20200113162924-86b910548bc1/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
 golang.org/x/sys v0.0.0-20200120151820-655fe14d7479/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
 golang.org/x/sys v0.0.0-20200122134326-e047566fdf82/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
@@ -402,14 +722,23 @@ golang.org/x/sys v0.0.0-20200501052902-10377860bb8e/go.mod h1:h1NjWce9XRLGQEsW7w
 golang.org/x/sys v0.0.0-20200511232937-7e40ca221e25/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
 golang.org/x/sys v0.0.0-20200515095857-1151b9dac4a9/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
 golang.org/x/sys v0.0.0-20200523222454-059865788121/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
+golang.org/x/sys v0.0.0-20200615200032-f1bc736245b1/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
 golang.org/x/sys v0.0.0-20200803210538-64077c9b5642/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
 golang.org/x/sys v0.0.0-20200905004654-be1d3432aa8f/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
+golang.org/x/sys v0.0.0-20200909081042-eff7692f9009/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
 golang.org/x/sys v0.0.0-20200916030750-2334cc1a136f/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
+golang.org/x/sys v0.0.0-20200922070232-aee5d888a860/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
 golang.org/x/sys v0.0.0-20200930185726-fdedc70b468f/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
+golang.org/x/sys v0.0.0-20201112073958-5cba982894dd/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
 golang.org/x/sys v0.0.0-20201119102817-f84b799fce68/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
 golang.org/x/sys v0.0.0-20201201145000-ef89a241ccb3/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
+golang.org/x/sys v0.0.0-20210104204734-6f8348627aad/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
 golang.org/x/sys v0.0.0-20210119212857-b64e53b001e4/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
+golang.org/x/sys v0.0.0-20210124154548-22da62e12c0c/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
+golang.org/x/sys v0.0.0-20210220050731-9a76102bfb43/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
+golang.org/x/sys v0.0.0-20210305230114-8fe3ee5dd75b/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
 golang.org/x/sys v0.0.0-20210309040221-94ec62e08169/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
+golang.org/x/sys v0.0.0-20210314195730-07df6a141424/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
 golang.org/x/sys v0.0.0-20210423082822-04245dca01da/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
 golang.org/x/sys v0.0.0-20210423185535-09eb48e85fd7 h1:iGu644GcxtEcrInvDsQRCwJjtCIOlT2V7IRt6ah2Whw=
 golang.org/x/sys v0.0.0-20210423185535-09eb48e85fd7/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
@@ -422,12 +751,15 @@ golang.org/x/text v0.3.1-0.20180807135948-17ff2d5776d2/go.mod h1:NqM8EUOU14njkJ3
 golang.org/x/text v0.3.2/go.mod h1:bEr9sfX3Q8Zfm5fL9x+3itogRgK3+ptLWKqgva+5dAk=
 golang.org/x/text v0.3.3/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ=
 golang.org/x/text v0.3.4/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ=
+golang.org/x/text v0.3.5/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ=
 golang.org/x/text v0.3.6/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ=
+golang.org/x/time v0.0.0-20180412165947-fbb02b2291d2/go.mod h1:tRJNPiyCQ0inRvYxbN9jk5I+vvW/OXSQhTDSoE431IQ=
 golang.org/x/time v0.0.0-20181108054448-85acf8d2951c/go.mod h1:tRJNPiyCQ0inRvYxbN9jk5I+vvW/OXSQhTDSoE431IQ=
 golang.org/x/time v0.0.0-20190308202827-9d24e82272b4/go.mod h1:tRJNPiyCQ0inRvYxbN9jk5I+vvW/OXSQhTDSoE431IQ=
 golang.org/x/time v0.0.0-20191024005414-555d28b269f0/go.mod h1:tRJNPiyCQ0inRvYxbN9jk5I+vvW/OXSQhTDSoE431IQ=
 golang.org/x/time v0.0.0-20210220033141-f8bda1e9f3ba h1:O8mE0/t419eoIwhTFpKVkHiTs/Igowgfkj25AcZrtiE=
 golang.org/x/time v0.0.0-20210220033141-f8bda1e9f3ba/go.mod h1:tRJNPiyCQ0inRvYxbN9jk5I+vvW/OXSQhTDSoE431IQ=
+golang.org/x/tools v0.0.0-20180221164845-07fd8470d635/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ=
 golang.org/x/tools v0.0.0-20180917221912-90fa682c2a6e/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ=
 golang.org/x/tools v0.0.0-20181011042414-1f849cf54d09/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ=
 golang.org/x/tools v0.0.0-20181030221726-6c7e314b6563/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ=
@@ -436,11 +768,13 @@ golang.org/x/tools v0.0.0-20190226205152-f727befe758c/go.mod h1:9Yl7xja0Znq3iFh3
 golang.org/x/tools v0.0.0-20190311212946-11955173bddd/go.mod h1:LCzVGOaR6xXOjkQ3onu1FJEFr0SW1gC7cKk1uF8kGRs=
 golang.org/x/tools v0.0.0-20190312151545-0bb0c0a6e846/go.mod h1:LCzVGOaR6xXOjkQ3onu1FJEFr0SW1gC7cKk1uF8kGRs=
 golang.org/x/tools v0.0.0-20190312170243-e65039ee4138/go.mod h1:LCzVGOaR6xXOjkQ3onu1FJEFr0SW1gC7cKk1uF8kGRs=
+golang.org/x/tools v0.0.0-20190328211700-ab21143f2384/go.mod h1:LCzVGOaR6xXOjkQ3onu1FJEFr0SW1gC7cKk1uF8kGRs=
 golang.org/x/tools v0.0.0-20190425150028-36563e24a262/go.mod h1:RgjU9mgBXZiqYHBnxXauZ1Gv1EHHAz9KjViQ78xBX0Q=
 golang.org/x/tools v0.0.0-20190506145303-2d16b83fe98c/go.mod h1:RgjU9mgBXZiqYHBnxXauZ1Gv1EHHAz9KjViQ78xBX0Q=
 golang.org/x/tools v0.0.0-20190524140312-2c0ae7006135/go.mod h1:RgjU9mgBXZiqYHBnxXauZ1Gv1EHHAz9KjViQ78xBX0Q=
 golang.org/x/tools v0.0.0-20190606124116-d0a3d012864b/go.mod h1:/rFqwRUd4F7ZHNgwSSTFct+R/Kf4OFW1sUzUTQQTgfc=
 golang.org/x/tools v0.0.0-20190621195816-6e04913cbbac/go.mod h1:/rFqwRUd4F7ZHNgwSSTFct+R/Kf4OFW1sUzUTQQTgfc=
+golang.org/x/tools v0.0.0-20190624222133-a101b041ded4/go.mod h1:/rFqwRUd4F7ZHNgwSSTFct+R/Kf4OFW1sUzUTQQTgfc=
 golang.org/x/tools v0.0.0-20190628153133-6cdbf07be9d0/go.mod h1:/rFqwRUd4F7ZHNgwSSTFct+R/Kf4OFW1sUzUTQQTgfc=
 golang.org/x/tools v0.0.0-20190816200558-6889da9d5479/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo=
 golang.org/x/tools v0.0.0-20190911174233-4f2ddba30aff/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo=
@@ -467,12 +801,16 @@ golang.org/x/tools v0.0.0-20200501065659-ab2804fb9c9d/go.mod h1:EkVYQZoAsY45+roY
 golang.org/x/tools v0.0.0-20200512131952-2bc93b1c0c88/go.mod h1:EkVYQZoAsY45+roYkvgYkIh4xh/qjgUK9TdY2XT94GE=
 golang.org/x/tools v0.0.0-20200515010526-7d3b6ebf133d/go.mod h1:EkVYQZoAsY45+roYkvgYkIh4xh/qjgUK9TdY2XT94GE=
 golang.org/x/tools v0.0.0-20200618134242-20370b0cb4b2/go.mod h1:EkVYQZoAsY45+roYkvgYkIh4xh/qjgUK9TdY2XT94GE=
+golang.org/x/tools v0.0.0-20200619180055-7c47624df98f/go.mod h1:EkVYQZoAsY45+roYkvgYkIh4xh/qjgUK9TdY2XT94GE=
 golang.org/x/tools v0.0.0-20200729194436-6467de6f59a7/go.mod h1:njjCfa9FT2d7l9Bc6FUM5FLjQPp3cFF28FI3qnDFljA=
 golang.org/x/tools v0.0.0-20200804011535-6c149bb5ef0d/go.mod h1:njjCfa9FT2d7l9Bc6FUM5FLjQPp3cFF28FI3qnDFljA=
 golang.org/x/tools v0.0.0-20200825202427-b303f430e36d/go.mod h1:njjCfa9FT2d7l9Bc6FUM5FLjQPp3cFF28FI3qnDFljA=
 golang.org/x/tools v0.0.0-20200904185747-39188db58858/go.mod h1:Cj7w3i3Rnn0Xh82ur9kSqwfTHTeVxaDqrfMjpcNT6bE=
 golang.org/x/tools v0.0.0-20201110124207-079ba7bd75cd/go.mod h1:emZCQorbCU4vsT4fOWvOPXz4eW1wZW4PmDk9uLelYpA=
 golang.org/x/tools v0.0.0-20201201161351-ac6f37ff4c2a/go.mod h1:emZCQorbCU4vsT4fOWvOPXz4eW1wZW4PmDk9uLelYpA=
+golang.org/x/tools v0.0.0-20201208233053-a543418bbed2/go.mod h1:emZCQorbCU4vsT4fOWvOPXz4eW1wZW4PmDk9uLelYpA=
+golang.org/x/tools v0.0.0-20210105154028-b0ab187a4818/go.mod h1:emZCQorbCU4vsT4fOWvOPXz4eW1wZW4PmDk9uLelYpA=
+golang.org/x/tools v0.0.0-20210106214847-113979e3529a/go.mod h1:emZCQorbCU4vsT4fOWvOPXz4eW1wZW4PmDk9uLelYpA=
 golang.org/x/tools v0.0.0-20210108195828-e2f9c7f1fc8e/go.mod h1:emZCQorbCU4vsT4fOWvOPXz4eW1wZW4PmDk9uLelYpA=
 golang.org/x/tools v0.1.0/go.mod h1:xkSsbof2nBLbhDlRMhhhyNLN/zl3eTqcnHD5viDpcZ0=
 golang.org/x/xerrors v0.0.0-20190717185122-a985d3407aa7/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
@@ -499,6 +837,9 @@ google.golang.org/api v0.29.0/go.mod h1:Lcubydp8VUV7KeIHD9z2Bys/sm/vGKnG1UHuDBSr
 google.golang.org/api v0.30.0/go.mod h1:QGmEvQ87FHZNiUVJkT14jQNYJ4ZJjdRF23ZXz5138Fc=
 google.golang.org/api v0.35.0/go.mod h1:/XrVsuzM0rZmrsbjJutiuftIzeuTQcEeaYcSk/mQ1dg=
 google.golang.org/api v0.36.0/go.mod h1:+z5ficQTmoYpPn8LCUNVpK5I7hwkpjbcgqA7I34qYtE=
+google.golang.org/api v0.40.0/go.mod h1:fYKFpnQN0DsDSKRVRcQSDQNtqWPfM9i+zNPxepjRCQ8=
+google.golang.org/api v0.41.0/go.mod h1:RkxM5lITDfTzmyKFPt+wGrCJbVfniCr2ool8kTBzRTU=
+google.golang.org/api v0.42.0/go.mod h1:+Oj4s6ch2SEGtPjGqfUfZonBH0GjQH89gTeKKAEGZKI=
 google.golang.org/appengine v1.1.0/go.mod h1:EbEs0AVv82hx2wNQdGPgUI5lhzA/G0D9YwlJXL52JkM=
 google.golang.org/appengine v1.4.0/go.mod h1:xpcJRLb0r/rnEns0DIKYYv+WjYCduHsrkT7/EB5XEv4=
 google.golang.org/appengine v1.5.0/go.mod h1:xpcJRLb0r/rnEns0DIKYYv+WjYCduHsrkT7/EB5XEv4=
@@ -506,6 +847,7 @@ google.golang.org/appengine v1.6.1/go.mod h1:i06prIuMbXzDqacNJfV5OdTW448YApPu5ww
 google.golang.org/appengine v1.6.5/go.mod h1:8WjMMxjGQR8xUklV/ARdw2HLXBOI7O7uCIDZVag1xfc=
 google.golang.org/appengine v1.6.6/go.mod h1:8WjMMxjGQR8xUklV/ARdw2HLXBOI7O7uCIDZVag1xfc=
 google.golang.org/appengine v1.6.7/go.mod h1:8WjMMxjGQR8xUklV/ARdw2HLXBOI7O7uCIDZVag1xfc=
+google.golang.org/cloud v0.0.0-20151119220103-975617b05ea8/go.mod h1:0H1ncTHf11KCFhTc/+EFRbzSCOZx+VUbRMk55Yv5MYk=
 google.golang.org/genproto v0.0.0-20180817151627-c66870c02cf8/go.mod h1:JiN7NxoALGmiZfu7CAH4rXhgtRTLTxftemlI0sWmxmc=
 google.golang.org/genproto v0.0.0-20190307195333-5fe7a883aa19/go.mod h1:VzzqZJRnGkLBvHegQrXjBqPurQTc5/KpmUdxsrq26oE=
 google.golang.org/genproto v0.0.0-20190418145605-e7d98fc518a7/go.mod h1:VzzqZJRnGkLBvHegQrXjBqPurQTc5/KpmUdxsrq26oE=
@@ -530,6 +872,7 @@ google.golang.org/genproto v0.0.0-20200312145019-da6875a35672/go.mod h1:55QSHmfG
 google.golang.org/genproto v0.0.0-20200331122359-1ee6d9798940/go.mod h1:55QSHmfGQM9UVYDPBsyGGes0y52j32PQ3BqQfXhyH3c=
 google.golang.org/genproto v0.0.0-20200430143042-b979b6f78d84/go.mod h1:55QSHmfGQM9UVYDPBsyGGes0y52j32PQ3BqQfXhyH3c=
 google.golang.org/genproto v0.0.0-20200511104702-f5ebc3bea380/go.mod h1:55QSHmfGQM9UVYDPBsyGGes0y52j32PQ3BqQfXhyH3c=
+google.golang.org/genproto v0.0.0-20200513103714-09dca8ec2884/go.mod h1:55QSHmfGQM9UVYDPBsyGGes0y52j32PQ3BqQfXhyH3c=
 google.golang.org/genproto v0.0.0-20200515170657-fc4c6c6a6587/go.mod h1:YsZOwe1myG/8QRHRsmBRE1LrgQY60beZKjly0O1fX9U=
 google.golang.org/genproto v0.0.0-20200526211855-cb27e3aa2013/go.mod h1:NbSheEEYHJ7i3ixzK3sjbqSGDJWnxyFXZblF3eUsNvo=
 google.golang.org/genproto v0.0.0-20200618031413-b414f8b61790/go.mod h1:jDfRM7FcilCzHH/e9qn6dsT145K34l5v+OpcnNgKAAA=
@@ -538,13 +881,22 @@ google.golang.org/genproto v0.0.0-20200804131852-c06518451d9c/go.mod h1:FWY/as6D
 google.golang.org/genproto v0.0.0-20200825200019-8632dd797987/go.mod h1:FWY/as6DDZQgahTzZj3fqbO1CbirC29ZNUFHwi0/+no=
 google.golang.org/genproto v0.0.0-20200904004341-0bd0a958aa1d/go.mod h1:FWY/as6DDZQgahTzZj3fqbO1CbirC29ZNUFHwi0/+no=
 google.golang.org/genproto v0.0.0-20201109203340-2640f1f9cdfb/go.mod h1:FWY/as6DDZQgahTzZj3fqbO1CbirC29ZNUFHwi0/+no=
+google.golang.org/genproto v0.0.0-20201110150050-8816d57aaa9a/go.mod h1:FWY/as6DDZQgahTzZj3fqbO1CbirC29ZNUFHwi0/+no=
 google.golang.org/genproto v0.0.0-20201201144952-b05cb90ed32e/go.mod h1:FWY/as6DDZQgahTzZj3fqbO1CbirC29ZNUFHwi0/+no=
+google.golang.org/genproto v0.0.0-20201210142538-e3217bee35cc/go.mod h1:FWY/as6DDZQgahTzZj3fqbO1CbirC29ZNUFHwi0/+no=
+google.golang.org/genproto v0.0.0-20201214200347-8c77b98c765d/go.mod h1:FWY/as6DDZQgahTzZj3fqbO1CbirC29ZNUFHwi0/+no=
 google.golang.org/genproto v0.0.0-20210108203827-ffc7fda8c3d7/go.mod h1:FWY/as6DDZQgahTzZj3fqbO1CbirC29ZNUFHwi0/+no=
+google.golang.org/genproto v0.0.0-20210222152913-aa3ee6e6a81c/go.mod h1:FWY/as6DDZQgahTzZj3fqbO1CbirC29ZNUFHwi0/+no=
+google.golang.org/genproto v0.0.0-20210303154014-9728d6b83eeb/go.mod h1:FWY/as6DDZQgahTzZj3fqbO1CbirC29ZNUFHwi0/+no=
+google.golang.org/genproto v0.0.0-20210310155132-4ce2db91004e/go.mod h1:FWY/as6DDZQgahTzZj3fqbO1CbirC29ZNUFHwi0/+no=
+google.golang.org/genproto v0.0.0-20210312152112-fc591d9ea70f/go.mod h1:FWY/as6DDZQgahTzZj3fqbO1CbirC29ZNUFHwi0/+no=
 google.golang.org/grpc v1.19.0/go.mod h1:mqu4LbDTu4XGKhr4mRzUsmM4RtVoemTSY81AxZiDr8c=
 google.golang.org/grpc v1.20.1/go.mod h1:10oTOabMzJvdu6/UiuZezV6QK5dSlG84ov/aaiqXj38=
+google.golang.org/grpc v1.21.0/go.mod h1:oYelfM1adQP15Ek0mdvEgi9Df8B9CZIaU1084ijfRaM=
 google.golang.org/grpc v1.21.1/go.mod h1:oYelfM1adQP15Ek0mdvEgi9Df8B9CZIaU1084ijfRaM=
 google.golang.org/grpc v1.23.0/go.mod h1:Y5yQAOtifL1yxbo5wqy6BxZv8vAUGQwXBOALyacEbxg=
 google.golang.org/grpc v1.23.1/go.mod h1:Y5yQAOtifL1yxbo5wqy6BxZv8vAUGQwXBOALyacEbxg=
+google.golang.org/grpc v1.24.0/go.mod h1:XDChyiUovWa60DnaeDeZmSW86xtLtjtZbwvSiRnRtcA=
 google.golang.org/grpc v1.25.1/go.mod h1:c3i+UQWmh7LiEpx4sFZnkU36qjEYZ0imhYfXVyQciAY=
 google.golang.org/grpc v1.26.0/go.mod h1:qbnxyOmOxrQa7FizSgH+ReBfzJrCY1pSN7KXBS8abTk=
 google.golang.org/grpc v1.27.0/go.mod h1:qbnxyOmOxrQa7FizSgH+ReBfzJrCY1pSN7KXBS8abTk=
@@ -554,9 +906,13 @@ google.golang.org/grpc v1.29.1/go.mod h1:itym6AZVZYACWQqET3MqgPpjcuV5QH3BxFS3Iji
 google.golang.org/grpc v1.30.0/go.mod h1:N36X2cJ7JwdamYAgDz+s+rVMFjt3numwzf/HckM8pak=
 google.golang.org/grpc v1.31.0/go.mod h1:N36X2cJ7JwdamYAgDz+s+rVMFjt3numwzf/HckM8pak=
 google.golang.org/grpc v1.31.1/go.mod h1:N36X2cJ7JwdamYAgDz+s+rVMFjt3numwzf/HckM8pak=
+google.golang.org/grpc v1.33.1/go.mod h1:fr5YgcSWrqhRRxogOsw7RzIpsmvOZ6IcH4kBYTpR3n0=
 google.golang.org/grpc v1.33.2/go.mod h1:JMHMWHQWaTccqQQlmk3MJZS+GWXOdAesneDmEnv2fbc=
 google.golang.org/grpc v1.34.0/go.mod h1:WotjhfgOW/POjDeRt8vscBtXq+2VjORFy659qA51WJ8=
+google.golang.org/grpc v1.35.0/go.mod h1:qjiiYl8FncCW8feJPdyg3v6XW24KsRHe+dy9BAGRRjU=
 google.golang.org/grpc v1.36.0-dev.0.20210208035533-9280052d3665/go.mod h1:qjiiYl8FncCW8feJPdyg3v6XW24KsRHe+dy9BAGRRjU=
+google.golang.org/grpc v1.36.0/go.mod h1:qjiiYl8FncCW8feJPdyg3v6XW24KsRHe+dy9BAGRRjU=
+google.golang.org/grpc v1.39.0-dev.0.20210518002758-2713b77e8526/go.mod h1:PImNr+rS9TWYb2O4/emRugxiyHZ5JyHW5F+RPnDzfrE=
 google.golang.org/protobuf v0.0.0-20200109180630-ec00e32a8dfd/go.mod h1:DFci5gLYBciE7Vtevhsrf46CRTquxDuWsQurQQe4oz8=
 google.golang.org/protobuf v0.0.0-20200221191635-4d8936d0db64/go.mod h1:kwYJMbMJ01Woi6D6+Kah6886xMZcty6N08ah7+eCXa0=
 google.golang.org/protobuf v0.0.0-20200228230310-ab0ca4ff8a60/go.mod h1:cfTl7dwQJ+fmap5saPgwCLgHXTUD7jkjRqWcaiX5VyM=
@@ -568,18 +924,38 @@ google.golang.org/protobuf v1.23.1-0.20200526195155-81db48ad09cc/go.mod h1:EGpAD
 google.golang.org/protobuf v1.24.0/go.mod h1:r/3tXBNzIEhYS9I1OUVjXDlt8tc493IdKGjtUeSXeh4=
 google.golang.org/protobuf v1.25.0/go.mod h1:9JNX74DMeImyA3h4bdi1ymwjUzf21/xIlbajtzgsN7c=
 google.golang.org/protobuf v1.25.1-0.20201020201750-d3470999428b/go.mod h1:hFxJC2f0epmp1elRCiEGJTKAWbwxZ2nvqZdHl3FQXCY=
+google.golang.org/protobuf v1.26.0-rc.1/go.mod h1:jlhhOSvTdKEhbULTjvd4ARK9grFBp09yW+WbY/TyQbw=
+google.golang.org/protobuf v1.26.0/go.mod h1:9q0QmTI4eRPtz6boOQmLYwt+qCgq0jsYwAQnmE0givc=
+gopkg.in/airbrake/gobrake.v2 v2.0.9/go.mod h1:/h5ZAUhDkGaJfjzjKLSjv6zCL6O0LLBxU4K+aSYdM/U=
+gopkg.in/alecthomas/kingpin.v2 v2.2.6/go.mod h1:FMv+mEhP44yOT+4EoQTLFTRgOQ1FBLkstjWtayDeSgw=
 gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
 gopkg.in/check.v1 v1.0.0-20180628173108-788fd7840127/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
+gopkg.in/check.v1 v1.0.0-20190902080502-41f04d3bba15/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
+gopkg.in/cheggaaa/pb.v1 v1.0.25/go.mod h1:V/YB90LKu/1FcN3WVnfiiE5oMCibMjukxqG/qStrOgw=
 gopkg.in/errgo.v2 v2.1.0/go.mod h1:hNsd1EY+bozCKY1Ytp96fpM3vjJbqLJn88ws8XvfDNI=
 gopkg.in/fsnotify.v1 v1.4.7/go.mod h1:Tz8NjZHkW78fSQdbUxIjBTcgA1z1m8ZHf0WmKUhAMys=
+gopkg.in/gemnasium/logrus-airbrake-hook.v2 v2.1.2/go.mod h1:Xk6kEKp8OKb+X14hQBKWaSkCsqBpgog8nAV2xsGOxlo=
 gopkg.in/inf.v0 v0.9.1/go.mod h1:cWUDdTG/fYaXco+Dcufb5Vnc6Gp2YChqWtbxRZE0mXw=
+gopkg.in/natefinch/lumberjack.v2 v2.0.0/go.mod h1:l0ndWWf7gzL7RNwBG7wST/UCcT4T24xpD6X8LsfU/+k=
+gopkg.in/resty.v1 v1.12.0/go.mod h1:mDo4pnntr5jdWRML875a/NmxYqAlA73dVijT2AXvQQo=
+gopkg.in/square/go-jose.v2 v2.3.1/go.mod h1:M9dMgbHiYLoDGQrXy7OpJDJWiKiU//h+vD76mk0e1AI=
+gopkg.in/square/go-jose.v2 v2.5.1/go.mod h1:M9dMgbHiYLoDGQrXy7OpJDJWiKiU//h+vD76mk0e1AI=
 gopkg.in/tomb.v1 v1.0.0-20141024135613-dd632973f1e7/go.mod h1:dt/ZhP58zS4L8KSrWDmTeBkI65Dw0HsyUHuEVlX15mw=
+gopkg.in/yaml.v2 v2.0.0-20170812160011-eb3733d160e7/go.mod h1:JAlM8MvJe8wmxCU4Bli9HhUf9+ttbYbLASfIpnQbh74=
 gopkg.in/yaml.v2 v2.2.1/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI=
 gopkg.in/yaml.v2 v2.2.2/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI=
+gopkg.in/yaml.v2 v2.2.3/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI=
+gopkg.in/yaml.v2 v2.2.4/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI=
+gopkg.in/yaml.v2 v2.2.5/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI=
 gopkg.in/yaml.v2 v2.2.8/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI=
+gopkg.in/yaml.v3 v3.0.0-20200313102051-9f266ea9e77c/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
 gotest.tools v2.2.0+incompatible/go.mod h1:DsYFclhRJ6vuDpmuTbkuFWG+y2sxOXAzmJt81HFBacw=
+gotest.tools/v3 v3.0.2/go.mod h1:3SzNCllyD9/Y+b5r9JIKQ474KzkZyqLqEfYqMsX94Bk=
+gotest.tools/v3 v3.0.3/go.mod h1:Z7Lb0S5l+klDB31fvDQX8ss/FlKDxtlFlw3Oa8Ymbl8=
 gvisor.dev/gvisor v0.0.0-20210506004418-fbfeba3024f0 h1:Ny53d6x76f7EgvYe7pi8SQcIzdRM69y1ckQ8rRtT6ww=
 gvisor.dev/gvisor v0.0.0-20210506004418-fbfeba3024f0/go.mod h1:ucHEMlckp+S/YzKEpwwAyGBhAh807Wxq/8Erc6gFxCE=
+gvisor.dev/gvisor v0.0.0-20211020211948-f76a604701b6 h1:lgV5mAyX6S2EZAkZcvFkAs18WZ7yJbRzEv/PCH8iSlw=
+gvisor.dev/gvisor v0.0.0-20211020211948-f76a604701b6/go.mod h1:m1RK/gef4nU1CWOFscQWVk7iUgGH2Hz9Ee+lgeCzOBo=
 honnef.co/go/tools v0.0.0-20190102054323-c2f93a96b099/go.mod h1:rf3lG4BRIbNafJWhAfAdb/ePZxsR/4RtNHQocxwk9r4=
 honnef.co/go/tools v0.0.0-20190106161140-3f1c8253044a/go.mod h1:rf3lG4BRIbNafJWhAfAdb/ePZxsR/4RtNHQocxwk9r4=
 honnef.co/go/tools v0.0.0-20190418001031-e561f6794a2a/go.mod h1:rf3lG4BRIbNafJWhAfAdb/ePZxsR/4RtNHQocxwk9r4=
@@ -592,14 +968,24 @@ k8s.io/api v0.16.13/go.mod h1:QWu8UWSTiuQZMMeYjwLs6ILu5O74qKSJ0c+4vrchDxs=
 k8s.io/apimachinery v0.16.13/go.mod h1:4HMHS3mDHtVttspuuhrJ1GGr/0S9B6iWYWZ57KnnZqQ=
 k8s.io/apimachinery v0.16.14-rc.0/go.mod h1:4HMHS3mDHtVttspuuhrJ1GGr/0S9B6iWYWZ57KnnZqQ=
 k8s.io/client-go v0.16.13/go.mod h1:UKvVT4cajC2iN7DCjLgT0KVY/cbY6DGdUCyRiIfws5M=
+k8s.io/component-base v0.16.13/go.mod h1:cNe9ZU2A6tqBG0gPQ4/T/KolI9Cv2NA1+7uvmkA7Cyc=
+k8s.io/cri-api v0.20.6/go.mod h1:ew44AjNXwyn1s0U4xCKGodU7J1HzBeZ1MpGrpa5r8Yc=
 k8s.io/gengo v0.0.0-20190128074634-0689ccc1d7d6/go.mod h1:ezvh/TsK7cY6rbqRK0oQQ8IAqLxYwwyPxAX1Pzy0ii0=
+k8s.io/gengo v0.0.0-20200413195148-3a45101e95ac/go.mod h1:ezvh/TsK7cY6rbqRK0oQQ8IAqLxYwwyPxAX1Pzy0ii0=
 k8s.io/klog v0.0.0-20181102134211-b9b56d5dfc92/go.mod h1:Gq+BEi5rUBO/HRz0bTSXDUcqjScdoY3a9IHpCEIOOfk=
 k8s.io/klog v0.3.0/go.mod h1:Gq+BEi5rUBO/HRz0bTSXDUcqjScdoY3a9IHpCEIOOfk=
 k8s.io/klog v1.0.0/go.mod h1:4Bi6QPql/J/LkTDqv7R/cd3hPo4k2DG6Ptcz060Ez5I=
+k8s.io/klog/v2 v2.0.0/go.mod h1:PBfzABfn139FHAV07az/IF9Wp1bkk3vpT2XSJ76fSDE=
+k8s.io/klog/v2 v2.4.0/go.mod h1:Od+F08eJP+W3HUb4pSrPpgp9DGU4GzlpG/TmITuYh/Y=
 k8s.io/kube-openapi v0.0.0-20200410163147-594e756bea31/go.mod h1:1TqjTSzOxsLGIKfj0lK8EeCP7K1iUG65v09OM0/WG5E=
+k8s.io/kubernetes v1.13.0/go.mod h1:ocZa8+6APFNC2tX1DZASIbocyYT5jHzqFVsY5aoB7Jk=
 k8s.io/utils v0.0.0-20190801114015-581e00157fb1/go.mod h1:sZAwmy6armz5eXlNoLmJcl4F1QuKu7sr+mFQ0byX7Ew=
+k8s.io/utils v0.0.0-20201110183641-67b214c5f920/go.mod h1:jPW/WVKK9YHAvNhRxK0md/EJ228hCsBRufyofKtW8HA=
 rsc.io/binaryregexp v0.2.0/go.mod h1:qTv7/COck+e2FymRvadv62gMdZztPaShugOCi3I+8D8=
 rsc.io/quote/v3 v3.1.0/go.mod h1:yEA65RcK8LyAZtP9Kv3t0HmxON59tX3rD+tICJqUlj0=
 rsc.io/sampler v1.3.0/go.mod h1:T1hPZKmBbMNahiBKFy5HrXp6adAjACjK9JXDnKaTXpA=
+sigs.k8s.io/apiserver-network-proxy/konnectivity-client v0.0.15/go.mod h1:LEScyzhFmoF5pso/YSeBstl57mOzx9xlU9n85RGrDQg=
 sigs.k8s.io/structured-merge-diff v0.0.0-20190525122527-15d366b2352e/go.mod h1:wWxsB5ozmmv/SG7nM11ayaAW51xMvak/t1r0CSlcokI=
+sigs.k8s.io/structured-merge-diff/v4 v4.0.3/go.mod h1:bJZC9H9iH24zzfZ/41RGcq60oK1F7G282QMXDPYydCw=
 sigs.k8s.io/yaml v1.1.0/go.mod h1:UJmg0vDUVViEyp3mgSv9WPwZCDxu4rQW1olrI1uml+o=
+sigs.k8s.io/yaml v1.2.0/go.mod h1:yfXDCHCao9+ENCvLSE62v9VSji2MKu5jeNfTrofGhJc=
diff --git a/tun/netstack/tun.go b/tun/netstack/tun.go
index 9d2e90a..24d0835 100644
--- a/tun/netstack/tun.go
+++ b/tun/netstack/tun.go
@@ -83,6 +83,10 @@ func (e *endpoint) WritePackets(stack.RouteInfo, stack.PacketBufferList, tcpip.N
 	panic("not implemented")
 }
 
+func (e *endpoint) WriteRawPacket(*stack.PacketBuffer) tcpip.Error {
+	panic("not implemented")
+}
+
 func (*endpoint) ARPHardwareType() header.ARPHardwareType {
 	return header.ARPHardwareNone
 }
@@ -109,15 +113,23 @@ func CreateNetTUN(localAddresses, dnsServers []net.IP, mtu int) (tun.Device, *Ne
 	}
 	for _, ip := range localAddresses {
 		if ip4 := ip.To4(); ip4 != nil {
-			tcpipErr = dev.stack.AddAddress(1, ipv4.ProtocolNumber, tcpip.Address(ip4))
+			protoAddr := tcpip.ProtocolAddress{
+				Protocol:          ipv4.ProtocolNumber,
+				AddressWithPrefix: tcpip.Address(ip4).WithPrefix(),
+			}
+			tcpipErr := dev.stack.AddProtocolAddress(1, protoAddr, stack.AddressProperties{})
 			if tcpipErr != nil {
-				return nil, nil, fmt.Errorf("AddAddress(%v): %v", ip4, tcpipErr)
+				return nil, nil, fmt.Errorf("AddProtocolAddress(%v): %v", ip4, tcpipErr)
 			}
 			dev.hasV4 = true
 		} else {
-			tcpipErr = dev.stack.AddAddress(1, ipv6.ProtocolNumber, tcpip.Address(ip))
+			protoAddr := tcpip.ProtocolAddress{
+				Protocol:          ipv6.ProtocolNumber,
+				AddressWithPrefix: tcpip.Address(ip).WithPrefix(),
+			}
+			tcpipErr := dev.stack.AddProtocolAddress(1, protoAddr, stack.AddressProperties{})
 			if tcpipErr != nil {
-				return nil, nil, fmt.Errorf("AddAddress(%v): %v", ip4, tcpipErr)
+				return nil, nil, fmt.Errorf("AddProtocolAddress(%v): %v", ip, tcpipErr)
 			}
 			dev.hasV6 = true
 		}
-- 
2.25.1


From Jason at zx2c4.com  Fri Oct 22 19:24:09 2021
From: Jason at zx2c4.com (Jason A. Donenfeld)
Date: Fri, 22 Oct 2021 13:24:09 -0600
Subject: [PATCH wireguard-go] tun/netstack: update gvisor
In-Reply-To: <20211021220420.168820-1-mikma.wg@lists.m7n.se>
References: <20211021220420.168820-1-mikma.wg@lists.m7n.se>
Message-ID: <CAHmME9rFG2_xRxrqpeMJScFrLKvnCRCw_LtAUwRsuZm4WhHLfA@mail.gmail.com>

Applied, thanks.

From Jason at zx2c4.com  Fri Oct 22 19:26:22 2021
From: Jason at zx2c4.com (Jason A. Donenfeld)
Date: Fri, 22 Oct 2021 13:26:22 -0600
Subject: [PATCH] wireguard-tools: embeddable-wg-library: add named
 wg_endpoint union
In-Reply-To: <b28e05e1-4209-8b82-1b00-e963b2e8fc85@m7n.se>
References: <b28e05e1-4209-8b82-1b00-e963b2e8fc85@m7n.se>
Message-ID: <CAHmME9qeKoZfp0KC8WxJX8-2YNaGTyXXVrf-+Uktr=isKLYh_Q@mail.gmail.com>

Applied, thanks.

From liuhangbin at gmail.com  Mon Oct 25 04:06:38 2021
From: liuhangbin at gmail.com (Hangbin Liu)
Date: Mon, 25 Oct 2021 12:06:38 +0800
Subject: [PATCH net] wireguard: remove peer cache in netns_pre_exit
In-Reply-To: <YS+GX/Y85bch4gMU@zx2c4.com>
References: <20210901122904.9094-1-liuhangbin@gmail.com>
 <YS+GX/Y85bch4gMU@zx2c4.com>
Message-ID: <YXYtTs/04zZ1SU6f@Laptop-X1>

Hi Jason,

Do you have a schedule to post this patch to upstream?

Thanks
Hangbin
On Wed, Sep 01, 2021 at 03:55:43PM +0200, Jason A. Donenfeld wrote:
> 
> From f9984a41eeaebfdcef5aba8a71966b77ba0de8c0 Mon Sep 17 00:00:00 2001
> From: "Jason A. Donenfeld" <Jason at zx2c4.com>
> Date: Wed, 1 Sep 2021 14:53:39 +0200
> Subject: [PATCH] wireguard: device: reset peer src endpoint when netns exits
> MIME-Version: 1.0
> Content-Type: text/plain; charset=UTF-8
> Content-Transfer-Encoding: 8bit
> 
> Each peer's endpoint contains a dst_cache entry that takes a reference
> to another netdev. When the containing namespace exits, we take down the
> socket and prevent future sockets from being created (by setting
> creating_net to NULL), which removes that potential reference on the
> netns. However, it doesn't release references to the netns that a netdev
> cached in dst_cache might be taking, so the netns still might fail to
> exit. Since the socket is gimped anyway, we can simply clear all the
> dst_caches (by way of clearing the endpoint src), which will release all
> references.
> 
> However, the current dst_cache_reset function only releases those
> references lazily. But it turns out that all of our usages of
> wg_socket_clear_peer_endpoint_src are called from contexts that are not
> exactly high-speed or bottle-necked. For example, when there's
> connection difficulty, or when userspace is reconfiguring the interface.
> And in particular for this patch, when the netns is exiting. So for
> those cases, it makes more sense to call dst_release immediately. For
> that, we add a small helper function to dst_cache.
> 
> This patch also adds a test to netns.sh from Hangbin Liu to ensure this
> doesn't regress.
> 
> Test-by: Hangbin Liu <liuhangbin at gmail.com>
> Reported-by: Xiumei Mu <xmu at redhat.com>
> Cc: Hangbin Liu <liuhangbin at gmail.com>
> Cc: Toke H?iland-J?rgensen <toke at redhat.com>
> Cc: Paolo Abeni <pabeni at redhat.com>
> Fixes: 900575aa33a3 ("wireguard: device: avoid circular netns references")
> Signed-off-by: Jason A. Donenfeld <Jason at zx2c4.com>
> ---
>  drivers/net/wireguard/device.c             |  3 +++
>  drivers/net/wireguard/socket.c             |  2 +-
>  include/net/dst_cache.h                    | 11 ++++++++++
>  net/core/dst_cache.c                       | 19 +++++++++++++++++
>  tools/testing/selftests/wireguard/netns.sh | 24 +++++++++++++++++++++-
>  5 files changed, 57 insertions(+), 2 deletions(-)
> 
> diff --git a/drivers/net/wireguard/device.c b/drivers/net/wireguard/device.c
> index 551ddaaaf540..77e64ea6be67 100644
> --- a/drivers/net/wireguard/device.c
> +++ b/drivers/net/wireguard/device.c
> @@ -398,6 +398,7 @@ static struct rtnl_link_ops link_ops __read_mostly = {
>  static void wg_netns_pre_exit(struct net *net)
>  {
>  	struct wg_device *wg;
> +	struct wg_peer *peer;
>  
>  	rtnl_lock();
>  	list_for_each_entry(wg, &device_list, device_list) {
> @@ -407,6 +408,8 @@ static void wg_netns_pre_exit(struct net *net)
>  			mutex_lock(&wg->device_update_lock);
>  			rcu_assign_pointer(wg->creating_net, NULL);
>  			wg_socket_reinit(wg, NULL, NULL);
> +			list_for_each_entry(peer, &wg->peer_list, peer_list)
> +				wg_socket_clear_peer_endpoint_src(peer);
>  			mutex_unlock(&wg->device_update_lock);
>  		}
>  	}
> diff --git a/drivers/net/wireguard/socket.c b/drivers/net/wireguard/socket.c
> index 8c496b747108..6f07b949cb81 100644
> --- a/drivers/net/wireguard/socket.c
> +++ b/drivers/net/wireguard/socket.c
> @@ -308,7 +308,7 @@ void wg_socket_clear_peer_endpoint_src(struct wg_peer *peer)
>  {
>  	write_lock_bh(&peer->endpoint_lock);
>  	memset(&peer->endpoint.src6, 0, sizeof(peer->endpoint.src6));
> -	dst_cache_reset(&peer->endpoint_cache);
> +	dst_cache_reset_now(&peer->endpoint_cache);
>  	write_unlock_bh(&peer->endpoint_lock);
>  }
>  
> diff --git a/include/net/dst_cache.h b/include/net/dst_cache.h
> index 67634675e919..df6622a5fe98 100644
> --- a/include/net/dst_cache.h
> +++ b/include/net/dst_cache.h
> @@ -79,6 +79,17 @@ static inline void dst_cache_reset(struct dst_cache *dst_cache)
>  	dst_cache->reset_ts = jiffies;
>  }
>  
> +/**
> + *	dst_cache_reset_now - invalidate the cache contents immediately
> + *	@dst_cache: the cache
> + *
> + *	The caller must be sure there are no concurrent users, as this frees
> + *	all dst_cache users immediately, rather than waiting for the next
> + *	per-cpu usage like dst_cache_reset does. Most callers should use the
> + *	higher speed lazily-freed dst_cache_reset function instead.
> + */
> +void dst_cache_reset_now(struct dst_cache *dst_cache);
> +
>  /**
>   *	dst_cache_init - initialize the cache, allocating the required storage
>   *	@dst_cache: the cache
> diff --git a/net/core/dst_cache.c b/net/core/dst_cache.c
> index be74ab4551c2..0ccfd5fa5cb9 100644
> --- a/net/core/dst_cache.c
> +++ b/net/core/dst_cache.c
> @@ -162,3 +162,22 @@ void dst_cache_destroy(struct dst_cache *dst_cache)
>  	free_percpu(dst_cache->cache);
>  }
>  EXPORT_SYMBOL_GPL(dst_cache_destroy);
> +
> +void dst_cache_reset_now(struct dst_cache *dst_cache)
> +{
> +	int i;
> +
> +	if (!dst_cache->cache)
> +		return;
> +
> +	dst_cache->reset_ts = jiffies;
> +	for_each_possible_cpu(i) {
> +		struct dst_cache_pcpu *idst = per_cpu_ptr(dst_cache->cache, i);
> +		struct dst_entry *dst = idst->dst;
> +
> +		idst->cookie = 0;
> +		idst->dst = NULL;
> +		dst_release(dst);
> +	}
> +}
> +EXPORT_SYMBOL_GPL(dst_cache_reset_now);
> diff --git a/tools/testing/selftests/wireguard/netns.sh b/tools/testing/selftests/wireguard/netns.sh
> index 2e5c1630885e..8a9461aa0878 100755
> --- a/tools/testing/selftests/wireguard/netns.sh
> +++ b/tools/testing/selftests/wireguard/netns.sh
> @@ -613,6 +613,28 @@ ip0 link set wg0 up
>  kill $ncat_pid
>  ip0 link del wg0
>  
> +# Ensure that dst_cache references don't outlive netns lifetime
> +ip1 link add dev wg0 type wireguard
> +ip2 link add dev wg0 type wireguard
> +configure_peers
> +ip1 link add veth1 type veth peer name veth2
> +ip1 link set veth2 netns $netns2
> +ip1 addr add fd00:aa::1/64 dev veth1
> +ip2 addr add fd00:aa::2/64 dev veth2
> +ip1 link set veth1 up
> +ip2 link set veth2 up
> +waitiface $netns1 veth1
> +waitiface $netns2 veth2
> +ip1 -6 route add default dev veth1 via fd00:aa::2
> +ip2 -6 route add default dev veth2 via fd00:aa::1
> +n1 wg set wg0 peer "$pub2" endpoint [fd00:aa::2]:2
> +n2 wg set wg0 peer "$pub1" endpoint [fd00:aa::1]:1
> +n1 ping6 -c 1 fd00::2
> +pp ip netns delete $netns1
> +pp ip netns delete $netns2
> +pp ip netns add $netns1
> +pp ip netns add $netns2
> +
>  # Ensure there aren't circular reference loops
>  ip1 link add wg1 type wireguard
>  ip2 link add wg2 type wireguard
> @@ -631,7 +653,7 @@ while read -t 0.1 -r line 2>/dev/null || [[ $? -ne 142 ]]; do
>  done < /dev/kmsg
>  alldeleted=1
>  for object in "${!objects[@]}"; do
> -	if [[ ${objects["$object"]} != *createddestroyed ]]; then
> +	if [[ ${objects["$object"]} != *createddestroyed && ${objects["$object"]} != *createdcreateddestroyeddestroyed ]]; then
>  		echo "Error: $object: merely ${objects["$object"]}" >&3
>  		alldeleted=0
>  	fi
> -- 
> 2.32.0

From Jason at zx2c4.com  Mon Oct 25 04:28:03 2021
From: Jason at zx2c4.com (Jason A. Donenfeld)
Date: Mon, 25 Oct 2021 06:28:03 +0200
Subject: [PATCH net] wireguard: remove peer cache in netns_pre_exit
In-Reply-To: <YXYtTs/04zZ1SU6f@Laptop-X1>
References: <20210901122904.9094-1-liuhangbin@gmail.com>
 <YS+GX/Y85bch4gMU@zx2c4.com> <YXYtTs/04zZ1SU6f@Laptop-X1>
Message-ID: <CAHmME9phOhLr0Y5TmuppVn5Zeht0ivwiNJV3S_Q9nzM5nAv7Sw@mail.gmail.com>

I've got a patch set staged to go out this week.

Jason

From dndx at idndx.com  Thu Oct 21 10:08:22 2021
From: dndx at idndx.com (Datong Sun)
Date: Thu, 21 Oct 2021 18:08:22 +0800
Subject: [TOOLS-ANN] Phantun - Run WireGuard over obfuscated TCP connections
 without the penalty of UDP over TCP (alternative to udp2raw)
Message-ID: <CAC=S+V8EsFWv0Y6TfzgiqNok04B87hcTdzk_WZ1mqexMQpBR2w@mail.gmail.com>

Hi everyone,

Apologize in advance if you have already saw my posting in the
WireGuard subreddit. I am posting here again in a hope more people
would find Phantun to be useful for their WireGuard setup and attract
more contributors.

I would like to share a tool that I developed for converting UDP based
connections to fake TCP connections in case UDP is unavailable or
throttled. I have been running the tool with multiple WireGuard setup
for a while and it has been very stable.

Note that I primarily developed Phantun to work with WireGuard in a
UDP restricted environment, but it will work with any other UDP based
protocol as well.

The project is called Phantun. Source code, binary releases and
detailed README are available at: https://github.com/dndx/phantun

In comparison to udp2raw, Phantun was designed to solve some of the
performance issues that I encountered while using udp2raw. In
particular, Phantun is able to utilize multiple CPU cores
simultaneously and have a more predictable MTU overhead, while having
a much leaner codebase and focus only on protocol obfuscation.

Note that this is very different from UDP in TCP which could cause
significant performance penalty because of TCP retransmission and
congestion controls. Phantun simply replaces the UDP header from
WireGuard to TCP header with some sequence number mangling and ACKing
so packets will be regarded by NAT devices and L4 firewalls as valid
packets of a TCP stream. Therefore, all of the desirable properties of
UDP such as or of order delivery are fully preserved. It also means
this protocol will only work between two Phantun instances and will
not work if the other end is a real TCP stack (e.g. when going through
L7 or SOCKS5 proxies).

Please share your feedback and feel free to contribute!

Best regards,
Datong

-- 
Datong Sun
dndx at idndx.com

From dwagner at suse.de  Thu Oct 21 12:37:13 2021
From: dwagner at suse.de (Daniel Wagner)
Date: Thu, 21 Oct 2021 14:37:13 +0200
Subject: Build wireguard-linux-compat on SLES15 (s390x)
In-Reply-To: <CAHmME9oL7UNWKbX3ZF-U_O+a9Kwn9gwsexzDQo9gngjhEdgc8Q@mail.gmail.com>
References: <YWFuczOf1dM3B4Pt@falared> <YW6ljZ3yvGsv9Njq@falared>
 <CAHmME9off=sXrL0hjkGvOgJUtJx6oCvhdAycHT=bTQfUtQjO=w@mail.gmail.com>
 <YW/awIN8jqUF0DPk@falared>
 <CAHmME9oL7UNWKbX3ZF-U_O+a9Kwn9gwsexzDQo9gngjhEdgc8Q@mail.gmail.com>
Message-ID: <20211021123713.a5l6uwfbkx2wxx7j@carbon.lan>

Hi,

On Wed, Oct 20, 2021 at 11:56:49AM -0600, Jason A. Donenfeld wrote:
> Do you think you could chime in here regarding the apparent absence of
> wireguard on s390x? The thread began as a discussion of building it
> yourself there, but I'm wondering why it isn't already there, seeing
> as it's on the other archs.

Sure, let me try to figure out what's happening. I usually try to stay
away from them the distro making business, so it's a bit of a closed
book for me.

> On Wed, Oct 20, 2021 at 3:01 AM Faustin Lammler <faustin at fala.red> wrote:
> >
> > Hi Jason!
> >
> > "Jason A. Donenfeld" <Jason at zx2c4.com>,
> > 19/10/2021 ? 17:51:01 (-0600):
> >
> > > Isn't WireGuard already in the SLES kernel?
> >
> > My understanding is that it should, that's why I was surprised to have
> > to build it. But apparently not.
> >
> > This may be related to a specific kernel from IBM LinuxONE cloud
> > infrastructure?
> >
> > | $ uname -r
> > | 5.3.18-24.86-default

This is SLE15-SP2 and the corresponding branch has the patches and the
s390x config does have CONFIG_WIREGUARD=m. So this looks good.

  https://github.com/openSUSE/kernel-source/tree/SLE15-SP2/patches.suse

grep for bsc#1169021 to see the list.

> > But again, my knowledge of SLES is very limited (and s390x architecture
> > even more) so I don't have any idea why this was needed.
> >
> > I'll be happy to help you if you need me to do some tests on the
> > machine.

I see the module is marked as not supported in supported.conf. After
looking at our spec file, I think the modules is packaged into the
kernel-extra rpm. At it might be that you need to enable the not
supported modules explicitly:

from https://github.com/openSUSE/kernel-source/blob/master/rpm/kernel-binary.spec.in

%if 0%{?sle_version} >= 150000
    # By default, loading unsupported modules is disabled on SLE through
    # /etc/modprobe.d/10-unsupported-modules.conf from the suse-module-tools
    # package.
    # modules in kernel-$flavor-extra don't have the supported flag set,
    # yet loading them should be possible if the package is installed.
    # CAUTION PACKAGERS: The file content below must not change between
    # kernel versions, otherwise file conflicts might arise with
    # multiversion(kernel).

    mkdir -p %buildroot/etc/modprobe.d
    cat >%buildroot/etc/modprobe.d/20-kernel-%{build_flavor}-extra.conf <<EOF
# This file overrides the default from 10-unsupported-modules.conf.
# This is necessary to load kernel modules from the
# kernel-%{build_flavor}-extra package.
#
# WARNING: loading unsupported modules may compromise SLE support.
# Please read the comments in 10-unsupported-modules.conf.
allow_unsupported_modules 1
EOF
    echo "%%dir /etc/modprobe.d" >> %my_builddir/kernel-extra.files
    echo '%%config(noreplace) /etc/modprobe.d/20-kernel-%{build_flavor}-extra.conf' >> %my_builddir/kernel-extra.files
%endif

HTH,
Daniel

From 2rw3n at mailo.com  Fri Oct 22 21:47:00 2021
From: 2rw3n at mailo.com (2rw3n at mailo.com)
Date: Fri, 22 Oct 2021 23:47:00 +0200 (CEST)
Subject: ICMP redirect messages throught wg
Message-ID: <ea-mime-61733154-7941-5ad484b1@www-8.mailo.com>

Dear all, 

 I hope it is the right place to ask my question. Sorry if it is not.  I have set up a wireguard VPN with 1 server A (192.168.100.1 with public IP and OpenBSD) and 2 clients, B (192.168.100.2, at home behind the ISP router 192.168.1.1 and Ubuntu) and C (192.168.100.3, at work behind a university network and OpenBSD). The VPN works, I mean I can ssh everywhere. Ping also works of course, but when I ping from B to C or C to B I have an ICMP redirect message:

192.168.100.1 > 192.168.100.3: icmp: redirect 192.168.100.2 to host 192.168.100.2

and 

192.168.100.1 > 192.168.100.2: icmp: redirect 192.168.100.3 to host 192.168.100.3

If I understand well it is because I have a sub-optimal routing table. Also messages can be ignore with net.inet.ip.redirect=0 on the server (I tried and  messages are indeed ignored). But I would like to understand where I loose this optimality, to improve my network (and increase my knowledge :o) because I use default config provide by wireguards tool. More details on my configuration are belows. 

Thanks for your kind help,
2rw3n.


On server A:
------------------

* Interface 

wg0: flags=81c3<UP,BROADCAST,RUNNING,NOARP,PROMISC,MULTICAST> mtu 1420
	index 4 priority 0 llprio 3
	wgport XXX
	wgpubkey XXX
	wgpeer client B pubkey
		wgendpoint X.X.X.X XXX
		tx: 1044, rx: 1244
		last handshake: 3 seconds ago
		wgaip 192.168.100.2/32
	wgpeer client C pubkey
		wgendpoint X.X.X.X XXX
		tx: 12864, rx: 12796
		last handshake: 75 seconds ago
		wgaip 192.168.100.3/32
	groups: wg
	inet 192.168.100.1 netmask 0xffffff00 broadcast 192.168.100.255
	
* Routing tables (XXX.XXX.XXX.242 is the public IP)

Destination        Gateway            Flags   Refs      Use   Mtu  Prio Iface
default            XXX.XXX.XXX.1        UGS       10     1136     -     8 vio0 
224/4              127.0.0.1          URS        0        0 32768     8 lo0  
127/8              127.0.0.1          UGRS       0        0 32768     8 lo0  
127.0.0.1          127.0.0.1          UHhl       1        2 32768     1 lo0  
XXX.XXX.XXX.1        32:8d:e2:42:a6:79  UHLch      1       25     -     7 vio0 
XXX.XXX.XXX.1/32     XXX.XXX.XXX.242      UCS        1        0     -     8 vio0 
XXX.XXX.XXX.242      fa:16:3e:db:cf:4b  UHLl       0      760     -     1 vio0 
XXX.XXX.XXX.242/32   XXX.XXX.XXX.242      UCn        0        0     -     4 vio0 
192.168.100/24     192.168.100.1      UCn        2        2     -     4 wg0  
192.168.100.1      wg0                UHl        0        0     -     1 wg0  
192.168.100.2      link#0             UHc        0       20     -     3 wg0  
192.168.100.3      link#0             UHc        1       16     -     3 wg0  
192.168.100.255    192.168.100.1      UHb        0        0     -     1 wg0  



On client B:
---------------

* Interface
 
interface: wg0
  public key: XXX
  private key: (hidden)
  listening port: XXX

peer: server A pubkey
  endpoint: server A public IP:XXX
  allowed ips: 192.168.100.0/24
  latest handshake: 11 seconds ago
  transfer: 38.16 KiB received, 39.15 KiB sent
  persistent keepalive: every 25 seconds

* Routes 

default via 192.168.1.1 dev wlp0s20f3 proto dhcp metric 600 
169.254.0.0/16 dev wlp0s20f3 scope link metric 1000 
192.168.1.0/24 dev wlp0s20f3 proto kernel scope link src 192.168.1.21 metric 600 
192.168.100.0/24 dev wg0 proto kernel scope link src 192.168.100.2 

and

Destination     Passerelle      Genmask         Indic   MSS Fen?tre irtt Iface
0.0.0.0         192.168.1.1     0.0.0.0         UG        0 0          0 wlp0s20f3
169.254.0.0     0.0.0.0         255.255.0.0     U         0 0          0 wlp0s20f3
192.168.1.0     0.0.0.0         255.255.255.0   U         0 0          0 wlp0s20f3
192.168.100.0   0.0.0.0         255.255.255.0   U         0 0          0 wg0


On client C :
-----------------

* Interface 

wg0: flags=80c3<UP,BROADCAST,RUNNING,NOARP,MULTICAST> mtu 1420
	index 4 priority 0 llprio 3
	wgport XXX
	wgpubkey XXX
	wgpeer server A pubkey
		wgpka 20 (sec)
		wgendpoint server A public IP XXX
		tx: 1053728, rx: 1269212
		last handshake: 32 seconds ago
		wgaip 192.168.100.0/24
	groups: wg
	inet 192.168.100.3 netmask 0xffffff00 broadcast 192.168.100.255
	
* Routing tables

Internet:
Destination        Gateway            Flags   Refs      Use   Mtu  Prio Iface
default            10.16.39.254       UGS        6       66     -     8 em0  
224/4              127.0.0.1          URS        0        0 32768     8 lo0  
10.16.38/23        10.16.38.180       UCn        1     1645     -     4 em0  
10.16.38.180       b0:7b:25:1e:e7:04  UHLl       0    11950     -     1 em0  
10.16.39.254       40:71:83:3a:a9:c0  UHLch      1      901     -     3 em0  
10.16.39.255       10.16.38.180       UHb        0     3207     -     1 em0  
127/8              127.0.0.1          UGRS       0        0 32768     8 lo0  
127.0.0.1          127.0.0.1          UHhl       1        6 32768     1 lo0  
192.168.100/24     192.168.100.3      UCn        2        0     -     4 wg0  
192.168.100.1      link#0             UHc        1       18     -     3 wg0  
192.168.100.2      link#0             UHc        2       81     - L   3 wg0  
192.168.100.3      wg0                UHl        0      252     -     1 wg0  
192.168.100.255    192.168.100.3      UHb        0        0     -     1 wg0  




From abliss at gmail.com  Sun Oct 24 16:47:23 2021
From: abliss at gmail.com (Adam Bliss)
Date: Sun, 24 Oct 2021 12:47:23 -0400
Subject: wireguard-go no longer builds with go1.16 ?
Message-ID: <CAEAeJWyPEhWgPQXEbJ=26G2FvK0xzyvmQ4reakdFpCt8wgpGPg@mail.gmail.com>

The README says 1.16, but it seems like go1.17 may now be required due
to dependencies? This is what I get with go1.16 on ubuntu focal:

$ make
go build -v -o "wireguard-go"
go: downloading golang.org/x/sys v0.0.0-20211020174200-9d6173849985
go: downloading golang.org/x/net v0.0.0-20211020060615-d418f374d309
go: downloading golang.org/x/crypto v0.0.0-20210921155107-089bfa567519
//go:build comment without // +build comment
make[1]: *** [Makefile:20: wireguard-go] Error 1
make: *** [Makefile:12: generate-version-and-build] Error 2


Thanks,
--Adam

From Jason at zx2c4.com  Mon Oct 25 15:59:23 2021
From: Jason at zx2c4.com (Jason A. Donenfeld)
Date: Mon, 25 Oct 2021 17:59:23 +0200
Subject: wireguard-go no longer builds with go1.16 ?
In-Reply-To: <CAEAeJWyPEhWgPQXEbJ=26G2FvK0xzyvmQ4reakdFpCt8wgpGPg@mail.gmail.com>
References: <CAEAeJWyPEhWgPQXEbJ=26G2FvK0xzyvmQ4reakdFpCt8wgpGPg@mail.gmail.com>
Message-ID: <CAHmME9qt5xSXKKWgpFZ=1kGBDrK-4MUHT4931b9bgeTVy-Rsog@mail.gmail.com>

Thanks for the report:

https://git.zx2c4.com/wireguard-go/commit/?id=828a885a7140e1fe23a684e4a8126cfd978ca0c8

From ryanroosa at gmail.com  Mon Oct 25 17:17:09 2021
From: ryanroosa at gmail.com (Ryan Roosa)
Date: Mon, 25 Oct 2021 13:17:09 -0400
Subject: wireguard-freebsd handshaking issue upon underlying WAN
Message-ID: <CABOEyuLiu96_KxEH5sdGLaE5-s9e8F_c0a1PKHSnFcMKp5YHow@mail.gmail.com>

Hello,
First off, I want to say thank you for the FreeBSD kernel module work
as it is greatly appreciated by myself and many others running *sense
firewalls :)

Generally wireguard-freebsd (wireguard-kmod 0.0.20210606_1) is running
quite well in my experience however, there is one issue which I have
been able to reproduce consistently: when the underlying WAN
connection that a tunnel is using is disrupted for the span of time
amounting to two missed handshake attempts (~4-5 minutes giving the ~2
minute average of handshake attempts), the tunnel will never handshake
again upon subsequent WAN restoration. This is the case even if one
resets the tunnel with 'wg-quick down ; wg-quick up' or restarts the
underlying OS (tried with both the latest stable versions of pfSense
and OPNSense community). For reference I am using a keep alive value
of 30 seconds in this scenario.

The only thing I've been able to do to get an existing tunnel
configuration handshaking with a peer endpoint again after its
Internet connection has been disrupted (outside of a complete removal
and rebuild) is to arbitrarily change the configured tunnel's
listening port (ex. 51820 to 51821 etc.). Upon saving and application
of the port change, the tunnel then handshakes with the peer endpoint
again immediately.

Given the symptom, it seems there may be some issue surrounding tunnel
handshaking resiliency when the underlying WAN drops out unexpectedly
for an extended period. If there is any way to look into this to
improve upon it so that after a 5+ minute internet outage a tunnel
could resume handshaking on its own without manual intervention, this
would be greatly appreciated.

I've got a 'bandaid' script running every 5 minutes currently which
checks the peer's handshake age and then changes the tunnel listen
port arbitrarily to restore connectivity then changes it back after 5
minutes of successful handshaking but obviously this is less than
ideal. As an additional data point I found if I switched the port and
tried to switch it back before another 5 minutes had passed, it would
stop handshaking again so there seems to be something special around
the 5 minute number regarding handshakes. Not sure if this is helpful
or not but thought I would include it.

Thank you in advance for looking into this and if there is any
additional information I can provide which may be of assistance I
would be happy to provide it.

Cheers,
Ryan Roosa

From Jason at zx2c4.com  Tue Oct 26 09:29:28 2021
From: Jason at zx2c4.com (Jason A. Donenfeld)
Date: Tue, 26 Oct 2021 11:29:28 +0200
Subject: wireguard-freebsd handshaking issue upon underlying WAN
In-Reply-To: <CABOEyuLiu96_KxEH5sdGLaE5-s9e8F_c0a1PKHSnFcMKp5YHow@mail.gmail.com>
References: <CABOEyuLiu96_KxEH5sdGLaE5-s9e8F_c0a1PKHSnFcMKp5YHow@mail.gmail.com>
Message-ID: <CAHmME9oPi=LvdcZC-FA+SvyxEmcRULtNB8hAQZoCAX=kQycTXA@mail.gmail.com>

Hi Ryan,

Thanks for the report. Kyle saw your reddit post earlier and tracked
this down, I think/hope, to a bug in the state machine cranking. I
committed the fix here -- https://w-g.pw/l/yQTw -- which will be part
of the next snapshot. Hopefully that will fix the issue, but if it
doesn't, please do update this thread so we can keep searching.

Regards,
Jason

From Jason at zx2c4.com  Tue Oct 26 10:05:57 2021
From: Jason at zx2c4.com (Jason A. Donenfeld)
Date: Tue, 26 Oct 2021 12:05:57 +0200
Subject: Windows Log Output to Event Viewer or Text File
In-Reply-To: <PH0PR05MB7962AD9044A813DE82AE796A99B99@PH0PR05MB7962.namprd05.prod.outlook.com>
References: <SA1PR05MB79668D58D886BB8D36F558FE99B29@SA1PR05MB7966.namprd05.prod.outlook.com>
 <CAHmME9pu=syh5zkig=e1KFQKMvCYfu6e+BhKENdwZARCrpss7Q@mail.gmail.com>
 <PH0PR05MB7962AD22BE333E844C0A79B799B69@PH0PR05MB7962.namprd05.prod.outlook.com>
 <CAHmME9pgbxyAxgBAfgLMVquzmPC9fiBzjWhQ5zJV+3bR2aR9dQ@mail.gmail.com>
 <PH0PR05MB79623EC9C12B4AE732B04E9299B79@PH0PR05MB7962.namprd05.prod.outlook.com>
 <CAHmME9q3j=2HG-4v5tpHRM9DCKpw+5iSMyzu30jAUHS1r=eDHw@mail.gmail.com>
 <PH0PR05MB79629688EB84BF07A2AEF54F99B89@PH0PR05MB7962.namprd05.prod.outlook.com>
 <CAHmME9rMPgJ3E82nLoXyFYDT7DyjkLsdV3ange9sd_Qm+1-BNw@mail.gmail.com>
 <PH0PR05MB79620040C1298C80677C4BF999B89@PH0PR05MB7962.namprd05.prod.outlook.com>
 <CAHmME9q9VB7eEsZCFUqo8XG7qj1pe2RwgLezJEHueF4_AjmA4A@mail.gmail.com>
 <PH0PR05MB7962AD9044A813DE82AE796A99B99@PH0PR05MB7962.namprd05.prod.outlook.com>
Message-ID: <CAHmME9qU7MMZd9RbmemvveTs9SDs48wCOUUp8AXwrvk29c-pMg@mail.gmail.com>

`wireguard /dumplog /tail | select` should now work in 0.5.1. Let me
know how that goes with your log ingestor.

Jason

From ryanroosa at gmail.com  Wed Oct 27 23:45:33 2021
From: ryanroosa at gmail.com (Ryan Roosa)
Date: Wed, 27 Oct 2021 19:45:33 -0400
Subject: wireguard-freebsd handshaking issue upon underlying WAN
In-Reply-To: <CAHmME9oPi=LvdcZC-FA+SvyxEmcRULtNB8hAQZoCAX=kQycTXA@mail.gmail.com>
References: <CABOEyuLiu96_KxEH5sdGLaE5-s9e8F_c0a1PKHSnFcMKp5YHow@mail.gmail.com>
 <CAHmME9oPi=LvdcZC-FA+SvyxEmcRULtNB8hAQZoCAX=kQycTXA@mail.gmail.com>
Message-ID: <CABOEyuJidG_u8SFGnhV6aw5y7PHVhaW_diDrpdMMHfOE4fHRWA@mail.gmail.com>

Hi Jason,
Thank you very much for this! I received word from the OPNSense team
that the referenced snapshot should be made available in OPNSense
21.7.5. I will test and provide feedback just as soon as I can get on
the aforementioned OPNSense release which includes the fix.

Cheers,
-Ryan

On Tue, Oct 26, 2021 at 5:29 AM Jason A. Donenfeld <Jason at zx2c4.com> wrote:
>
> Hi Ryan,
>
> Thanks for the report. Kyle saw your reddit post earlier and tracked
> this down, I think/hope, to a bug in the state machine cranking. I
> committed the fix here -- https://w-g.pw/l/yQTw -- which will be part
> of the next snapshot. Hopefully that will fix the issue, but if it
> doesn't, please do update this thread so we can keep searching.
>
> Regards,
> Jason

From stephen at slarew.net  Thu Oct 28 07:16:37 2021
From: stephen at slarew.net (Stephen Larew)
Date: Thu, 28 Oct 2021 00:16:37 -0700
Subject: Split DNS for macOS
Message-ID: <20211028071638.88001-1-stephen@slarew.net>

For many months now, I have been running a patched WireGuard macOS app
that enables a split DNS configuration. I would like to try to upstream
my patches for split DNS.

There has been some interest in this patch:
- "Mac APP DNS Search Domain" thread from July and August 2021 [1]
- A commenter on my GitHub fork of wireguard-apple.

What is split DNS? It allows sending DNS queries to a specific server
based on the domain name. Systemd-resolved calls it a routing domain.
Apple's Network Extension framework calls it a match domain.  Split DNS
is especially useful for internal DNS servers.

For example, if corp.example.com is a routing domain for the DNS server
at 192.0.2.1 (only accessible over WireGuard), then
server.corp.example.com is resolved using 192.0.2.1 while
www.example.com is resolved using some other DNS resolver (depending on
the other network settings in macOS).

The proposed patch adds new syntax to the wg-quick DNS= line.
Specifically, a tilde prefixed domain is treated as a routing domain.
Multiple routing domains can be added.

Limitations:
- Needs modifications to iOS UI to work on iOS.
- Only matching routing domains are sent to the DNS servers specified in
  the DNS= config line.  No separate fallback catch-all DNS server can
  be set.
- Routing/match domains are also included in the list of search domains.
  This could be changed with the matchDomainsNoSearch API, but lacking
  more UI or config file changes to expose this option to the user, I
  went with the default.

[1] https://lore.kernel.org/wireguard/20210810074232.aah5ktq5yzysaaey at SvensMacBookAir-2.local/T/
[2] https://github.com/slarew/wireguard-apple/commit/6ebc356d9e11ab91443e06de5e89f1af57fcdff8



From stephen at slarew.net  Thu Oct 28 07:16:38 2021
From: stephen at slarew.net (Stephen Larew)
Date: Thu, 28 Oct 2021 00:16:38 -0700
Subject: [PATCH] Enable "split DNS" configurations for an interface
In-Reply-To: <20211028071638.88001-1-stephen@slarew.net>
References: <20211028071638.88001-1-stephen@slarew.net>
Message-ID: <20211028071638.88001-2-stephen@slarew.net>

By adding a tilde prefix to a domain name entry in the DNS= line, the
domain is interpreted as a "matching domain" for DNS routing instead of
a "search domain."  This corresponds to setting a non-empty
NEDNSSettings.matchDomains property for the network tunnel.  Using tilde
as a prefix is borrowed from systemd-resolved's equivalent usage.

If one or more match domains are specified, then the specified DNS
resolvers are only used for those matching domains instead of acting as
the first resolver before the system's primary DNS resolvers.

Signed-off-by: Stephen Larew <stephen at slarew.net>
---
 .../Shared/Model/TunnelConfiguration+WgQuickConfig.swift | 5 +++++
 .../Tunnel/TunnelConfiguration+UapiConfig.swift          | 1 +
 Sources/WireGuardApp/UI/TunnelViewModel.swift            | 7 ++++++-
 Sources/WireGuardApp/UI/macOS/View/highlighter.c         | 9 ++++++++-
 Sources/WireGuardKit/InterfaceConfiguration.swift        | 4 +++-
 Sources/WireGuardKit/PacketTunnelSettingsGenerator.swift | 7 ++++++-
 6 files changed, 29 insertions(+), 4 deletions(-)

diff --git a/Sources/Shared/Model/TunnelConfiguration+WgQuickConfig.swift b/Sources/Shared/Model/TunnelConfiguration+WgQuickConfig.swift
index 5d5216c..87bc93f 100644
--- a/Sources/Shared/Model/TunnelConfiguration+WgQuickConfig.swift
+++ b/Sources/Shared/Model/TunnelConfiguration+WgQuickConfig.swift
@@ -136,6 +136,7 @@ extension TunnelConfiguration {
         if !interface.dns.isEmpty || !interface.dnsSearch.isEmpty {
             var dnsLine = interface.dns.map { $0.stringRepresentation }
             dnsLine.append(contentsOf: interface.dnsSearch)
+            dnsLine.append(contentsOf: interface.dnsMatchDomains.map { "~" + $0 })
             let dnsString = dnsLine.joined(separator: ", ")
             output.append("DNS = \(dnsString)\n")
         }
@@ -191,15 +192,19 @@ extension TunnelConfiguration {
         if let dnsString = attributes["dns"] {
             var dnsServers = [DNSServer]()
             var dnsSearch = [String]()
+            var dnsMatchDomains = [String]()
             for dnsServerString in dnsString.splitToArray(trimmingCharacters: .whitespacesAndNewlines) {
                 if let dnsServer = DNSServer(from: dnsServerString) {
                     dnsServers.append(dnsServer)
+                } else if dnsServerString.first == "~" && !dnsServerString.dropFirst().isEmpty {
+                    dnsMatchDomains.append(String(dnsServerString.dropFirst()))
                 } else {
                     dnsSearch.append(dnsServerString)
                 }
             }
             interface.dns = dnsServers
             interface.dnsSearch = dnsSearch
+            interface.dnsMatchDomains = dnsMatchDomains
         }
         if let mtuString = attributes["mtu"] {
             guard let mtu = UInt16(mtuString) else {
diff --git a/Sources/WireGuardApp/Tunnel/TunnelConfiguration+UapiConfig.swift b/Sources/WireGuardApp/Tunnel/TunnelConfiguration+UapiConfig.swift
index cdc81ce..f2ff763 100644
--- a/Sources/WireGuardApp/Tunnel/TunnelConfiguration+UapiConfig.swift
+++ b/Sources/WireGuardApp/Tunnel/TunnelConfiguration+UapiConfig.swift
@@ -75,6 +75,7 @@ extension TunnelConfiguration {
         interfaceConfiguration?.addresses = base?.interface.addresses ?? []
         interfaceConfiguration?.dns = base?.interface.dns ?? []
         interfaceConfiguration?.dnsSearch = base?.interface.dnsSearch ?? []
+        interfaceConfiguration?.dnsMatchDomains = base?.interface.dnsMatchDomains ?? []
         interfaceConfiguration?.mtu = base?.interface.mtu
 
         if let interfaceConfiguration = interfaceConfiguration {
diff --git a/Sources/WireGuardApp/UI/TunnelViewModel.swift b/Sources/WireGuardApp/UI/TunnelViewModel.swift
index b65c8cc..3b3fcda 100644
--- a/Sources/WireGuardApp/UI/TunnelViewModel.swift
+++ b/Sources/WireGuardApp/UI/TunnelViewModel.swift
@@ -139,9 +139,10 @@ class TunnelViewModel {
             if let mtu = config.mtu {
                 scratchpad[.mtu] = String(mtu)
             }
-            if !config.dns.isEmpty || !config.dnsSearch.isEmpty {
+            if !config.dns.isEmpty || !config.dnsSearch.isEmpty || !config.dnsMatchDomains.isEmpty {
                 var dns = config.dns.map { $0.stringRepresentation }
                 dns.append(contentsOf: config.dnsSearch)
+                dns.append(contentsOf: config.dnsMatchDomains.map { "~" + $0 })
                 scratchpad[.dns] = dns.joined(separator: ", ")
             }
             return scratchpad
@@ -197,15 +198,19 @@ class TunnelViewModel {
             if let dnsString = scratchpad[.dns] {
                 var dnsServers = [DNSServer]()
                 var dnsSearch = [String]()
+                var dnsMatchDomains = [String]()
                 for dnsServerString in dnsString.splitToArray(trimmingCharacters: .whitespacesAndNewlines) {
                     if let dnsServer = DNSServer(from: dnsServerString) {
                         dnsServers.append(dnsServer)
+                    } else if dnsServerString.first == "~" && !dnsServerString.dropFirst().isEmpty {
+                        dnsMatchDomains.append(String(dnsServerString.dropFirst()))
                     } else {
                         dnsSearch.append(dnsServerString)
                     }
                 }
                 config.dns = dnsServers
                 config.dnsSearch = dnsSearch
+                config.dnsMatchDomains = dnsMatchDomains
             }
 
             guard errorMessages.isEmpty else { return .error(errorMessages.first!) }
diff --git a/Sources/WireGuardApp/UI/macOS/View/highlighter.c b/Sources/WireGuardApp/UI/macOS/View/highlighter.c
index d89feda..e005377 100644
--- a/Sources/WireGuardApp/UI/macOS/View/highlighter.c
+++ b/Sources/WireGuardApp/UI/macOS/View/highlighter.c
@@ -121,6 +121,13 @@ static bool is_valid_hostname(string_span_t s)
 	return num_digit != num_entity;
 }
 
+static bool is_valid_dns_match_hostname(string_span_t s)
+{
+    if (!s.len || s.s[0] != '~')
+        return false;
+    return is_valid_hostname((string_span_t){ s.s + 1, s.len - 1});
+}
+
 static bool is_valid_ipv4(string_span_t s)
 {
 	for (size_t j, i = 0, pos = 0; i < 4 && pos < s.len; ++i) {
@@ -448,7 +455,7 @@ static void highlight_multivalue_value(struct highlight_span_array *ret, const s
 	case DNS:
 		if (is_valid_ipv4(s) || is_valid_ipv6(s))
 			append_highlight_span(ret, parent.s, s, HighlightIP);
-		else if (is_valid_hostname(s))
+		else if (is_valid_hostname(s) || is_valid_dns_match_hostname(s))
 			append_highlight_span(ret, parent.s, s, HighlightHost);
 		else
 			append_highlight_span(ret, parent.s, s, HighlightError);
diff --git a/Sources/WireGuardKit/InterfaceConfiguration.swift b/Sources/WireGuardKit/InterfaceConfiguration.swift
index 4fb8f1b..521d4b8 100644
--- a/Sources/WireGuardKit/InterfaceConfiguration.swift
+++ b/Sources/WireGuardKit/InterfaceConfiguration.swift
@@ -11,6 +11,7 @@ public struct InterfaceConfiguration {
     public var mtu: UInt16?
     public var dns = [DNSServer]()
     public var dnsSearch = [String]()
+    public var dnsMatchDomains = [String]()
 
     public init(privateKey: PrivateKey) {
         self.privateKey = privateKey
@@ -27,6 +28,7 @@ extension InterfaceConfiguration: Equatable {
             lhs.listenPort == rhs.listenPort &&
             lhs.mtu == rhs.mtu &&
             lhs.dns == rhs.dns &&
-            lhs.dnsSearch == rhs.dnsSearch
+            lhs.dnsSearch == rhs.dnsSearch &&
+            lhs.dnsMatchDomains == rhs.dnsMatchDomains
     }
 }
diff --git a/Sources/WireGuardKit/PacketTunnelSettingsGenerator.swift b/Sources/WireGuardKit/PacketTunnelSettingsGenerator.swift
index c53a82c..dac7648 100644
--- a/Sources/WireGuardKit/PacketTunnelSettingsGenerator.swift
+++ b/Sources/WireGuardKit/PacketTunnelSettingsGenerator.swift
@@ -88,7 +88,12 @@ class PacketTunnelSettingsGenerator {
             let dnsSettings = NEDNSSettings(servers: dnsServerStrings)
             dnsSettings.searchDomains = tunnelConfiguration.interface.dnsSearch
             if !tunnelConfiguration.interface.dns.isEmpty {
-                dnsSettings.matchDomains = [""] // All DNS queries must first go through the tunnel's DNS
+                if tunnelConfiguration.interface.dnsMatchDomains.isEmpty {
+                    // All DNS queries must first go through the tunnel's DNS
+                    dnsSettings.matchDomains = [""]
+                } else {
+                    dnsSettings.matchDomains = tunnelConfiguration.interface.dnsMatchDomains
+                }
             }
             networkSettings.dnsSettings = dnsSettings
         }
-- 
2.30.1 (Apple Git-130)


From Jason at zx2c4.com  Fri Oct 29 14:27:17 2021
From: Jason at zx2c4.com (Jason A. Donenfeld)
Date: Fri, 29 Oct 2021 16:27:17 +0200
Subject: [PATCH] wireguard: queueing: Fix implicit type conversion
In-Reply-To: <1635469664-1708957-1-git-send-email-jiasheng@iscas.ac.cn>
References: <1635469664-1708957-1-git-send-email-jiasheng@iscas.ac.cn>
Message-ID: <CAHmME9p9EA0qCw2ha_X9HR7NWSt1Zam4+srYHCs=-U4LvQiJdA@mail.gmail.com>

On Fri, Oct 29, 2021 at 3:08 AM Jiasheng Jiang <jiasheng at iscas.ac.cn> wrote:
> It is universally accepted that the implicit type conversion is
> terrible.

I'm not so sure about this, but either way, I think this needs a bit
more justification and analysis to merge. cpumask_weight returns an
unsigned, for example, and is used as a modulo operand later in the
function. It looks like nr_cpumask_bits is also unsigned. And so on.
So you're really trading one implicit type conversion package for
another. If you're swapping these around, why? It can't be because,
"it is universally accepted that the implicit type conversion is
terrible," since you're adding more of it in a different form. Is your
set of implicit type conversions semantically more proper? If so,
please describe that. Alternatively, is there a way to harmonize
everything into one type? Is there a minimal set of casts that enables
that?

Jason

From bferrell at baywinds.org  Thu Oct 28 09:58:34 2021
From: bferrell at baywinds.org (Bruce Ferrell)
Date: Thu, 28 Oct 2021 02:58:34 -0700
Subject: Split DNS for macOS
In-Reply-To: <20211028071638.88001-1-stephen@slarew.net>
References: <20211028071638.88001-1-stephen@slarew.net>
Message-ID: <0e80daf1-b4c2-b177-28ca-79e6575a339c@baywinds.org>

On 10/28/21 12:16 AM, Stephen Larew wrote:
> For many months now, I have been running a patched WireGuard macOS app
> that enables a split DNS configuration. I would like to try to upstream
> my patches for split DNS.
>
> There has been some interest in this patch:
> - "Mac APP DNS Search Domain" thread from July and August 2021 [1]
> - A commenter on my GitHub fork of wireguard-apple.
>
> What is split DNS? It allows sending DNS queries to a specific server
> based on the domain name. Systemd-resolved calls it a routing domain.
> Apple's Network Extension framework calls it a match domain.  Split DNS
> is especially useful for internal DNS servers.
>
> For example, if corp.example.com is a routing domain for the DNS server
> at 192.0.2.1 (only accessible over WireGuard), then
> server.corp.example.com is resolved using 192.0.2.1 while
> www.example.com is resolved using some other DNS resolver (depending on
> the other network settings in macOS).
>
> The proposed patch adds new syntax to the wg-quick DNS= line.
> Specifically, a tilde prefixed domain is treated as a routing domain.
> Multiple routing domains can be added.
>
> Limitations:
> - Needs modifications to iOS UI to work on iOS.
> - Only matching routing domains are sent to the DNS servers specified in
>    the DNS= config line.  No separate fallback catch-all DNS server can
>    be set.
> - Routing/match domains are also included in the list of search domains.
>    This could be changed with the matchDomainsNoSearch API, but lacking
>    more UI or config file changes to expose this option to the user, I
>    went with the default.
>
> [1] https://lore.kernel.org/wireguard/20210810074232.aah5ktq5yzysaaey at SvensMacBookAir-2.local/T/
> [2] https://github.com/slarew/wireguard-apple/commit/6ebc356d9e11ab91443e06de5e89f1af57fcdff8

That seems to be a redefinition of the existing definition of split DNS.

Most usually, split DNS is done at the DNS server and different zones are served to the resolver based on various criteria... Usually the origination IP of the query;? If the query 
comes from a client on your LAN, or a particular subnet, the contents of a particular, private, zone are returned.? If the query comes from, say the internet,? a public zone is 
returned.

YOUR description, is how DNS works in general... Except? your patch also seems to either bypass the resolver libraries or wedge itself in front of them The system resolver 
libraries well tested and understood and they handle the following very nicely.

There is the issue of what happens with large DNS responses.? Any DNS response over 512 bytes UDP fails and is required to be retried as a TCP query, which can handle the large 
response.? It's late and I'm too tired to look it up, but there IS an RFC for this.

It's a little known issue, but I've seen it when working with other VPN products that ignored/didn't understand the behavior. The results weren't pretty and really embarrassing.

Systemd has a bad habit of re-inventing the wheel... Badly. Eventually it get's sorted, but is that really progress? In the long haul, "move fast and break things" is a MAJOR pita 
for the vast majority of us.? But some like it.

For some real fun, look into DHCPCD.? It faithfully implements a particular RFC.? In some networking environments using VPNs, it breaks routing horribly.... But it meets the RFC.? 
You'll find it in Raspbian by default, and most other Debian derived distros.? I automatically rip it out and replace it with the also available ISC DHCP client.? That one is fully 
compatible with Windows, OS X, iOS, and every android and smart device I could test it with.? A DHCP server configured to be compatible with DHCPCD broke all of the previously named.

I'm giving this opinion away for free, so it's worth what you paid for it.



From jiasheng at iscas.ac.cn  Fri Oct 29 01:07:44 2021
From: jiasheng at iscas.ac.cn (Jiasheng Jiang)
Date: Fri, 29 Oct 2021 01:07:44 +0000
Subject: [PATCH] wireguard: queueing: Fix implicit type conversion
Message-ID: <1635469664-1708957-1-git-send-email-jiasheng@iscas.ac.cn>

The parameter 'cpu' is defined as unsigned int.
However in the cpumask_next() it is implicitly type conversed
to int.
It is universally accepted that the implicit type conversion is
terrible.
Also, having the good programming custom will set an example for
others.
Thus, it might be better to change the type of 'cpu' from
unsigned int to int.

Fixes: e7096c1 ("net: WireGuard secure network tunnel")
Signed-off-by: Jiasheng Jiang <jiasheng at iscas.ac.cn>
---
 drivers/net/wireguard/queueing.h | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/drivers/net/wireguard/queueing.h b/drivers/net/wireguard/queueing.h
index 4ef2944..64f397f 100644
--- a/drivers/net/wireguard/queueing.h
+++ b/drivers/net/wireguard/queueing.h
@@ -106,7 +106,7 @@ static inline void wg_reset_packet(struct sk_buff *skb, bool encapsulating)
 
 static inline int wg_cpumask_choose_online(int *stored_cpu, unsigned int id)
 {
-	unsigned int cpu = *stored_cpu, cpu_index, i;
+	int cpu = *stored_cpu, cpu_index, i;
 
 	if (unlikely(cpu == nr_cpumask_bits ||
 		     !cpumask_test_cpu(cpu, cpu_online_mask))) {
-- 
2.7.4


From igorbzin at gmail.com  Fri Oct 29 13:59:23 2021
From: igorbzin at gmail.com (Igor Bozin)
Date: Fri, 29 Oct 2021 15:59:23 +0200
Subject: Support for M1 MacBook Simulators
Message-ID: <18206D0A-EA19-4230-AAEC-8D52BE66A1C7@gmail.com>

Hello,

I am trying to build a project which contains a dependency on WireGuard through a Swift package.
The laptop I am running on (M1 MacBook) can't build the project, because libwg-go can't be compiled for the arm64 architecture for iPhone simulators. 
I am aware that Network Extensions functionality doesn't work on simulators, but not making the binaries available for simulators makes it impossible to test other features of the App in any project that WireGuard is part of.
Are there any plans to support simulator architectures as well, or any other workarounds for this?

Best regards,

Igor

From dotneutron at protonmail.ch  Fri Oct 29 15:24:41 2021
From: dotneutron at protonmail.ch (Neutron)
Date: Fri, 29 Oct 2021 15:24:41 +0000
Subject: Support for M1 MacBook Simulators
In-Reply-To: <18206D0A-EA19-4230-AAEC-8D52BE66A1C7@gmail.com>
References: <18206D0A-EA19-4230-AAEC-8D52BE66A1C7@gmail.com>
Message-ID: <re37dSyY_rXzrSL5DCMtppy1I37FlAT-pqCZjPdPpw8CHDxX3Nn3xeVECxTBMRFJBk-XEINrSbXY4KvWEtfii8GbwxZPNg7CZgCW-3asMus=@protonmail.ch>

Hey Igor,

I believe I encountered this in the past.
https://lists.zx2c4.com/pipermail/wireguard/2021-September/007001.html

I'm not sure what your exact setup is since you mentioned it's a separate project presumably depending on WireGuardKit, but I can build libwg-go for arm64 just fine using the trick I mentioned in the linked thread. Just add this flag to Sources/WireGuardKitGo/Makefile.
GOOS_iphonesimulator := ios

Hope it helps.


From eric.dumazet at gmail.com  Fri Oct 29 15:30:26 2021
From: eric.dumazet at gmail.com (Eric Dumazet)
Date: Fri, 29 Oct 2021 08:30:26 -0700
Subject: [PATCH] wireguard: queueing: Fix implicit type conversion
In-Reply-To: <CAHmME9p9EA0qCw2ha_X9HR7NWSt1Zam4+srYHCs=-U4LvQiJdA@mail.gmail.com>
References: <1635469664-1708957-1-git-send-email-jiasheng@iscas.ac.cn>
 <CAHmME9p9EA0qCw2ha_X9HR7NWSt1Zam4+srYHCs=-U4LvQiJdA@mail.gmail.com>
Message-ID: <99a32425-ca42-8d99-1276-efb889300cce@gmail.com>



On 10/29/21 7:27 AM, Jason A. Donenfeld wrote:
> On Fri, Oct 29, 2021 at 3:08 AM Jiasheng Jiang <jiasheng at iscas.ac.cn> wrote:
>> It is universally accepted that the implicit type conversion is
>> terrible.
> 
> I'm not so sure about this, but either way, I think this needs a bit
> more justification and analysis to merge. cpumask_weight returns an
> unsigned, for example, and is used as a modulo operand later in the
> function. It looks like nr_cpumask_bits is also unsigned. And so on.
> So you're really trading one implicit type conversion package for
> another. If you're swapping these around, why? It can't be because,
> "it is universally accepted that the implicit type conversion is
> terrible," since you're adding more of it in a different form. Is your
> set of implicit type conversions semantically more proper? If so,
> please describe that. Alternatively, is there a way to harmonize
> everything into one type? Is there a minimal set of casts that enables
> that?
>

I agree with you.

Even standard iterators play/mix with signed/unsigned in plain sight.

extern unsigned int nr_cpu_ids;

unsigned int cpumask_next(int n, const struct cpumask *srcp);

int cpumask_next_wrap(int n, const struct cpumask *mask, int start, bool wrap);

#define for_each_cpu(cpu, mask)				\
	for ((cpu) = -1;				\
		(cpu) = cpumask_next((cpu), (mask)),	\
		(cpu) < nr_cpu_ids;)


From stephen at slarew.net  Fri Oct 29 15:33:22 2021
From: stephen at slarew.net (Stephen Larew)
Date: Fri, 29 Oct 2021 08:33:22 -0700
Subject: Split DNS for macOS
In-Reply-To: <0e80daf1-b4c2-b177-28ca-79e6575a339c@baywinds.org>
References: <20211028071638.88001-1-stephen@slarew.net>
 <0e80daf1-b4c2-b177-28ca-79e6575a339c@baywinds.org>
Message-ID: <ED3FB4B3-96AD-4B98-86EC-D9F7999A530F@slarew.net>



> On Oct 28, 2021, at 02:58, Bruce Ferrell <bferrell at baywinds.org> wrote:
> 
> On 10/28/21 12:16 AM, Stephen Larew wrote:
>> For many months now, I have been running a patched WireGuard macOS app
>> that enables a split DNS configuration. I would like to try to upstream
>> my patches for split DNS.
>> 
>> There has been some interest in this patch:
>> - "Mac APP DNS Search Domain" thread from July and August 2021 [1]
>> - A commenter on my GitHub fork of wireguard-apple.
>> 
>> What is split DNS? It allows sending DNS queries to a specific server
>> based on the domain name. Systemd-resolved calls it a routing domain.
>> Apple's Network Extension framework calls it a match domain.  Split DNS
>> is especially useful for internal DNS servers.
>> 
>> For example, if corp.example.com is a routing domain for the DNS server
>> at 192.0.2.1 (only accessible over WireGuard), then
>> server.corp.example.com is resolved using 192.0.2.1 while
>> www.example.com is resolved using some other DNS resolver (depending on
>> the other network settings in macOS).
>> 
>> The proposed patch adds new syntax to the wg-quick DNS= line.
>> Specifically, a tilde prefixed domain is treated as a routing domain.
>> Multiple routing domains can be added.
>> 
>> Limitations:
>> - Needs modifications to iOS UI to work on iOS.
>> - Only matching routing domains are sent to the DNS servers specified in
>>   the DNS= config line.  No separate fallback catch-all DNS server can
>>   be set.
>> - Routing/match domains are also included in the list of search domains.
>>   This could be changed with the matchDomainsNoSearch API, but lacking
>>   more UI or config file changes to expose this option to the user, I
>>   went with the default.
>> 
>> [1] https://lore.kernel.org/wireguard/20210810074232.aah5ktq5yzysaaey at SvensMacBookAir-2.local/T/
>> [2] https://github.com/slarew/wireguard-apple/commit/6ebc356d9e11ab91443e06de5e89f1af57fcdff8
> 
> That seems to be a redefinition of the existing definition of split DNS.
> 
> Most usually, split DNS is done at the DNS server and different zones are served to the resolver based on various criteria... Usually the origination IP of the query;  If the query comes from a client on your LAN, or a particular subnet, the contents of a particular, private, zone are returned.  If the query comes from, say the internet,  a public zone is returned.
> 
> YOUR description, is how DNS works in general... Except  your patch also seems to either bypass the resolver libraries or wedge itself in front of them The system resolver libraries well tested and understood and they handle the following very nicely.
> 
> There is the issue of what happens with large DNS responses.  Any DNS response over 512 bytes UDP fails and is required to be retried as a TCP query, which can handle the large response.  It's late and I'm too tired to look it up, but there IS an RFC for this.
> 
> It's a little known issue, but I've seen it when working with other VPN products that ignored/didn't understand the behavior. The results weren't pretty and really embarrassing.
> 
> Systemd has a bad habit of re-inventing the wheel... Badly. Eventually it get's sorted, but is that really progress? In the long haul, "move fast and break things" is a MAJOR pita for the vast majority of us.  But some like it.
> 
> For some real fun, look into DHCPCD.  It faithfully implements a particular RFC.  In some networking environments using VPNs, it breaks routing horribly.... But it meets the RFC.  You'll find it in Raspbian by default, and most other Debian derived distros.  I automatically rip it out and replace it with the also available ISC DHCP client.  That one is fully compatible with Windows, OS X, iOS, and every android and smart device I could test it with.  A DHCP server configured to be compatible with DHCPCD broke all of the previously named.
> 
> I'm giving this opinion away for free, so it's worth what you paid for it.

Regardless of naming or definitions, I think the ability for a *local device* to route DNS queries to different DNS servers based on domain matching criteria is certainly useful. It?s not always possible or desirable to control and configure an upstream DNS server. Hence, this patch enables the local device to do split DNS.

To be clear, this patch does not bypass or wedge around anything. In fact, it configures the native macOS DNS settings in the appropriate manner to effect a split DNS configuration.

As a result of controlling the native macOS DNS resolution logic, any feature, absent or present, in the macOS DNS resolver libraries should be unaffected. This includes the large DNS response and TCP behavior. I do not expect the small/large UDP/TCP DNS features to change behavior when using a split DNS configuration as proposed in this patch.

-Stephen

From afried at spamteq.com  Fri Oct 29 17:03:22 2021
From: afried at spamteq.com (Andrew Fried)
Date: Fri, 29 Oct 2021 17:03:22 +0000
Subject: Split DNS for macOS
In-Reply-To: <ED3FB4B3-96AD-4B98-86EC-D9F7999A530F@slarew.net>
References: <20211028071638.88001-1-stephen@slarew.net>
 <0e80daf1-b4c2-b177-28ca-79e6575a339c@baywinds.org>
 <ED3FB4B3-96AD-4B98-86EC-D9F7999A530F@slarew.net>
Message-ID: <K25247-14.1635527002@spamhaus.org>

Hi Stephen,

A better solution to your problem would be to deploy DNSDIST:
	https://dnsdist.org/

I for one would hope that esoteric requests that address a solution for 
less than 1% of the users would be rejected with the overall goal of 
preventing feature creep and bloat.

Andrew


On 10/29/21 11:33 AM, Stephen Larew wrote:
> 
> 
>> On Oct 28, 2021, at 02:58, Bruce Ferrell <bferrell at baywinds.org> wrote:
>>
>> On 10/28/21 12:16 AM, Stephen Larew wrote:
>>> For many months now, I have been running a patched WireGuard macOS app
>>> that enables a split DNS configuration. I would like to try to upstream
>>> my patches for split DNS.
>>>
>>> There has been some interest in this patch:
>>> - "Mac APP DNS Search Domain" thread from July and August 2021 [1]
>>> - A commenter on my GitHub fork of wireguard-apple.
>>>
>>> What is split DNS? It allows sending DNS queries to a specific server
>>> based on the domain name. Systemd-resolved calls it a routing domain.
>>> Apple's Network Extension framework calls it a match domain.  Split DNS
>>> is especially useful for internal DNS servers.
>>>
>>> For example, if corp.example.com is a routing domain for the DNS server
>>> at 192.0.2.1 (only accessible over WireGuard), then
>>> server.corp.example.com is resolved using 192.0.2.1 while
>>> www.example.com is resolved using some other DNS resolver (depending on
>>> the other network settings in macOS).
>>>
>>> The proposed patch adds new syntax to the wg-quick DNS= line.
>>> Specifically, a tilde prefixed domain is treated as a routing domain.
>>> Multiple routing domains can be added.
>>>
>>> Limitations:
>>> - Needs modifications to iOS UI to work on iOS.
>>> - Only matching routing domains are sent to the DNS servers specified in
>>>    the DNS= config line.  No separate fallback catch-all DNS server can
>>>    be set.
>>> - Routing/match domains are also included in the list of search domains.
>>>    This could be changed with the matchDomainsNoSearch API, but lacking
>>>    more UI or config file changes to expose this option to the user, I
>>>    went with the default.
>>>
>>> [1] https://lore.kernel.org/wireguard/20210810074232.aah5ktq5yzysaaey at SvensMacBookAir-2.local/T/
>>> [2] https://github.com/slarew/wireguard-apple/commit/6ebc356d9e11ab91443e06de5e89f1af57fcdff8
>>
>> That seems to be a redefinition of the existing definition of split DNS.
>>
>> Most usually, split DNS is done at the DNS server and different zones are served to the resolver based on various criteria... Usually the origination IP of the query;  If the query comes from a client on your LAN, or a particular subnet, the contents of a particular, private, zone are returned.  If the query comes from, say the internet,  a public zone is returned.
>>
>> YOUR description, is how DNS works in general... Except  your patch also seems to either bypass the resolver libraries or wedge itself in front of them The system resolver libraries well tested and understood and they handle the following very nicely.
>>
>> There is the issue of what happens with large DNS responses.  Any DNS response over 512 bytes UDP fails and is required to be retried as a TCP query, which can handle the large response.  It's late and I'm too tired to look it up, but there IS an RFC for this.
>>
>> It's a little known issue, but I've seen it when working with other VPN products that ignored/didn't understand the behavior. The results weren't pretty and really embarrassing.
>>
>> Systemd has a bad habit of re-inventing the wheel... Badly. Eventually it get's sorted, but is that really progress? In the long haul, "move fast and break things" is a MAJOR pita for the vast majority of us.  But some like it.
>>
>> For some real fun, look into DHCPCD.  It faithfully implements a particular RFC.  In some networking environments using VPNs, it breaks routing horribly.... But it meets the RFC.  You'll find it in Raspbian by default, and most other Debian derived distros.  I automatically rip it out and replace it with the also available ISC DHCP client.  That one is fully compatible with Windows, OS X, iOS, and every android and smart device I could test it with.  A DHCP server configured to be compatible with DHCPCD broke all of the previously named.
>>
>> I'm giving this opinion away for free, so it's worth what you paid for it.
> 
> Regardless of naming or definitions, I think the ability for a *local device* to route DNS queries to different DNS servers based on domain matching criteria is certainly useful. It?s not always possible or desirable to control and configure an upstream DNS server. Hence, this patch enables the local device to do split DNS.
> 
> To be clear, this patch does not bypass or wedge around anything. In fact, it configures the native macOS DNS settings in the appropriate manner to effect a split DNS configuration.
> 
> As a result of controlling the native macOS DNS resolution logic, any feature, absent or present, in the macOS DNS resolver libraries should be unaffected. This includes the large DNS response and TCP behavior. I do not expect the small/large UDP/TCP DNS features to change behavior when using a split DNS configuration as proposed in this patch.
> 
> -Stephen
> 

-- 
Andrew Fried
afried at spamteq.com
+1.703.667.4050 Office
+1.703.362.0067 Mobile

From stephen at slarew.net  Fri Oct 29 21:07:38 2021
From: stephen at slarew.net (Stephen Larew)
Date: Fri, 29 Oct 2021 14:07:38 -0700
Subject: Split DNS for macOS
In-Reply-To: <K25247-14.1635527002@spamhaus.org>
References: <20211028071638.88001-1-stephen@slarew.net>
 <0e80daf1-b4c2-b177-28ca-79e6575a339c@baywinds.org>
 <ED3FB4B3-96AD-4B98-86EC-D9F7999A530F@slarew.net>
 <K25247-14.1635527002@spamhaus.org>
Message-ID: <E3607380-8E46-42CC-80B5-BB38F206E9BC@slarew.net>

> On Oct 29, 2021, at 10:03, Andrew Fried <afried at spamteq.com> wrote:
> 
>> On Oct 29, 2021, at 08:33, Stephen Larew <stephen at slarew.net> wrote:
>> 
>>> On Oct 28, 2021, at 02:58, Bruce Ferrell <bferrell at baywinds.org> wrote:
>>> 
>>> On 10/28/21 12:16 AM, Stephen Larew wrote:
>>>> For many months now, I have been running a patched WireGuard macOS app
>>>> that enables a split DNS configuration. I would like to try to upstream
>>>> my patches for split DNS.
>>>> 
>>>> There has been some interest in this patch:
>>>> - "Mac APP DNS Search Domain" thread from July and August 2021 [1]
>>>> - A commenter on my GitHub fork of wireguard-apple.
>>>> 
>>>> What is split DNS? It allows sending DNS queries to a specific server
>>>> based on the domain name. Systemd-resolved calls it a routing domain.
>>>> Apple's Network Extension framework calls it a match domain.  Split DNS
>>>> is especially useful for internal DNS servers.
>>>> 
>>>> For example, if corp.example.com is a routing domain for the DNS server
>>>> at 192.0.2.1 (only accessible over WireGuard), then
>>>> server.corp.example.com is resolved using 192.0.2.1 while
>>>> www.example.com is resolved using some other DNS resolver (depending on
>>>> the other network settings in macOS).
>>>> 
>>>> The proposed patch adds new syntax to the wg-quick DNS= line.
>>>> Specifically, a tilde prefixed domain is treated as a routing domain.
>>>> Multiple routing domains can be added.
>>>> 
>>>> Limitations:
>>>> - Needs modifications to iOS UI to work on iOS.
>>>> - Only matching routing domains are sent to the DNS servers specified in
>>>>  the DNS= config line.  No separate fallback catch-all DNS server can
>>>>  be set.
>>>> - Routing/match domains are also included in the list of search domains.
>>>>  This could be changed with the matchDomainsNoSearch API, but lacking
>>>>  more UI or config file changes to expose this option to the user, I
>>>>  went with the default.
>>>> 
>>>> [1] https://lore.kernel.org/wireguard/20210810074232.aah5ktq5yzysaaey at SvensMacBookAir-2.local/T/
>>>> [2] https://github.com/slarew/wireguard-apple/commit/6ebc356d9e11ab91443e06de5e89f1af57fcdff8
>>> 
>>> That seems to be a redefinition of the existing definition of split DNS.
>>> 
>>> Most usually, split DNS is done at the DNS server and different zones are served to the resolver based on various criteria... Usually the origination IP of the query;  If the query comes from a client on your LAN, or a particular subnet, the contents of a particular, private, zone are returned.  If the query comes from, say the internet,  a public zone is returned.
>>> 
>>> YOUR description, is how DNS works in general... Except  your patch also seems to either bypass the resolver libraries or wedge itself in front of them The system resolver libraries well tested and understood and they handle the following very nicely.
>>> 
>>> There is the issue of what happens with large DNS responses.  Any DNS response over 512 bytes UDP fails and is required to be retried as a TCP query, which can handle the large response.  It's late and I'm too tired to look it up, but there IS an RFC for this.
>>> 
>>> It's a little known issue, but I've seen it when working with other VPN products that ignored/didn't understand the behavior. The results weren't pretty and really embarrassing.
>>> 
>>> Systemd has a bad habit of re-inventing the wheel... Badly. Eventually it get's sorted, but is that really progress? In the long haul, "move fast and break things" is a MAJOR pita for the vast majority of us.  But some like it.
>>> 
>>> For some real fun, look into DHCPCD.  It faithfully implements a particular RFC.  In some networking environments using VPNs, it breaks routing horribly.... But it meets the RFC.  You'll find it in Raspbian by default, and most other Debian derived distros.  I automatically rip it out and replace it with the also available ISC DHCP client.  That one is fully compatible with Windows, OS X, iOS, and every android and smart device I could test it with.  A DHCP server configured to be compatible with DHCPCD broke all of the previously named.
>>> 
>>> I'm giving this opinion away for free, so it's worth what you paid for it.
>> 
>> Regardless of naming or definitions, I think the ability for a *local device* to route DNS queries to different DNS servers based on domain matching criteria is certainly useful. It?s not always possible or desirable to control and configure an upstream DNS server. Hence, this patch enables the local device to do split DNS.
>> 
>> To be clear, this patch does not bypass or wedge around anything. In fact, it configures the native macOS DNS settings in the appropriate manner to effect a split DNS configuration.
>> 
>> As a result of controlling the native macOS DNS resolution logic, any feature, absent or present, in the macOS DNS resolver libraries should be unaffected. This includes the large DNS response and TCP behavior. I do not expect the small/large UDP/TCP DNS features to change behavior when using a split DNS configuration as proposed in this patch.
>> 
>> -Stephen
> 
> Hi Stephen,
> 
> A better solution to your problem would be to deploy DNSDIST:
> 	https://dnsdist.org/
> 
> I for one would hope that esoteric requests that address a solution for less than 1% of the users would be rejected with the overall goal of preventing feature creep and bloat.
> 
> Andrew

DNSDIST may allow (I have not tried) one to create a split DNS scenario, but it is an extra piece of software that would need to be discovered, installed, configured, and maintained by a user or system administrator. I do not believe it would properly integrate with the macOS DNS resolution machinery. I do not believe it is a better solution to the problem.

As I understand it, under some circumstances, the Tailscale macOS app also implements split DNS in roughly the same manner as done in my patch. Namely, the tailscale app appears to use the Network Extension APIs to directly integrate with the macOS DNS resolution machinery. Perhaps the relevant difference is that tailscale approaches configuration differently (not based on wg-quick) than the WireGuard macOS app.

I do not have statistics or polling on who desires this split DNS feature. I have received private and public requests to upstream the feature. Tailscale also implements split DNS; presumable customers demand it. I suspect if the feature was available to users of the WireGuard app, then it would be used with precision to great effect. Users who do not need this split DNS feature do not lose any previous functionality in the macOS WireGuard app.

Personally, my one hesitation with this patch is that, as currently implemented, a new syntax is added to the wg-quick config file (tilde prefixed route/match domains). My patch does not address compatibility issues nor does it add documentation to wg-quick for the new syntax.

-Stephen

From mhp.driessen at gmail.com  Fri Oct 29 21:06:50 2021
From: mhp.driessen at gmail.com (Matty Driessen)
Date: Fri, 29 Oct 2021 23:06:50 +0200
Subject: Split DNS for macOS
Message-ID: <CAHsVW7N2JA25KXDRM0ghNrn-9BZpuqY0OMHX4WTi7hmXjS9nGA@mail.gmail.com>

Hello Andrew,

I just want to chime in here and say that I think the current
implementation of search domains is simply not working the way it
should on the MacOS client.

My use case is pretty common, an internal DNS server that has entries
for internal servers. I defined a search domain in the WireGuard
configuration; DNS = 10.13.13.1 mydomain.internal. The search domain
is for convenience, so I can just use the servername instead of
servername.mydomain.internal. Now this works fine when I route all the
traffic through the VPN (AllowedIPs = 0.0.0.0/0) but the search domain
is completely ignored when I only route the traffic I need to
(AllowedIPs = 10.13.13.0/24 192.168.0.0/24).

I don't think this is a configuration error on my side. The DNS
responds fine when using servername.mydomain.internal. This problem is
even mentioned in the "WireGuard macOS & iOS TODO List"
9. matchDomains=[??] doesn?t do what the documentation says.
Specifically, DNS servers are not used if allowed IPs isn?t 0.0.0.0/0.

The description isn't 100% accurate (or outdated), the DNS server is
used but the search domain isn't being set on the primary resolver.
Some have solved this issue by adding the search domains to the list
of matchDomains; dnsSettings.matchDomains = [""] +
dnsSettings.searchDomains. But that way the DNS server specified in
WireGuard is still the primary resolver for all DNS queries.

Here is a link on how OpenVPN handles this and I think it's how it
should work when not using AllowedIPs 0.0.0.0/0.
https://openvpn.net/faq/how-does-ios-interpret-pushed-dns-servers-and-search-domains/
On a split-tunnel, where redirect-gateway is not pushed by the server,
and at least one pushed DNS server is present:
- route all DNS requests through pushed DNS server(s) if no added
search domains.
- route DNS requests for added search domains only, if at least one
added search domain.

Yours sincerely,
Matty

--

Hi Stephen,

A better solution to your problem would be to deploy DNSDIST:
        https://dnsdist.org/

I for one would hope that esoteric requests that address a solution
for less than 1% of the users would be rejected with the overall goal
of preventing feature creep and bloat.

Andrew

From dz at ct.de  Sat Oct 30 21:00:38 2021
From: dz at ct.de (Dusan Zivadinovic)
Date: Sat, 30 Oct 2021 23:00:38 +0200
Subject: Split DNS for macOS
In-Reply-To: <E3607380-8E46-42CC-80B5-BB38F206E9BC@slarew.net>
References: <20211028071638.88001-1-stephen@slarew.net>
 <0e80daf1-b4c2-b177-28ca-79e6575a339c@baywinds.org>
 <ED3FB4B3-96AD-4B98-86EC-D9F7999A530F@slarew.net>
 <K25247-14.1635527002@spamhaus.org>
 <E3607380-8E46-42CC-80B5-BB38F206E9BC@slarew.net>
Message-ID: <C824C50B-843C-4C77-B7C3-62E850B73CB0@ct.de>

Hi list,

I would appreciate that, its a usefull feature.

Think of connecting to multiple remote VPN peers simultaneously,
and use different DNSes for certain domains in each VPN connection.

Just as an example, the iOS app DNSCloak also sports a split DNS function.

regards,

Dusan

> Am 29.10.2021 um 23:07 schrieb Stephen Larew <stephen at slarew.net>:
> 
>> On Oct 29, 2021, at 10:03, Andrew Fried <afried at spamteq.com> wrote:
>> 
>>> On Oct 29, 2021, at 08:33, Stephen Larew <stephen at slarew.net> wrote:
>>> 
>>>> On Oct 28, 2021, at 02:58, Bruce Ferrell <bferrell at baywinds.org> wrote:
>>>> 
>>>> On 10/28/21 12:16 AM, Stephen Larew wrote:
>>>>> For many months now, I have been running a patched WireGuard macOS app
>>>>> that enables a split DNS configuration. I would like to try to upstream
>>>>> my patches for split DNS.
>>>>> 
>>>>> There has been some interest in this patch:
>>>>> - "Mac APP DNS Search Domain" thread from July and August 2021 [1]
>>>>> - A commenter on my GitHub fork of wireguard-apple.
>>>>> 
>>>>> What is split DNS? It allows sending DNS queries to a specific server
>>>>> based on the domain name. Systemd-resolved calls it a routing domain.
>>>>> Apple's Network Extension framework calls it a match domain.  Split DNS
>>>>> is especially useful for internal DNS servers.
>>>>> 
>>>>> For example, if corp.example.com is a routing domain for the DNS server
>>>>> at 192.0.2.1 (only accessible over WireGuard), then
>>>>> server.corp.example.com is resolved using 192.0.2.1 while
>>>>> www.example.com is resolved using some other DNS resolver (depending on
>>>>> the other network settings in macOS).
>>>>> 
>>>>> The proposed patch adds new syntax to the wg-quick DNS= line.
>>>>> Specifically, a tilde prefixed domain is treated as a routing domain.
>>>>> Multiple routing domains can be added.
>>>>> 
>>>>> Limitations:
>>>>> - Needs modifications to iOS UI to work on iOS.
>>>>> - Only matching routing domains are sent to the DNS servers specified in
>>>>> the DNS= config line.  No separate fallback catch-all DNS server can
>>>>> be set.
>>>>> - Routing/match domains are also included in the list of search domains.
>>>>> This could be changed with the matchDomainsNoSearch API, but lacking
>>>>> more UI or config file changes to expose this option to the user, I
>>>>> went with the default.
>>>>> 
>>>>> [1] https://lore.kernel.org/wireguard/20210810074232.aah5ktq5yzysaaey at SvensMacBookAir-2.local/T/
>>>>> [2] https://github.com/slarew/wireguard-apple/commit/6ebc356d9e11ab91443e06de5e89f1af57fcdff8
>>>> 
>>>> That seems to be a redefinition of the existing definition of split DNS.
>>>> 
>>>> Most usually, split DNS is done at the DNS server and different zones are served to the resolver based on various criteria... Usually the origination IP of the query;  If the query comes from a client on your LAN, or a particular subnet, the contents of a particular, private, zone are returned.  If the query comes from, say the internet,  a public zone is returned.
>>>> 
>>>> YOUR description, is how DNS works in general... Except  your patch also seems to either bypass the resolver libraries or wedge itself in front of them The system resolver libraries well tested and understood and they handle the following very nicely.
>>>> 
>>>> There is the issue of what happens with large DNS responses.  Any DNS response over 512 bytes UDP fails and is required to be retried as a TCP query, which can handle the large response.  It's late and I'm too tired to look it up, but there IS an RFC for this.
>>>> 
>>>> It's a little known issue, but I've seen it when working with other VPN products that ignored/didn't understand the behavior. The results weren't pretty and really embarrassing.
>>>> 
>>>> Systemd has a bad habit of re-inventing the wheel... Badly. Eventually it get's sorted, but is that really progress? In the long haul, "move fast and break things" is a MAJOR pita for the vast majority of us.  But some like it.
>>>> 
>>>> For some real fun, look into DHCPCD.  It faithfully implements a particular RFC.  In some networking environments using VPNs, it breaks routing horribly.... But it meets the RFC.  You'll find it in Raspbian by default, and most other Debian derived distros.  I automatically rip it out and replace it with the also available ISC DHCP client.  That one is fully compatible with Windows, OS X, iOS, and every android and smart device I could test it with.  A DHCP server configured to be compatible with DHCPCD broke all of the previously named.
>>>> 
>>>> I'm giving this opinion away for free, so it's worth what you paid for it.
>>> 
>>> Regardless of naming or definitions, I think the ability for a *local device* to route DNS queries to different DNS servers based on domain matching criteria is certainly useful. It?s not always possible or desirable to control and configure an upstream DNS server. Hence, this patch enables the local device to do split DNS.
>>> 
>>> To be clear, this patch does not bypass or wedge around anything. In fact, it configures the native macOS DNS settings in the appropriate manner to effect a split DNS configuration.
>>> 
>>> As a result of controlling the native macOS DNS resolution logic, any feature, absent or present, in the macOS DNS resolver libraries should be unaffected. This includes the large DNS response and TCP behavior. I do not expect the small/large UDP/TCP DNS features to change behavior when using a split DNS configuration as proposed in this patch.
>>> 
>>> -Stephen
>> 
>> Hi Stephen,
>> 
>> A better solution to your problem would be to deploy DNSDIST:
>> 	https://dnsdist.org/
>> 
>> I for one would hope that esoteric requests that address a solution for less than 1% of the users would be rejected with the overall goal of preventing feature creep and bloat.
>> 
>> Andrew
> 
> DNSDIST may allow (I have not tried) one to create a split DNS scenario, but it is an extra piece of software that would need to be discovered, installed, configured, and maintained by a user or system administrator. I do not believe it would properly integrate with the macOS DNS resolution machinery. I do not believe it is a better solution to the problem.
> 
> As I understand it, under some circumstances, the Tailscale macOS app also implements split DNS in roughly the same manner as done in my patch. Namely, the tailscale app appears to use the Network Extension APIs to directly integrate with the macOS DNS resolution machinery. Perhaps the relevant difference is that tailscale approaches configuration differently (not based on wg-quick) than the WireGuard macOS app.
> 
> I do not have statistics or polling on who desires this split DNS feature. I have received private and public requests to upstream the feature. Tailscale also implements split DNS; presumable customers demand it. I suspect if the feature was available to users of the WireGuard app, then it would be used with precision to great effect. Users who do not need this split DNS feature do not lose any previous functionality in the macOS WireGuard app.
> 
> Personally, my one hesitation with this patch is that, as currently implemented, a new syntax is added to the wg-quick config file (tilde prefixed route/match domains). My patch does not address compatibility issues nor does it add documentation to wg-quick for the new syntax.
> 
> -Stephen


From frank at deze.org  Sun Oct 31 18:41:50 2021
From: frank at deze.org (Frank Volf)
Date: Sun, 31 Oct 2021 19:41:50 +0100
Subject: Wireguard on FreeBSD - a few questions
Message-ID: <cfddc41d-0768-af1a-d527-d2bccd5cfd56@deze.org>


Hi,

This weekend I installed Wireguard on FreeBSD 13.0 and until now 
everything seems to work fine (I use the kernel module).
Installation and configuration was easy and connecting with the Android 
app works great as well.

I do have a few questions.

1) Is it possible on FreeBSD to enable some kind of logging? I did made 
a small configuration error with my first client and it was hard to find 
the error, because there does not seem to be any logging at all.? Some 
logging information would be appreciated and probably wold have pointed 
me faster to the fact that I needed to switch two keys in my config.

2) I noticed that Wireguard uses a wildcard to listen to all IP 
addresses on my multi-homed machine on his dedicated UDP port. I would 
prefer if Wireguard would only bind to the specific IP address on the 
outside interface that is designated for that use. Is this possible?

3) Final question: is it possible on the server side to restrict the 
destinations that clients can connect to it? I know, that I can set the 
AllowedIPs on the client side to restrict that, but that setting can be 
changed at the client side. It would be nice if I could restrict 
destinations at the server side (so client X can only connect to an IP 
address of an internal server that it needs access to but nothing else). 
I can probably use a state full packet filtering firewall for this, but 
it would it be possible to configure this on the Wireguard server side 
as well?

That said, I'm pleased with the first test results of Wireguard on 
FreeBSD and hopefully it keeps on running fine. Great product!

Kind regards,

Frank