Wireguard Link issue

[Joshua Grimm]

Hey Guys,

I have used Wireguard for quite some time now and I'm facing a problem.

every other day one wireguard link stops working. It just stops
accepting traffic:

root at bladerunner-2:~# ping -c 1 10.0.1.1
PING 10.0.1.1 (10.0.1.1) 56(84) bytes of data.

--- 10.0.1.1 ping statistics ---
1 packets transmitted, 0 received, 100% packet loss, time 0ms

wg tells me that the link is up and handshakes are still working:

peer: Q7pO6XUeBKi2dtSZOIyXjpkyUESbvGHfSF7cu1KmD1g=
  endpoint: redacted:51820
  allowed ips: 0.0.0.0/0
  latest handshake: 11 seconds ago
  transfer: 162.67 GiB received, 8.32 GiB sent
  persistent keepalive: every 1 minute

if I restart the link with

systemctl restart wg-quick at wg0

it works again.

Here is the relevent part of my config:

[Interface]
Address = 10.0.1.3
PrivateKey = redacted
ListenPort = 51820
Table = off
FwMark = 1234

[Peer]
PublicKey = redacted
AllowedIps = 0.0.0.0/0
Endpoint = redacted:51820
PersistentKeepalive=60

I tried running wireguard in this debug mode with

echo "module wireguard +p" >/sys/kernel/debug/dynamic_debug/control

the only messages I get are:

[677569.105551] wireguard: wg0: Sending handshake initiation to peer 1
(redacted:51820)
[677569.116744] wireguard: wg0: Receiving handshake response from peer 1
(redacted:51820)
[677569.116761] wireguard: wg0: Keypair 28551 destroyed for peer 1
[677569.116765] wireguard: wg0: Keypair 28554 created for peer 1
[677569.116775] wireguard: wg0: Sending keepalive packet to peer 1
(redacted:51820)
[677585.105603] wireguard: wg0: Retrying handshake with peer 1
(redacted:51820) because we stopped hearing back after 15 seconds
[677585.105644] wireguard: wg0: Sending handshake initiation to peer 1
(redacted:51820)
[677585.112877] wireguard: wg0: Receiving handshake response from peer 1
(redacted:51820)
[677585.112893] wireguard: wg0: Keypair 28553 destroyed for peer 1
[677585.112898] wireguard: wg0: Keypair 28555 created for peer 1
[677585.112907] wireguard: wg0: Sending keepalive packet to peer 1
(redacted:51820)

Here are some more details:

this is a 6 node network with one node acting as gateway, I set the
routes via PreUp/PostUp and remove them via PreDown/PostDown.

root at bladerunner-2:~# uname -a
Linux bladerunner-2 4.4.0-145-generic #171-Ubuntu SMP Tue Mar 26
12:43:40 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux

root at bladerunner-2:~# ip link show wg0
3: wg0: <POINTOPOINT,NOARP,UP,LOWER_UP> mtu 1420 qdisc noqueue state
UNKNOWN mode DEFAULT group default qlen 1
    link/none
    
root at bladerunner-2:~# dpkg -l | grep wireguard
ii  wireguard-dkms                  
0.0.20190406-wg1~xenial                    all          fast, modern,
secure kernel VPN tunnel (DKMS version)
ii  wireguard-tools                 
0.0.20190406-wg1~xenial                    amd64        fast, modern,
secure kernel VPN tunnel (userland utilities)

Please help me debugging this Issue, I'm a big fan of Wireguard and
would love to use it with mode projects

Have a nice Day,
Joshi