From wireguard at bulletin.elitemail.org Sun Oct 2 03:03:51 2022 From: wireguard at bulletin.elitemail.org (wireguard at bulletin.elitemail.org) Date: Sun, 02 Oct 2022 03:03:51 +0000 Subject: APK outside of Play Store? In-Reply-To: <286505c7-67b8-29dd-3cff-570851c3169c@tempforever.com> References: <517b5e80-d3ac-4829-9096-1415ecad5aa4@www.fastmail.com> <286505c7-67b8-29dd-3cff-570851c3169c@tempforever.com> Message-ID: <17e101ab-1434-43cb-af2c-437623bfecbf@app.fastmail.com> F-Droid builds on EOL Debian Stretch using an outdated toolchain and dependencies is not the way. An official build on a supported non-EOL OS signed by first-party development is. On Wed, Sep 28, 2022, at 4:34 PM, tempforever wrote: > wireguard at bulletin.elitemail.org wrote: >> For users who prefer to avoid Play Store as a delivery channel, is there an official pre-built APK available? Such users are typically steered towards APKPure/APKMirror/F-Droid with questionabl authenticity and (in the case of F-Droid) the prospect of old build dependencies built on an EOL OS (Strech). Aurora is an option, though a dev provided build (with accompanying checksum/signing cert fingerprint) would be preferable. >> > It's on the wireguard installation/download page > https://www.wireguard.com/install/#android-play-store-f-droid > > The f-droid link is https://f-droid.org/en/packages/com.wireguard.android/ From Jason at zx2c4.com Sun Oct 2 08:56:59 2022 From: Jason at zx2c4.com (Jason A. Donenfeld) Date: Sun, 2 Oct 2022 10:56:59 +0200 Subject: APK outside of Play Store? In-Reply-To: <17e101ab-1434-43cb-af2c-437623bfecbf@app.fastmail.com> References: <517b5e80-d3ac-4829-9096-1415ecad5aa4@www.fastmail.com> <286505c7-67b8-29dd-3cff-570851c3169c@tempforever.com> <17e101ab-1434-43cb-af2c-437623bfecbf@app.fastmail.com> Message-ID: On Sun, Oct 02, 2022 at 03:03:51AM +0000, wireguard at bulletin.elitemail.org wrote: > F-Droid builds on EOL Debian Stretch using an outdated toolchain and > dependencies is not the way. An official build on a supported non-EOL > OS signed by first-party development is. Is there actually something inferior about the apks it generates? Yes, we all know fdroid's infra is crumbly. But is there something wrong with its output files? Can you point to something technically substantive there? If you can identify a real technical problem, maybe there's something I can do about that, by working with the fdroid developers, or otherwise find some way. In the past, they've been able to bump NDK/SDK revisions and whatnot without too much hassle. And these days, all is usually fine so long as we stick with the non-beta releases (that is, the open source releases) of AGP. But maybe now you've identified a new problem? On the other hand, if you cannot identity a real technical problem, I'm not sure I have much to comment on. Jason From weichen302 at zoho.com Sun Oct 2 23:13:22 2022 From: weichen302 at zoho.com (Wei Chen) Date: Sun, 2 Oct 2022 18:13:22 -0500 Subject: Iptables WireGuard obfuscation extension In-Reply-To: <20220928163356.183baef9@nvm> References: <183272e3203.12ada1173180167.8469340361616836666@zoho.com> <20220928163356.183baef9@nvm> Message-ID: Hi Roman, > The "Usage" section speaks of "server" and "client". However in the WG world > there's not really a server or client per se, but all WG network members are > peers. As such, is it possible to propose an universal set of iptables rules > that would be fine to use on any network node? > > As I understand, all INPUT packets to our local --dport need to be --unobfs, > and all OUTPUT packets from us to any other node need to be --obfs. Right? > Yes, you are right. Besides unobfs/obfs INPUT/OUTPUT chain for a local WG installation, one can also use it on a Linux gateway, mangle the FORWARD chain. I haven't test it but it should work. Wei From weichen302 at zoho.com Sun Oct 2 23:35:35 2022 From: weichen302 at zoho.com (Wei Chen) Date: Sun, 2 Oct 2022 18:35:35 -0500 Subject: Iptables WireGuard obfuscation extension In-Reply-To: References: <183272e3203.12ada1173180167.8469340361616836666@zoho.com> Message-ID: Hi Jason, Thank you for the suggestions! > - Instead of using siphash, if you can make use of 64 bytes of > randomness at a time, you might be able to get away with chacha8 (or > even lower). The input to chacha20 is typically a 256 bit key and a > nonce, but because we don't care about the cryptographic security here > -- wireguard handles that part -- we can play fast and lose, and make > our threat model, "would be too computationally complex to detect in > real time". Things become quite fun when you don't need real crypto. > To that end, we could perhaps get away with using chacha8 instead of > chacha20, and doing so with a 128-bit key. This then provides lots of > input to chacha: > It does need more than 16 bytes. Currently it gets away with it by borrowing one byte from where supposed to XOR with reserved field. If the all zeros reserved field changes as WG protocol evolving, then this siphash128 approach will stop working. > - get_random_bytes() is slow if all you need is a byte at a time. That > computes 96 bytes and then throws away 88 bytes of it. Instead, you > can use get_random_u32(), which batches, and throw away 3 bytes. Or, I > think I'll add to kernel 6.1 get_random_u8(), which will waste > nothing. > > But actually, do you really need to do that? Can't you just run chacha > or siphash or whatever super fast non-cryptographic thing you have, > and just have an incrementing nonce? Or, better, since those keepalive > messages already have a suitably random poly1305 tag, just run siphash > on that, and discard if the resultant first byte is high/low/whatever. > I have a feeling get_random_bytes() might not fast, but have no idea I have wasted so many bytes. Thank you! I will try get_random_u32() first. > - If this is to ever go upstream, you might want to add a `--obfs-type > N` parameter to the XT userspace library and the IPC struct, and make > it mandatory. To begin, everybody would use `--obfs-type 1`, since > Added to todo list. Wei From michael.adm at gmail.com Wed Oct 5 13:45:05 2022 From: michael.adm at gmail.com (Michael Pro) Date: Wed, 5 Oct 2022 16:45:05 +0300 Subject: Wireguard not compile, - udp: typedef udp tunneling functions to functions, not pointers Message-ID: https://reviews.freebsd.org/D36724 https://cgit.freebsd.org/src/commit/?id=bb77f0c2049311f0661c2493838d81a5a66c449c #> uname -K 1400072 in wireguard-freebsd/src/if_wg.c: rc = udp_set_kernel_tunneling(so4, (udp_tun_func_t)wg_input, NULL, sc); change to: rc = udp_set_kernel_tunneling(so4, wg_input, NULL, sc); From syzbot+4d46c43d81c3bd155060 at syzkaller.appspotmail.com Tue Oct 11 08:44:36 2022 From: syzbot+4d46c43d81c3bd155060 at syzkaller.appspotmail.com (syzbot) Date: Tue, 11 Oct 2022 01:44:36 -0700 Subject: [syzbot] upstream test error: WARNING in wg_cpumask_next_online Message-ID: <0000000000003b373905eabe492f@google.com> Hello, syzbot found the following issue on: HEAD commit: 60bb8154d1d7 Merge tag 'xfs-6.1-for-linus' of git://git.ke.. git tree: upstream console output: https://syzkaller.appspot.com/x/log.txt?x=10ce0ea4880000 kernel config: https://syzkaller.appspot.com/x/.config?x=184ee7ce557d8550 dashboard link: https://syzkaller.appspot.com/bug?extid=4d46c43d81c3bd155060 compiler: aarch64-linux-gnu-gcc (Debian 10.2.1-6) 10.2.1 20210110, GNU ld (GNU Binutils for Debian) 2.35.2 userspace arch: arm64 IMPORTANT: if you fix the issue, please add the following tag to the commit: Reported-by: syzbot+4d46c43d81c3bd155060 at syzkaller.appspotmail.com ------------[ cut here ]------------ WARNING: CPU: 1 PID: 2267 at include/linux/cpumask.h:110 wg_cpumask_next_online+0x1c0/0x2c0 drivers/net/wireguard/queueing.h:132 Modules linked in: CPU: 1 PID: 2267 Comm: kworker/u4:7 Tainted: G W 6.0.0-syzkaller-10822-g60bb8154d1d7 #0 Hardware name: linux,dummy-virt (DT) Workqueue: wg-kex-wg2 wg_packet_handshake_send_worker pstate: 60000005 (nZCv daif -PAN -UAO -TCO -DIT -SSBS BTYPE=--) pc : cpu_max_bits_warn include/linux/cpumask.h:110 [inline] pc : cpumask_check include/linux/cpumask.h:117 [inline] pc : cpumask_next include/linux/cpumask.h:178 [inline] pc : wg_cpumask_next_online+0x1c0/0x2c0 drivers/net/wireguard/queueing.h:133 lr : wg_packet_receive+0x978/0x1560 drivers/net/wireguard/receive.c:568 sp : ffff800010ab7480 x29: ffff800010ab7480 x28: 0000000000000001 x27: 1fffe000015d2219 x26: 0000000000000000 x25: ffff80000de5c000 x24: 0000000000000000 x23: 0000000000000003 x22: ffff80000de5cb68 x21: 0000000000000001 x20: ffff00000ae910c8 x19: ffff80000de5cd50 x18: 000000002b1800ff x17: ffff80005cbe4000 x16: ffff800010ab8000 x15: ffff000018c3eca8 x14: 1ffff00002156e68 x13: 0000000000000000 x12: ffff6000015d2291 x11: 1fffe000015d2290 x10: ffff6000015d2290 x9 : dfff800000000000 x8 : ffff00000ae91483 x7 : 00009ffffea2dd70 x6 : 0000000000000001 x5 : ffff00000ae91480 x4 : ffff700001bcb9aa x3 : dfff800000000000 x2 : 0000000000000002 x1 : 0000000000000002 x0 : 0000000000000001 Call trace: wg_cpumask_next_online+0x1c0/0x2c0 drivers/net/wireguard/queueing.h:132 wg_packet_receive+0x978/0x1560 drivers/net/wireguard/receive.c:568 wg_receive+0x58/0xb0 drivers/net/wireguard/socket.c:326 udpv6_queue_rcv_one_skb+0x8f4/0x17c0 net/ipv6/udp.c:714 udpv6_queue_rcv_skb+0x134/0x7e0 net/ipv6/udp.c:775 udp6_unicast_rcv_skb+0xe8/0x270 net/ipv6/udp.c:918 __udp6_lib_rcv+0x8a4/0x2330 net/ipv6/udp.c:1003 udpv6_rcv+0x1c/0x2c net/ipv6/udp.c:1111 ip6_protocol_deliver_rcu+0x154/0x14f0 net/ipv6/ip6_input.c:439 ip6_input_finish+0x108/0x220 net/ipv6/ip6_input.c:484 NF_HOOK include/linux/netfilter.h:302 [inline] NF_HOOK include/linux/netfilter.h:296 [inline] ip6_input+0xbc/0x2b0 net/ipv6/ip6_input.c:493 dst_input include/net/dst.h:455 [inline] ip6_rcv_finish net/ipv6/ip6_input.c:79 [inline] NF_HOOK include/linux/netfilter.h:302 [inline] NF_HOOK include/linux/netfilter.h:296 [inline] ipv6_rcv+0x39c/0x47c net/ipv6/ip6_input.c:309 __netif_receive_skb_one_core+0xf4/0x170 net/core/dev.c:5485 __netif_receive_skb+0x24/0x184 net/core/dev.c:5599 process_backlog+0x24c/0x6b0 net/core/dev.c:5927 __napi_poll+0x94/0x3a4 net/core/dev.c:6494 napi_poll net/core/dev.c:6561 [inline] net_rx_action+0x78c/0xb60 net/core/dev.c:6672 _stext+0x28c/0x107c ____do_softirq+0x10/0x20 arch/arm64/kernel/irq.c:79 call_on_irq_stack+0x2c/0x54 arch/arm64/kernel/entry.S:889 do_softirq_own_stack+0x1c/0x30 arch/arm64/kernel/irq.c:84 do_softirq.part.0+0xd0/0xf4 kernel/softirq.c:472 do_softirq kernel/softirq.c:464 [inline] __local_bh_enable_ip+0x50c/0x5d0 kernel/softirq.c:396 __raw_read_unlock_bh include/linux/rwlock_api_smp.h:257 [inline] _raw_read_unlock_bh+0x54/0x64 kernel/locking/spinlock.c:284 wg_socket_send_skb_to_peer+0xf0/0x190 drivers/net/wireguard/socket.c:184 wg_socket_send_buffer_to_peer+0x110/0x160 drivers/net/wireguard/socket.c:200 wg_packet_send_handshake_initiation+0x1a8/0x274 drivers/net/wireguard/send.c:40 wg_packet_handshake_send_worker+0x1c/0x34 drivers/net/wireguard/send.c:51 process_one_work+0x780/0x184c kernel/workqueue.c:2289 worker_thread+0x3cc/0xc40 kernel/workqueue.c:2436 kthread+0x23c/0x2a0 kernel/kthread.c:376 ret_from_fork+0x10/0x20 arch/arm64/kernel/entry.S:860 irq event stamp: 6169 hardirqs last enabled at (6168): [] __local_bh_enable_ip+0x1e4/0x5d0 kernel/softirq.c:401 hardirqs last disabled at (6169): [] el1_dbg+0x24/0x80 arch/arm64/kernel/entry-common.c:404 softirqs last enabled at (6160): [] wg_socket_send_skb_to_peer+0xf0/0x190 drivers/net/wireguard/socket.c:184 softirqs last disabled at (6161): [] ____do_softirq+0x10/0x20 arch/arm64/kernel/irq.c:79 ---[ end trace 0000000000000000 ]--- --- This report is generated by a bot. It may contain errors. See https://goo.gl/tpsmEJ for more information about syzbot. syzbot engineers can be reached at syzkaller at googlegroups.com. syzbot will keep track of this issue. See: https://goo.gl/tpsmEJ#status for how to communicate with syzbot. From post at steffenvogel.de Sun Oct 16 09:35:04 2022 From: post at steffenvogel.de (Steffen Vogel) Date: Sun, 16 Oct 2022 11:35:04 +0200 Subject: [PATCH] wireguard-go/device: add new handshake handler and keylog writer In-Reply-To: <4d-63121e80-3-6ef3e80@7869515> References: <4d-63121e80-3-6ef3e80@7869515> Message-ID: <146011B3-5B31-422C-B23E-3FC45969CA90@steffenvogel.de> Does somebody know who the current maintainer of wireguard-go is? My patch seems to be forgotten ? ?On 04.09.22, 19:00, "WireGuard on behalf of Steffen Vogel" wrote: (This path is also tracked as PR: https://github.com/WireGuard/wireguard-go/pull/56) This change adds support for a new environment variable 'WG_KEYLOGFILE' in resemblance to the 'SSLKEYLOGFILE' environment variable used by curl, Chrome & Firefox to log ephemeral TLS encryption keys When set, wireguard-go will log ephemeral keys generated during each handshake to a file specified by the environment variable in the WireGuard key log format. The format used is the same as then one generated by the extract-handshakes.sh script. See also: - https://git.zx2c4.com/wireguard-tools/tree/contrib/extract-handshakes - https://wiki.wireshark.org/WireGuard#key-log-format - https://everything.curl.dev/usingcurl/tls/sslkeylogfile Signed-off-by: Steffen Vogel post at steffenvogel.de From wireguard-mail at chil.at Wed Oct 19 22:47:04 2022 From: wireguard-mail at chil.at (Christoph Loesch) Date: Thu, 20 Oct 2022 00:47:04 +0200 Subject: Source IP selection or multi-home or multi-interface issue - solved with source-based route - was: Re: Wireguard source ip selection issue with multi interfaces In-Reply-To: <60b5d439-e117-f3c7-3a6b-8ce46bc48fc2@chil.at> References: <60b5d439-e117-f3c7-3a6b-8ce46bc48fc2@chil.at> Message-ID: Now I've read enough (rather old and also pretty recent) posts about that topic and I want to help finally fix this issue. Not just because it bugs our network too after we optimized/simplified our router-setup but also because it seems over the years many users seem to be affected. Until now every thread I've read ends with an open question rather than a solution. In Jason's post here from 2017 I found his script that should reproduce the issue, I tried it and could not reproduce the issue with that script.. To be honest I do not fully understand it to modify it to reproduce the issue in my environment. Here I posted one example and the (temporary) solution by deleting the route that wireguard had set and re-creating the same route but defining the (correct) IP as src. https://lists.zx2c4.com/pipermail/wireguard/2021-November/007324.html How could I help to fix this issue? If it helps, I could give full access to a test-router which has the same setup as all other routers in our network. Kind regards Am 23.11.2021 um 14:52 schrieb Christoph Loesch: > Hi, > > Is this related to my issue with using wrong source ip on outgoing connections? > (https://lists.zx2c4.com/pipermail/wireguard/2021-November/007307.html) > > Which bug would that be? Would be interesting in more detail to understand the issue. > I'm having a hard time to reproduce (or somehow "fix") the issue I am facing, since this issue is on only 5 devices from over 20 devices where all are configured same. > > The configuration or better all routes do not even have any metric defined so I am unsure what to add/remove to change the behaviour. > Also in my configuration there are multiple interfaces (br0,br1,eth0,eth1,etc..) as it is a router but there are no multiple default routes. > Comparing perfectly fine working devices to one of those five devices which have this issue I am not able to find a difference in IP-config or routes. > Even kernel and firmware versions are all same. > > The only difference I see is while looking at tcpdump where a simple ping to the wireguard server's internal IP (172.27.0.1/24 in my case) originates with the public IP of the device. > > If it is a known bug, it would be nice to know what (and if) I could try to fix and possibly confirm the bug. Or even better beeing able to reproduce the issue. > > Thanks and kind regards, > Christoph > > Am 19.11.2021 um 14:10 schrieb ??: >> Hi, >> As Jason said in mailing list >> (https://lists.zx2c4.com/pipermail/wireguard/2017-November/002018.html) >> years ago, if wireguard reply the client with wrong src ip, then it is >> a bug. And now I'm trying to set up a minimal configuration. >> >> Server info: >> >> Ubuntu 20.04 in VMware with two Host-only network adapters: >> >> wacke at Ubuntu:/etc$ lsb_release -a >> No LSB modules are available. >> Distributor ID: Ubuntu >> Description: Ubuntu 20.04.3 LTS >> Release: 20.04 >> Codename: focal >> >> wacke at Ubuntu:/etc$ uname -a >> Linux Ubuntu 5.14.8-051408-generic #202109280455 SMP Tue Sep 28 >> 04:57:31 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux >> >> wacke at Ubuntu:/etc$ sudo ip route show >> 169.254.0.0/16 dev ens33 scope link metric 1000 >> 172.17.0.0/16 dev docker0 proto kernel scope link src 172.17.0.1 linkdown >> 192.168.187.0/24 dev ens33 proto kernel scope link src 192.168.187.129 >> metric 100 >> 192.168.187.0/24 dev ens35 proto kernel scope link src 192.168.187.128 >> metric 101 >> 192.168.199.0/24 dev wg0 proto kernel scope link src 192.168.199.1 >> >> wacke at Ubuntu:/etc$ sudo cat /etc/wireguard/wg0.conf >> [Interface] >> PrivateKey = oLLeTlvwWsfGoyQpDwZo0k7AsMf0LkkfycGPwFjamEA= >> ListenPort = 5999 >> >> [Peer] >> PublicKey = JYvK2/FBbbSrDbkH29ejWejkBJS3ch/SFtyZwNf5O1c= >> AllowedIPs = 192.168.199.2/32 >> PersistentKeepalive = 25 >> >> [Peer] >> PublicKey = keIPXZFgvB79biV74kGc5R9vAHpzyVyQHci8KBSDIUw= >> AllowedIPs = 192.168.199.3/32 >> PersistentKeepalive = 25 >> >> wacke at Ubuntu:/etc$ sudo wg setconf wg0 /etc/wireguard/wg0.conf >> wacke at Ubuntu:/etc$ sudo ip address add dev wg0 192.168.199.1/24 >> wacke at Ubuntu:/etc$ sudo ip link set up dev wg0 >> wacke at Ubuntu:/etc$ sudo wg show >> interface: wg0 >> ?? public key: Vkdx0MhQkc9anLaUehR/Df1zuKjxceflxMCAeAaCWCo= >> ?? private key: (hidden) >> ?? listening port: 5999 >> >> peer: JYvK2/FBbbSrDbkH29ejWejkBJS3ch/SFtyZwNf5O1c= >> ?? endpoint: 192.168.187.1:35560 >> ?? allowed ips: 192.168.199.2/32 >> ?? transfer: 2.89 KiB received, 1.80 KiB sent >> ?? persistent keepalive: every 25 seconds >> >> peer: keIPXZFgvB79biV74kGc5R9vAHpzyVyQHci8KBSDIUw= >> ?? allowed ips: 192.168.199.3/32 >> ?? persistent keepalive: every 25 seconds >> >> Client info: >> >> HOME-Server:/NAS/Software/Software # uname -a >> Linux HOME-Server 5.14.14-2-default #1 SMP Thu Oct 21 05:05:03 UTC >> 2021 (2b5383f) x86_64 x86_64 x86_64 GNU/Linux >> >> HOME-Server:/NAS/Software/Software # cat wg0.conf >> [Interface] >> PrivateKey = 4Pw1cetxd9TdfH3TSab+9UcRBlHwZ1o/vmrUgkerZls= >> >> [Peer] >> PublicKey = Vkdx0MhQkc9anLaUehR/Df1zuKjxceflxMCAeAaCWCo= >> Endpoint = 192.168.187.128:5999 >> AllowedIPs = 192.168.199.0/24 >> PersistentKeepalive = 25 >> >> HOME-Server:/NAS/Software/Software # ip link add wg0 type wireguard >> HOME-Server:/NAS/Software/Software # ip address add dev wg0 192.168.199.2/24 >> HOME-Server:/NAS/Software/Software # wg setconf wg0 wg0.conf >> HOME-Server:/NAS/Software/Software # ip link set wg0 up >> HOME-Server:/NAS/Software/Software # wg show >> interface: wg0 >> ?? public key: JYvK2/FBbbSrDbkH29ejWejkBJS3ch/SFtyZwNf5O1c= >> ?? private key: (hidden) >> ?? listening port: 53058 >> >> peer: Vkdx0MhQkc9anLaUehR/Df1zuKjxceflxMCAeAaCWCo= >> ?? endpoint: 192.168.187.128:5999 >> ?? allowed ips: 192.168.199.0/24 >> ?? transfer: 0 B received, 59.26 KiB sent >> ?? persistent keepalive: every 25 seconds >> >> >> tcpdump of client side: >> >> HOME-Server:/NAS/Software/Software # tcpdump -i any -vn "(dst port 5999)" >> tcpdump: data link type LINUX_SLL2 >> tcpdump: listening on any, link-type LINUX_SLL2 (Linux cooked v2), >> snapshot length 262144 bytes >> 22:10:37.794301 vmnet1 Out IP (tos 0x88, ttl 64, id 54724, offset 0, >> flags [none], proto UDP (17), length 176) >> ???? 192.168.187.1.53058 > 192.168.187.128.5999: UDP, length 148 >> 22:10:43.170306 vmnet1 Out IP (tos 0x88, ttl 64, id 55327, offset 0, >> flags [none], proto UDP (17), length 176) >> ???? 192.168.187.1.53058 > 192.168.187.128.5999: UDP, length 148 >> 22:10:48.546281 vmnet1 Out IP (tos 0x88, ttl 64, id 56142, offset 0, >> flags [none], proto UDP (17), length 176) >> ???? 192.168.187.1.53058 > 192.168.187.128.5999: UDP, length 148 >> 22:10:53.670343 vmnet1 Out IP (tos 0x88, ttl 64, id 57053, offset 0, >> flags [none], proto UDP (17), length 176) >> ???? 192.168.187.1.53058 > 192.168.187.128.5999: UDP, length 148 >> 22:11:18.754308 vmnet1 Out IP (tos 0x88, ttl 64, id 58505, offset 0, >> flags [none], proto UDP (17), length 176) >> ???? 192.168.187.1.53058 > 192.168.187.128.5999: UDP, length 148 >> 22:11:24.130286 vmnet1 Out IP (tos 0x88, ttl 64, id 58621, offset 0, >> flags [none], proto UDP (17), length 176) >> ???? 192.168.187.1.53058 > 192.168.187.128.5999: UDP, length 148 >> 22:11:29.506305 vmnet1 Out IP (tos 0x88, ttl 64, id 58810, offset 0, >> flags [none], proto UDP (17), length 176) >> ???? 192.168.187.1.53058 > 192.168.187.128.5999: UDP, length 148 >> 22:11:34.882326 vmnet1 Out IP (tos 0x88, ttl 64, id 59444, offset 0, >> flags [none], proto UDP (17), length 176) >> ???? 192.168.187.1.53058 > 192.168.187.128.5999: UDP, length 148 >> ^C >> 8 packets captured >> 8 packets received by filter >> 0 packets dropped by kernel >> >> tcpdump of server side: >> >> wacke at Ubuntu:~$ sudo tcpdump -i any -vn "(src port 5999)" >> tcpdump: listening on any, link-type LINUX_SLL (Linux cooked v1), >> capture size 262144 bytes >> 22:10:11.754862 IP (tos 0x88, ttl 64, id 22024, offset 0, flags >> [none], proto UDP (17), length 120) >> ???? 192.168.187.129.5999 > 192.168.187.1.53058: UDP, length 92 >> 22:10:16.874400 IP (tos 0x88, ttl 64, id 22399, offset 0, flags >> [none], proto UDP (17), length 120) >> ???? 192.168.187.129.5999 > 192.168.187.1.53058: UDP, length 92 >> 22:10:21.994433 IP (tos 0x88, ttl 64, id 23057, offset 0, flags >> [none], proto UDP (17), length 120) >> ???? 192.168.187.129.5999 > 192.168.187.1.53058: UDP, length 92 >> 22:10:27.370558 IP (tos 0x88, ttl 64, id 23912, offset 0, flags >> [none], proto UDP (17), length 120) >> ???? 192.168.187.129.5999 > 192.168.187.1.53058: UDP, length 92 >> 22:10:32.494605 IP (tos 0x88, ttl 64, id 24944, offset 0, flags >> [none], proto UDP (17), length 120) >> ???? 192.168.187.129.5999 > 192.168.187.1.53058: UDP, length 92 >> 22:10:37.610779 IP (tos 0x88, ttl 64, id 25327, offset 0, flags >> [none], proto UDP (17), length 120) >> ???? 192.168.187.129.5999 > 192.168.187.1.53058: UDP, length 92 >> 22:10:42.986834 IP (tos 0x88, ttl 64, id 25498, offset 0, flags >> [none], proto UDP (17), length 120) >> ???? 192.168.187.129.5999 > 192.168.187.1.53058: UDP, length 92 >> 22:10:48.362908 IP (tos 0x88, ttl 64, id 26521, offset 0, flags >> [none], proto UDP (17), length 120) >> ???? 192.168.187.129.5999 > 192.168.187.1.53058: UDP, length 92 >> 22:10:53.486942 IP (tos 0x88, ttl 64, id 27214, offset 0, flags >> [none], proto UDP (17), length 120) >> ???? 192.168.187.129.5999 > 192.168.187.1.53058: UDP, length 92 >> 22:11:18.571437 IP (tos 0x88, ttl 64, id 31112, offset 0, flags >> [none], proto UDP (17), length 120) >> ???? 192.168.187.129.5999 > 192.168.187.1.53058: UDP, length 92 >> 22:11:23.947460 IP (tos 0x88, ttl 64, id 31897, offset 0, flags >> [none], proto UDP (17), length 120) >> ???? 192.168.187.129.5999 > 192.168.187.1.53058: UDP, length 92 >> 22:11:29.323608 IP (tos 0x88, ttl 64, id 32715, offset 0, flags >> [none], proto UDP (17), length 120) >> ???? 192.168.187.129.5999 > 192.168.187.1.53058: UDP, length 92 >> 22:11:34.699676 IP (tos 0x88, ttl 64, id 33840, offset 0, flags >> [none], proto UDP (17), length 120) >> ???? 192.168.187.129.5999 > 192.168.187.1.53058: UDP, length 92 >> ^C >> 13 packets captured >> 13 packets received by filter >> 0 packets dropped by kernel >> >> As the tcpdump log showed, the wireguard server always chose the ip >> with lowest metric as source ip (192.168.187.129) to reply to the >> client (while the client tried to connect 192.168.187.128). >> >> Hope the info above can help to debug. Many thanks. From wireguard-mail at chil.at Wed Oct 19 22:56:40 2022 From: wireguard-mail at chil.at (Christoph Loesch) Date: Thu, 20 Oct 2022 00:56:40 +0200 Subject: Source IP selection or multi-home or multi-interface issue - solved with source-based route - was: Re: Wireguard source ip selection issue with multi interfaces In-Reply-To: <60b5d439-e117-f3c7-3a6b-8ce46bc48fc2@chil.at> References: <60b5d439-e117-f3c7-3a6b-8ce46bc48fc2@chil.at> Message-ID: <19530c0e-6e5e-f47c-8490-8338512f2cb2@chil.at> Now I've read enough (rather old and also pretty recent) posts about that topic and I want to help finally fix this issue. Not just because it bugs our network too after we optimized/simplified our router-setup but also because it seems over the years many users seem to beaffected. Until now every thread I've read ends with an open question rather than a solution. In Jason's post here from 2017 I found his script that should reproduce theissue, I tried it and could not reproduce the issue with that script.. https://lists.zx2c4.com/pipermail/wireguard/2017-November/002092.html To be honest I do not fully understand it to modify it to reproduce the issue in my environment. Here I posted one example and the (temporary) solution by deleting the route that wireguard had set and re-creating the same route but defining the (correct) IP as src. https://lists.zx2c4.com/pipermail/wireguard/2021-November/007324.html How could I help to fix this issue? If it helps, I could give full access to a test-router which has the same setup as all other routers in our network. Kind regards Am 23.11.2021 um 14:52 schrieb Christoph Loesch: > Hi, > > Is this related to my issue with using wrong source ip on outgoing connections? > (https://lists.zx2c4.com/pipermail/wireguard/2021-November/007307.html) > > Which bug would that be? Would be interesting in more detail to understand the issue. > I'm having a hard time to reproduce (or somehow "fix") the issue I am facing, since this issue is on only 5 devices from over 20 devices where all are configured same. > > The configuration or better all routes do not even have any metric defined so I am unsure what to add/remove to change the behaviour. > Also in my configuration there are multiple interfaces (br0,br1,eth0,eth1,etc..) as it is a router but there are no multiple default routes. > Comparing perfectly fine working devices to one of those five devices which have this issue I am not able to find a difference in IP-config or routes. > Even kernel and firmware versions are all same. > > The only difference I see is while looking at tcpdump where a simple pingto the wireguard server's internal IP (172.27.0.1/24 in my case) originates with the public IP of the device. > > If it is a known bug, it would be nice to know what (and if) I could try to fix and possibly confirm the bug. Or even better beeing able to reproduce the issue. > > Thanks and kind regards, > Christoph > > Am 19.11.2021 um 14:10 schrieb ??: >> Hi, >> As Jason said in mailing list >> (https://lists.zx2c4.com/pipermail/wireguard/2017-November/002018.html) >> years ago, if wireguard reply the client with wrong src ip, then it is >> a bug. And now I'm trying to set up a minimal configuration. >> >> Server info: >> >> Ubuntu 20.04 in VMware with two Host-only network adapters: >> >> wacke at Ubuntu:/etc$ lsb_release -a >> No LSB modules are available. >> Distributor ID: Ubuntu >> Description: Ubuntu 20.04.3 LTS >> Release: 20.04 >> Codename: focal >> >> wacke at Ubuntu:/etc$ uname -a >> Linux Ubuntu 5.14.8-051408-generic #202109280455 SMP Tue Sep 28 >> 04:57:31 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux >> >> wacke at Ubuntu:/etc$ sudo ip route show >> 169.254.0.0/16 dev ens33 scope link metric 1000 >> 172.17.0.0/16 dev docker0 proto kernel scope link src 172.17.0.1 linkdown >> 192.168.187.0/24 dev ens33 proto kernel scope link src 192.168.187.129 >> metric 100 >> 192.168.187.0/24 dev ens35 proto kernel scope link src 192.168.187.128 >> metric 101 >> 192.168.199.0/24 dev wg0 proto kernel scope link src 192.168.199.1 >> >> wacke at Ubuntu:/etc$ sudo cat /etc/wireguard/wg0.conf >> [Interface] >> PrivateKey = oLLeTlvwWsfGoyQpDwZo0k7AsMf0LkkfycGPwFjamEA= >> ListenPort = 5999 >> >> [Peer] >> PublicKey = JYvK2/FBbbSrDbkH29ejWejkBJS3ch/SFtyZwNf5O1c= >> AllowedIPs = 192.168.199.2/32 >> PersistentKeepalive = 25 >> >> [Peer] >> PublicKey = keIPXZFgvB79biV74kGc5R9vAHpzyVyQHci8KBSDIUw= >> AllowedIPs = 192.168.199.3/32 >> PersistentKeepalive = 25 >> >> wacke at Ubuntu:/etc$ sudo wg setconf wg0 /etc/wireguard/wg0.conf >> wacke at Ubuntu:/etc$ sudo ip address add dev wg0 192.168.199.1/24 >> wacke at Ubuntu:/etc$ sudo ip link set up dev wg0 >> wacke at Ubuntu:/etc$ sudo wg show >> interface: wg0 >> ?? public key: Vkdx0MhQkc9anLaUehR/Df1zuKjxceflxMCAeAaCWCo= >> ?? private key: (hidden) >> ?? listening port: 5999 >> >> peer: JYvK2/FBbbSrDbkH29ejWejkBJS3ch/SFtyZwNf5O1c= >> ?? endpoint: 192.168.187.1:35560 >> ?? allowed ips: 192.168.199.2/32 >> ?? transfer: 2.89 KiB received, 1.80 KiB sent >> ?? persistent keepalive: every 25 seconds >> >> peer: keIPXZFgvB79biV74kGc5R9vAHpzyVyQHci8KBSDIUw= >> ?? allowed ips: 192.168.199.3/32 >> ?? persistent keepalive: every 25 seconds >> >> Client info: >> >> HOME-Server:/NAS/Software/Software # uname -a >> Linux HOME-Server 5.14.14-2-default #1 SMP Thu Oct 21 05:05:03 UTC >> 2021 (2b5383f) x86_64 x86_64 x86_64 GNU/Linux >> >> HOME-Server:/NAS/Software/Software # cat wg0.conf >> [Interface] >> PrivateKey = 4Pw1cetxd9TdfH3TSab+9UcRBlHwZ1o/vmrUgkerZls= >> >> [Peer] >> PublicKey = Vkdx0MhQkc9anLaUehR/Df1zuKjxceflxMCAeAaCWCo= >> Endpoint = 192.168.187.128:5999 >> AllowedIPs = 192.168.199.0/24 >> PersistentKeepalive = 25 >> >> HOME-Server:/NAS/Software/Software # ip link add wg0 type wireguard >> HOME-Server:/NAS/Software/Software # ip address add dev wg0 192.168.199.2/24 >> HOME-Server:/NAS/Software/Software # wg setconf wg0 wg0.conf >> HOME-Server:/NAS/Software/Software # ip link set wg0 up >> HOME-Server:/NAS/Software/Software # wg show >> interface: wg0 >> ?? public key: JYvK2/FBbbSrDbkH29ejWejkBJS3ch/SFtyZwNf5O1c= >> ?? private key: (hidden) >> ?? listening port: 53058 >> >> peer: Vkdx0MhQkc9anLaUehR/Df1zuKjxceflxMCAeAaCWCo= >> ?? endpoint: 192.168.187.128:5999 >> ?? allowed ips: 192.168.199.0/24 >> ?? transfer: 0 B received, 59.26 KiB sent >> ?? persistent keepalive: every 25 seconds >> >> >> tcpdump of client side: >> >> HOME-Server:/NAS/Software/Software # tcpdump -i any -vn "(dst port 5999)" >> tcpdump: data link type LINUX_SLL2 >> tcpdump: listening on any, link-type LINUX_SLL2 (Linux cooked v2), >> snapshot length 262144 bytes >> 22:10:37.794301 vmnet1 Out IP (tos 0x88, ttl 64, id 54724, offset 0, >> flags [none], proto UDP (17), length 176) >> ???? 192.168.187.1.53058 > 192.168.187.128.5999: UDP, length 148 >> 22:10:43.170306 vmnet1 Out IP (tos 0x88, ttl 64, id 55327, offset 0, >> flags [none], proto UDP (17), length 176) >> ???? 192.168.187.1.53058 > 192.168.187.128.5999: UDP, length 148 >> 22:10:48.546281 vmnet1 Out IP (tos 0x88, ttl 64, id 56142, offset 0, >> flags [none], proto UDP (17), length 176) >> ???? 192.168.187.1.53058 > 192.168.187.128.5999: UDP, length 148 >> 22:10:53.670343 vmnet1 Out IP (tos 0x88, ttl 64, id 57053, offset 0, >> flags [none], proto UDP (17), length 176) >> ???? 192.168.187.1.53058 > 192.168.187.128.5999: UDP, length 148 >> 22:11:18.754308 vmnet1 Out IP (tos 0x88, ttl 64, id 58505, offset 0, >> flags [none], proto UDP (17), length 176) >> ???? 192.168.187.1.53058 > 192.168.187.128.5999: UDP, length 148 >> 22:11:24.130286 vmnet1 Out IP (tos 0x88, ttl 64, id 58621, offset 0, >> flags [none], proto UDP (17), length 176) >> ???? 192.168.187.1.53058 > 192.168.187.128.5999: UDP, length 148 >> 22:11:29.506305 vmnet1 Out IP (tos 0x88, ttl 64, id 58810, offset 0, >> flags [none], proto UDP (17), length 176) >> ???? 192.168.187.1.53058 > 192.168.187.128.5999: UDP, length 148 >> 22:11:34.882326 vmnet1 Out IP (tos 0x88, ttl 64, id 59444, offset 0, >> flags [none], proto UDP (17), length 176) >> ???? 192.168.187.1.53058 > 192.168.187.128.5999: UDP, length 148 >> ^C >> 8 packets captured >> 8 packets received by filter >> 0 packets dropped by kernel >> >> tcpdump of server side: >> >> wacke at Ubuntu:~$ sudo tcpdump -i any -vn "(src port 5999)" >> tcpdump: listening on any, link-type LINUX_SLL (Linux cooked v1), >> capture size 262144 bytes >> 22:10:11.754862 IP (tos 0x88, ttl 64, id 22024, offset 0, flags >> [none], proto UDP (17), length 120) >> ???? 192.168.187.129.5999 > 192.168.187.1.53058: UDP, length 92 >> 22:10:16.874400 IP (tos 0x88, ttl 64, id 22399, offset 0, flags >> [none], proto UDP (17), length 120) >> ???? 192.168.187.129.5999 > 192.168.187.1.53058: UDP, length 92 >> 22:10:21.994433 IP (tos 0x88, ttl 64, id 23057, offset 0, flags >> [none], proto UDP (17), length 120) >> ???? 192.168.187.129.5999 > 192.168.187.1.53058: UDP, length 92 >> 22:10:27.370558 IP (tos 0x88, ttl 64, id 23912, offset 0, flags >> [none], proto UDP (17), length 120) >> ???? 192.168.187.129.5999 > 192.168.187.1.53058: UDP, length 92 >> 22:10:32.494605 IP (tos 0x88, ttl 64, id 24944, offset 0, flags >> [none], proto UDP (17), length 120) >> ???? 192.168.187.129.5999 > 192.168.187.1.53058: UDP, length 92 >> 22:10:37.610779 IP (tos 0x88, ttl 64, id 25327, offset 0, flags >> [none], proto UDP (17), length 120) >> ???? 192.168.187.129.5999 > 192.168.187.1.53058: UDP, length 92 >> 22:10:42.986834 IP (tos 0x88, ttl 64, id 25498, offset 0, flags >> [none], proto UDP (17), length 120) >> ???? 192.168.187.129.5999 > 192.168.187.1.53058: UDP, length 92 >> 22:10:48.362908 IP (tos 0x88, ttl 64, id 26521, offset 0, flags >> [none], proto UDP (17), length 120) >> ???? 192.168.187.129.5999 > 192.168.187.1.53058: UDP, length 92 >> 22:10:53.486942 IP (tos 0x88, ttl 64, id 27214, offset 0, flags >> [none], proto UDP (17), length 120) >> ???? 192.168.187.129.5999 > 192.168.187.1.53058: UDP, length 92 >> 22:11:18.571437 IP (tos 0x88, ttl 64, id 31112, offset 0, flags >> [none], proto UDP (17), length 120) >> ???? 192.168.187.129.5999 > 192.168.187.1.53058: UDP, length 92 >> 22:11:23.947460 IP (tos 0x88, ttl 64, id 31897, offset 0, flags >> [none], proto UDP (17), length 120) >> ???? 192.168.187.129.5999 > 192.168.187.1.53058: UDP, length 92 >> 22:11:29.323608 IP (tos 0x88, ttl 64, id 32715, offset 0, flags >> [none], proto UDP (17), length 120) >> ???? 192.168.187.129.5999 > 192.168.187.1.53058: UDP, length 92 >> 22:11:34.699676 IP (tos 0x88, ttl 64, id 33840, offset 0, flags >> [none], proto UDP (17), length 120) >> ???? 192.168.187.129.5999 > 192.168.187.1.53058: UDP, length 92 >> ^C >> 13 packets captured >> 13 packets received by filter >> 0 packets dropped by kernel >> >> As the tcpdump log showed, the wireguard server always chose the ip >> with lowest metric as source ip (192.168.187.129) to reply to the >> client (while the client tried to connect 192.168.187.128). >> >> Hope the info above can help to debug. Many thanks. From wireguard-mail at chil.at Fri Oct 21 13:11:08 2022 From: wireguard-mail at chil.at (Christoph Loesch) Date: Fri, 21 Oct 2022 15:11:08 +0200 Subject: Source IP selection or multi-home or multi-interface issue - solved with source-based route - was: Re: Wireguard source ip selection issue with multi interfaces In-Reply-To: <19530c0e-6e5e-f47c-8490-8338512f2cb2@chil.at> References: <60b5d439-e117-f3c7-3a6b-8ce46bc48fc2@chil.at> <19530c0e-6e5e-f47c-8490-8338512f2cb2@chil.at> Message-ID: So I tried to come up with a simple setup to showcase the issue and then I realized: "How should wireguard know which IP is the correct source IP address?" In the simple setup wireguard even selected an IP from an interface which link is down which is kind of strange. After removing the IP from the linkdown interface, wireguard took the IP as source from the next available interface. Adding the IP back to the linkdown interface and wireguard prefered that IP again. I digged into the source code but couldn't find the part where the source-IP is selected for outgoing connections (like a simple ping). I am not a developer but would like to try to understand it. Could someone give me a pointer? Nevertheless thinking about the question how to select the IP, I wondered if it would be possible to set the IP in wireguard settings, so I do not have to update the routes with the src parameter. Then I saw the address setting and setting the IP there just fixed my "issue". I don't know why I overlooked that setting.. sorry for that Thank you and kind regards Am 20.10.2022 um 00:56 schrieb Christoph Loesch: > Now I've read enough (rather old and also pretty recent) posts about that topic and I want to help finally fix this issue. > > Not just because it bugs our network too after we optimized/simplified our router-setup but also because it seems over the years many users seem to beaffected. Until now every thread I've read ends with an open question rather than a solution. > > In Jason's post here from 2017 I found his script that should reproduce theissue, I tried it and could not reproduce the issue with that script.. > https://lists.zx2c4.com/pipermail/wireguard/2017-November/002092.html > To be honest I do not fully understand it to modify it to reproduce the issue in my environment. > > Here I posted one example and the (temporary) solution by deleting the route that wireguard had set and re-creating the same route but defining the (correct) IP as src. > https://lists.zx2c4.com/pipermail/wireguard/2021-November/007324.html > > How could I help to fix this issue? > If it helps, I could give full access to a test-router which has the same setup as all other routers in our network. > > Kind regards > > > Am 23.11.2021 um 14:52 schrieb Christoph Loesch: >> Hi, >> >> Is this related to my issue with using wrong source ip on outgoing connections? >> (https://lists.zx2c4.com/pipermail/wireguard/2021-November/007307.html) >> >> Which bug would that be? Would be interesting in more detail to understand the issue. >> I'm having a hard time to reproduce (or somehow "fix") the issue I am facing, since this issue is on only 5 devices from over 20 devices where all are configured same. >> >> The configuration or better all routes do not even have any metric defined so I am unsure what to add/remove to change the behaviour. >> Also in my configuration there are multiple interfaces (br0,br1,eth0,eth1,etc..) as it is a router but there are no multiple default routes. >> Comparing perfectly fine working devices to one of those five devices which have this issue I am not able to find a difference in IP-config or routes. >> Even kernel and firmware versions are all same. >> >> The only difference I see is while looking at tcpdump where a simple pingto the wireguard server's internal IP (172.27.0.1/24 in my case) originates with the public IP of the device. >> >> If it is a known bug, it would be nice to know what (and if) I could try to fix and possibly confirm the bug. Or even better beeing able to reproduce the issue. >> >> Thanks and kind regards, >> Christoph >> >> Am 19.11.2021 um 14:10 schrieb ??: >>> Hi, >>> As Jason said in mailing list >>> (https://lists.zx2c4.com/pipermail/wireguard/2017-November/002018.html) >>> years ago, if wireguard reply the client with wrong src ip, then it is >>> a bug. And now I'm trying to set up a minimal configuration. >>> >>> Server info: >>> >>> Ubuntu 20.04 in VMware with two Host-only network adapters: >>> >>> wacke at Ubuntu:/etc$ lsb_release -a >>> No LSB modules are available. >>> Distributor ID: Ubuntu >>> Description: Ubuntu 20.04.3 LTS >>> Release: 20.04 >>> Codename: focal >>> >>> wacke at Ubuntu:/etc$ uname -a >>> Linux Ubuntu 5.14.8-051408-generic #202109280455 SMP Tue Sep 28 >>> 04:57:31 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux >>> >>> wacke at Ubuntu:/etc$ sudo ip route show >>> 169.254.0.0/16 dev ens33 scope link metric 1000 >>> 172.17.0.0/16 dev docker0 proto kernel scope link src 172.17.0.1 linkdown >>> 192.168.187.0/24 dev ens33 proto kernel scope link src 192.168.187.129 >>> metric 100 >>> 192.168.187.0/24 dev ens35 proto kernel scope link src 192.168.187.128 >>> metric 101 >>> 192.168.199.0/24 dev wg0 proto kernel scope link src 192.168.199.1 >>> >>> wacke at Ubuntu:/etc$ sudo cat /etc/wireguard/wg0.conf >>> [Interface] >>> PrivateKey = oLLeTlvwWsfGoyQpDwZo0k7AsMf0LkkfycGPwFjamEA= >>> ListenPort = 5999 >>> >>> [Peer] >>> PublicKey = JYvK2/FBbbSrDbkH29ejWejkBJS3ch/SFtyZwNf5O1c= >>> AllowedIPs = 192.168.199.2/32 >>> PersistentKeepalive = 25 >>> >>> [Peer] >>> PublicKey = keIPXZFgvB79biV74kGc5R9vAHpzyVyQHci8KBSDIUw= >>> AllowedIPs = 192.168.199.3/32 >>> PersistentKeepalive = 25 >>> >>> wacke at Ubuntu:/etc$ sudo wg setconf wg0 /etc/wireguard/wg0.conf >>> wacke at Ubuntu:/etc$ sudo ip address add dev wg0 192.168.199.1/24 >>> wacke at Ubuntu:/etc$ sudo ip link set up dev wg0 >>> wacke at Ubuntu:/etc$ sudo wg show >>> interface: wg0 >>> ??? public key: Vkdx0MhQkc9anLaUehR/Df1zuKjxceflxMCAeAaCWCo= >>> ??? private key: (hidden) >>> ??? listening port: 5999 >>> >>> peer: JYvK2/FBbbSrDbkH29ejWejkBJS3ch/SFtyZwNf5O1c= >>> ??? endpoint: 192.168.187.1:35560 >>> ??? allowed ips: 192.168.199.2/32 >>> ??? transfer: 2.89 KiB received, 1.80 KiB sent >>> ??? persistent keepalive: every 25 seconds >>> >>> peer: keIPXZFgvB79biV74kGc5R9vAHpzyVyQHci8KBSDIUw= >>> ??? allowed ips: 192.168.199.3/32 >>> ??? persistent keepalive: every 25 seconds >>> >>> Client info: >>> >>> HOME-Server:/NAS/Software/Software # uname -a >>> Linux HOME-Server 5.14.14-2-default #1 SMP Thu Oct 21 05:05:03 UTC >>> 2021 (2b5383f) x86_64 x86_64 x86_64 GNU/Linux >>> >>> HOME-Server:/NAS/Software/Software # cat wg0.conf >>> [Interface] >>> PrivateKey = 4Pw1cetxd9TdfH3TSab+9UcRBlHwZ1o/vmrUgkerZls= >>> >>> [Peer] >>> PublicKey = Vkdx0MhQkc9anLaUehR/Df1zuKjxceflxMCAeAaCWCo= >>> Endpoint = 192.168.187.128:5999 >>> AllowedIPs = 192.168.199.0/24 >>> PersistentKeepalive = 25 >>> >>> HOME-Server:/NAS/Software/Software # ip link add wg0 type wireguard >>> HOME-Server:/NAS/Software/Software # ip address add dev wg0 192.168.199.2/24 >>> HOME-Server:/NAS/Software/Software # wg setconf wg0 wg0.conf >>> HOME-Server:/NAS/Software/Software # ip link set wg0 up >>> HOME-Server:/NAS/Software/Software # wg show >>> interface: wg0 >>> ??? public key: JYvK2/FBbbSrDbkH29ejWejkBJS3ch/SFtyZwNf5O1c= >>> ??? private key: (hidden) >>> ??? listening port: 53058 >>> >>> peer: Vkdx0MhQkc9anLaUehR/Df1zuKjxceflxMCAeAaCWCo= >>> ??? endpoint: 192.168.187.128:5999 >>> ??? allowed ips: 192.168.199.0/24 >>> ??? transfer: 0 B received, 59.26 KiB sent >>> ??? persistent keepalive: every 25 seconds >>> >>> >>> tcpdump of client side: >>> >>> HOME-Server:/NAS/Software/Software # tcpdump -i any -vn "(dst port 5999)" >>> tcpdump: data link type LINUX_SLL2 >>> tcpdump: listening on any, link-type LINUX_SLL2 (Linux cooked v2), >>> snapshot length 262144 bytes >>> 22:10:37.794301 vmnet1 Out IP (tos 0x88, ttl 64, id 54724, offset 0, >>> flags [none], proto UDP (17), length 176) >>> ????? 192.168.187.1.53058 > 192.168.187.128.5999: UDP, length 148 >>> 22:10:43.170306 vmnet1 Out IP (tos 0x88, ttl 64, id 55327, offset 0, >>> flags [none], proto UDP (17), length 176) >>> ????? 192.168.187.1.53058 > 192.168.187.128.5999: UDP, length 148 >>> 22:10:48.546281 vmnet1 Out IP (tos 0x88, ttl 64, id 56142, offset 0, >>> flags [none], proto UDP (17), length 176) >>> ????? 192.168.187.1.53058 > 192.168.187.128.5999: UDP, length 148 >>> 22:10:53.670343 vmnet1 Out IP (tos 0x88, ttl 64, id 57053, offset 0, >>> flags [none], proto UDP (17), length 176) >>> ????? 192.168.187.1.53058 > 192.168.187.128.5999: UDP, length 148 >>> 22:11:18.754308 vmnet1 Out IP (tos 0x88, ttl 64, id 58505, offset 0, >>> flags [none], proto UDP (17), length 176) >>> ????? 192.168.187.1.53058 > 192.168.187.128.5999: UDP, length 148 >>> 22:11:24.130286 vmnet1 Out IP (tos 0x88, ttl 64, id 58621, offset 0, >>> flags [none], proto UDP (17), length 176) >>> ????? 192.168.187.1.53058 > 192.168.187.128.5999: UDP, length 148 >>> 22:11:29.506305 vmnet1 Out IP (tos 0x88, ttl 64, id 58810, offset 0, >>> flags [none], proto UDP (17), length 176) >>> ????? 192.168.187.1.53058 > 192.168.187.128.5999: UDP, length 148 >>> 22:11:34.882326 vmnet1 Out IP (tos 0x88, ttl 64, id 59444, offset 0, >>> flags [none], proto UDP (17), length 176) >>> ????? 192.168.187.1.53058 > 192.168.187.128.5999: UDP, length 148 >>> ^C >>> 8 packets captured >>> 8 packets received by filter >>> 0 packets dropped by kernel >>> >>> tcpdump of server side: >>> >>> wacke at Ubuntu:~$ sudo tcpdump -i any -vn "(src port 5999)" >>> tcpdump: listening on any, link-type LINUX_SLL (Linux cooked v1), >>> capture size 262144 bytes >>> 22:10:11.754862 IP (tos 0x88, ttl 64, id 22024, offset 0, flags >>> [none], proto UDP (17), length 120) >>> ????? 192.168.187.129.5999 > 192.168.187.1.53058: UDP, length 92 >>> 22:10:16.874400 IP (tos 0x88, ttl 64, id 22399, offset 0, flags >>> [none], proto UDP (17), length 120) >>> ????? 192.168.187.129.5999 > 192.168.187.1.53058: UDP, length 92 >>> 22:10:21.994433 IP (tos 0x88, ttl 64, id 23057, offset 0, flags >>> [none], proto UDP (17), length 120) >>> ????? 192.168.187.129.5999 > 192.168.187.1.53058: UDP, length 92 >>> 22:10:27.370558 IP (tos 0x88, ttl 64, id 23912, offset 0, flags >>> [none], proto UDP (17), length 120) >>> ????? 192.168.187.129.5999 > 192.168.187.1.53058: UDP, length 92 >>> 22:10:32.494605 IP (tos 0x88, ttl 64, id 24944, offset 0, flags >>> [none], proto UDP (17), length 120) >>> ????? 192.168.187.129.5999 > 192.168.187.1.53058: UDP, length 92 >>> 22:10:37.610779 IP (tos 0x88, ttl 64, id 25327, offset 0, flags >>> [none], proto UDP (17), length 120) >>> ????? 192.168.187.129.5999 > 192.168.187.1.53058: UDP, length 92 >>> 22:10:42.986834 IP (tos 0x88, ttl 64, id 25498, offset 0, flags >>> [none], proto UDP (17), length 120) >>> ????? 192.168.187.129.5999 > 192.168.187.1.53058: UDP, length 92 >>> 22:10:48.362908 IP (tos 0x88, ttl 64, id 26521, offset 0, flags >>> [none], proto UDP (17), length 120) >>> ????? 192.168.187.129.5999 > 192.168.187.1.53058: UDP, length 92 >>> 22:10:53.486942 IP (tos 0x88, ttl 64, id 27214, offset 0, flags >>> [none], proto UDP (17), length 120) >>> ????? 192.168.187.129.5999 > 192.168.187.1.53058: UDP, length 92 >>> 22:11:18.571437 IP (tos 0x88, ttl 64, id 31112, offset 0, flags >>> [none], proto UDP (17), length 120) >>> ????? 192.168.187.129.5999 > 192.168.187.1.53058: UDP, length 92 >>> 22:11:23.947460 IP (tos 0x88, ttl 64, id 31897, offset 0, flags >>> [none], proto UDP (17), length 120) >>> ????? 192.168.187.129.5999 > 192.168.187.1.53058: UDP, length 92 >>> 22:11:29.323608 IP (tos 0x88, ttl 64, id 32715, offset 0, flags >>> [none], proto UDP (17), length 120) >>> ????? 192.168.187.129.5999 > 192.168.187.1.53058: UDP, length 92 >>> 22:11:34.699676 IP (tos 0x88, ttl 64, id 33840, offset 0, flags >>> [none], proto UDP (17), length 120) >>> ????? 192.168.187.129.5999 > 192.168.187.1.53058: UDP, length 92 >>> ^C >>> 13 packets captured >>> 13 packets received by filter >>> 0 packets dropped by kernel >>> >>> As the tcpdump log showed, the wireguard server always chose the ip >>> with lowest metric as source ip (192.168.187.129) to reply to the >>> client (while the client tried to connect 192.168.187.128). >>> >>> Hope the info above can help to debug. Many thanks. From icepic.dz at gmail.com Fri Oct 21 13:25:41 2022 From: icepic.dz at gmail.com (Janne Johansson) Date: Fri, 21 Oct 2022 15:25:41 +0200 Subject: Source IP selection or multi-home or multi-interface issue - solved with source-based route - was: Re: Wireguard source ip selection issue with multi interfaces In-Reply-To: References: <60b5d439-e117-f3c7-3a6b-8ce46bc48fc2@chil.at> <19530c0e-6e5e-f47c-8490-8338512f2cb2@chil.at> Message-ID: > "How should wireguard know which IP is the correct source IP address?" > > I digged into the source code but couldn't find the part where the source-IP is selected for outgoing connections (like a simple ping). > I am not a developer but would like to try to understand it. Could someone give me a pointer? I think it is left to the OS and it's tcp stack to pick the ip (and interface) based on the UDP destination ip and the current routing table when sending udp packets. -- May the most significant bit of your life be positive. From cao88yu at gmail.com Fri Oct 21 14:12:58 2022 From: cao88yu at gmail.com (=?UTF-8?B?5pu554Wc?=) Date: Fri, 21 Oct 2022 22:12:58 +0800 Subject: Source IP selection or multi-home or multi-interface issue - solved with source-based route - was: Re: Wireguard source ip selection issue with multi interfaces In-Reply-To: References: <60b5d439-e117-f3c7-3a6b-8ce46bc48fc2@chil.at> <19530c0e-6e5e-f47c-8490-8338512f2cb2@chil.at> Message-ID: Hi, Actually, I've hacked the wireguard source code by myself months ago, and it's working as expected on openwrt with mwan3. See details and the last hack patch on my github issue: https://github.com/openwrt/packages/issues/9538 As I said on the issue, the problem is wireguard always reset the incoming ip and ipmark, and then select the source ip from main routing table. Janne Johansson ?2022?10?21??? 21:27??? > > > "How should wireguard know which IP is the correct source IP address?" > > > > I digged into the source code but couldn't find the part where the source-IP is selected for outgoing connections (like a simple ping). > > I am not a developer but would like to try to understand it. Could someone give me a pointer? > > I think it is left to the OS and it's tcp stack to pick the ip (and > interface) based on the UDP destination ip and the current routing > table when sending udp packets. > > -- > May the most significant bit of your life be positive. From user6 at michaelhorowitz.com Tue Oct 11 00:03:30 2022 From: user6 at michaelhorowitz.com (user6 at michaelhorowitz.com) Date: Mon, 10 Oct 2022 20:03:30 -0400 Subject: DNS leak Wireguard Android app on ChromeOS Message-ID: <5d496a35-f43e-776b-c20c-ed869fb5cb96@michaelhorowitz.com> I found what appears to be a bug, DNS requests outside the VPN tunnel. This happened on a Chromebook using the Wireguard android app. The Wireguard app was version 1.0.20220516 ChromeOS was version 105.0.5195.134 32bit I have a screen shot of the Wireguard app, but I am new to this list and don't know if it allows attachments. If it does, I can provide the screen shot later. The VPN provider was Windscribe and they use 10.255.255.4 for their internal DNS. Below are log records from the router that the Chromebook was connected to. Clearly, it is making DNS requests to their internal DNS server that are outside the VPN tunnel. If they were inside the tunnel, the router would never have seen them. The 10.1.1.5 IP is my local LAN. Oct 10 12:20:44 Firewall: Denied CONN=vlan SRC=10.1.1.5 DST=10.255.255.4 LEN=64 TTL=63 ID=61082 DF PROTO=UDP SPT=35763 DPT=53 LEN=44 Oct 10 10:20:03 Firewall: Denied CONN=vlan SRC=10.1.1.5 DST=10.255.255.4 LEN=73 TTL=62 ID=52035 DF PROTO=UDP SPT=60940 DPT=53 LEN=53 Oct 10 10:19:26 Firewall: Denied CONN=vlan SRC=10.1.1.5 DST=10.255.255.4 LEN=71 TTL=62 ID=32736 DF PROTO=UDP SPT=53213 DPT=53 LEN=51 Oct 10 10:19:26 Firewall: Denied CONN=vlan SRC=10.1.1.5 DST=10.255.255.4 LEN=71 TTL=62 ID=32735 DF PROTO=UDP SPT=53213 DPT=53 LEN=51 Oct 10 10:19:26 Firewall: Denied CONN=vlan SRC=10.1.1.5 DST=10.255.255.4 LEN=71 TTL=63 ID=36817 DF PROTO=UDP SPT=24575 DPT=53 LEN=51 Oct 10 10:19:26 Firewall: Denied CONN=vlan SRC=10.1.1.5 DST=10.255.255.4 LEN=71 TTL=63 ID=65082 DF PROTO=UDP SPT=30781 DPT=53 LEN=51 Oct 10 10:19:26 Firewall: Denied CONN=vlan SRC=10.1.1.5 DST=10.255.255.4 LEN=71 TTL=63 ID=50536 DF PROTO=UDP SPT=32428 DPT=53 LEN=51 Oct 10 10:19:26 Firewall: Denied CONN=vlan SRC=10.1.1.5 DST=10.255.255.4 LEN=71 TTL=63 ID=2381 DF PROTO=UDP SPT=6459 DPT=53 LEN=51 Oct 10 10:19:26 Firewall: Denied CONN=vlan SRC=10.1.1.5 DST=10.255.255.4 LEN=71 TTL=63 ID=30935 DF PROTO=UDP SPT=12559 DPT=53 LEN=51 Oct 10 10:19:26 Firewall: Denied CONN=vlan SRC=10.1.1.5 DST=10.255.255.4 LEN=71 TTL=63 ID=29472 DF PROTO=UDP SPT=16243 DPT=53 LEN=51 Oct 10 10:19:26 Firewall: Denied CONN=vlan SRC=10.1.1.5 DST=10.255.255.4 LEN=71 TTL=63 ID=38528 DF PROTO=UDP SPT=54329 DPT=53 LEN=51 Oct 10 10:19:26 Firewall: Denied CONN=vlan SRC=10.1.1.5 DST=10.255.255.4 LEN=71 TTL=63 ID=53402 DF PROTO=UDP SPT=13893 DPT=53 LEN=51 Oct 10 10:19:26 Firewall: Denied CONN=vlan SRC=10.1.1.5 DST=10.255.255.4 LEN=71 TTL=63 ID=16001 DF PROTO=UDP SPT=46864 DPT=53 LEN=51 Oct 10 10:19:26 Firewall: Denied CONN=vlan SRC=10.1.1.5 DST=10.255.255.4 LEN=71 TTL=63 ID=32123 DF PROTO=UDP SPT=63327 DPT=53 LEN=51 Oct 10 10:19:26 Firewall: Denied CONN=vlan SRC=10.1.1.5 DST=10.255.255.4 LEN=71 TTL=63 ID=33030 DF PROTO=UDP SPT=56642 DPT=53 LEN=51 Oct 10 10:19:26 Firewall: Denied CONN=vlan SRC=10.1.1.5 DST=10.255.255.4 LEN=71 TTL=63 ID=38599 DF PROTO=UDP SPT=25267 DPT=53 LEN=51 Oct 10 10:19:26 Firewall: Denied CONN=vlan SRC=10.1.1.5 DST=10.255.255.4 LEN=73 TTL=62 ID=33582 DF PROTO=UDP SPT=53072 DPT=53 LEN=53 I was not looking for this, so I am not sure if these requests were during the tunnel creation or afterwards. Pretty sure they were not during the shutdown of the tunnel. This is not a fluke, it can be replicated. Michael Horowitz - - - - - End of Message - - - - - From n+wireguard at monade.li Thu Oct 13 22:28:17 2022 From: n+wireguard at monade.li (n+wireguard at monade.li) Date: Fri, 14 Oct 2022 00:28:17 +0200 Subject: [PATCH wireguard-tools] wg-quick: linux: improve and document rpfilter logic Message-ID: <20221013222817.2054793-1-n+wireguard@monade.li> From: Na?m Favier Only restore the fwmark from conntrack if it matches the wireguard mark, so that we don't restore an empty mark. This makes reordering the PREROUTING rules easier. Don't check for UDP when restoring the mark. This makes it possible to use the same mechanism (setting the conntrack mark to the wireguard mark) for other types of connections that need to be exempt from the tunnel; for example, forwarded packets (think Internet connection sharing). Add an example to the man page for this use case. Document this particularly unobvious bit of logic in the code, so that future generations won't have to do as much digging around as I did. Signed-off-by: Na?m Favier --- Tested with both iptables (+ strict rpfilter) and nftables, in a NAT scenario. src/man/wg-quick.8 | 12 ++++++++++-- src/wg-quick/linux.bash | 22 ++++++++++++++++++---- 2 files changed, 28 insertions(+), 6 deletions(-) diff --git a/src/man/wg-quick.8 b/src/man/wg-quick.8 index b84eb64..48d96e5 100644 --- a/src/man/wg-quick.8 +++ b/src/man/wg-quick.8 @@ -21,7 +21,7 @@ wg-quick - set up a WireGuard interface simply .SH DESCRIPTION -This is an extremely simple script for easily bringing up a WireGuard interface, +This is a simple script for easily bringing up a WireGuard interface, suitable for a few common use cases. Use \fIup\fP to add and set up an interface, and use \fIdown\fP to tear down and remove @@ -87,7 +87,7 @@ MTU \(em if not specified, the MTU is automatically determined from the endpoint or the system default route, which is usually a sane choice. However, to manually specify an MTU to override this automatic discovery, this value may be specified explicitly. .IP \(bu -Table \(em Controls the routing table to which routes are added. There are two +Table \(em controls the routing table to which routes are added. There are two special values: `off' disables the creation of routes altogether, and `auto' (the default) adds routes to the default table and enables special handling of default routes. @@ -165,6 +165,14 @@ that this continues to allow most DHCP traffic through, since most DHCP clients sockets, which bypass Netfilter.) When IPv6 is in use, additional similar lines could be added using .BR ip6tables (8). +Another possible use case (Linux only) would be to exempt forwarded traffic from going through the tunnel, so that a machine +with a 0.0.0.0/0 peer can share its Internet connection (for example, using NAT) in a transparent manner: + + \fBPostUp = iptables -t mangle -I PREROUTING -m addrtype ! --dst-type LOCAL -j CONNMARK --set-mark $(wg show %i fwmark)\fP +.br + \fBPreDown = iptables -t mangle -D PREROUTING -m addrtype ! --dst-type LOCAL -j CONNMARK --set-mark $(wg show %i fwmark)\fP +.br + Or, perhaps it is desirable to store private keys in encrypted form, such as through use of .BR pass (1): diff --git a/src/wg-quick/linux.bash b/src/wg-quick/linux.bash index 69e5bef..f755c15 100755 --- a/src/wg-quick/linux.bash +++ b/src/wg-quick/linux.bash @@ -224,7 +224,7 @@ add_default() { cmd ip $proto rule add table main suppress_prefixlength 0 cmd ip $proto route add "$1" dev "$INTERFACE" table $table - local marker="-m comment --comment \"wg-quick(8) rule for $INTERFACE\"" restore=$'*raw\n' nftable="wg-quick-$INTERFACE" nftcmd + local marker="-m comment --comment \"wg-quick(8) rule for $INTERFACE\"" restore=$'*raw\n' nftable="wg-quick-$INTERFACE" nftcmd printf -v nftcmd '%sadd table %s %s\n' "$nftcmd" "$pf" "$nftable" printf -v nftcmd '%sadd chain %s %s preraw { type filter hook prerouting priority -300; }\n' "$nftcmd" "$pf" "$nftable" printf -v nftcmd '%sadd chain %s %s premangle { type filter hook prerouting priority -150; }\n' "$nftcmd" "$pf" "$nftable" @@ -234,10 +234,24 @@ add_default() { printf -v restore '%s-I PREROUTING ! -i %s -d %s -m addrtype ! --src-type LOCAL -j DROP %s\n' "$restore" "$INTERFACE" "${BASH_REMATCH[1]}" "$marker" printf -v nftcmd '%sadd rule %s %s preraw iifname != "%s" %s daddr %s fib saddr type != local drop\n' "$nftcmd" "$pf" "$nftable" "$INTERFACE" "$pf" "${BASH_REMATCH[1]}" done < <(ip -o $proto addr show dev "$INTERFACE" 2>/dev/null) - printf -v restore '%sCOMMIT\n*mangle\n-I POSTROUTING -m mark --mark %d -p udp -j CONNMARK --save-mark %s\n-I PREROUTING -p udp -j CONNMARK --restore-mark %s\nCOMMIT\n' "$restore" $table "$marker" "$marker" - printf -v nftcmd '%sadd rule %s %s postmangle meta l4proto udp mark %d ct mark set mark \n' "$nftcmd" "$pf" "$nftable" $table - printf -v nftcmd '%sadd rule %s %s premangle meta l4proto udp meta mark set ct mark \n' "$nftcmd" "$pf" "$nftable" + printf -v restore '%sCOMMIT\n' "$restore" + + # When strict reverse path filtering is enabled, we need to make sure that WireGuard UDP packets + # arriving on an external interface would be routed through that same interface with their source + # and destination swapped. To do this, we save the fwmark of outgoing WireGuard packets in the + # connection tracking module and restore it for incoming packets. + # As a convenience, we only check for UDP when setting the connection mark, so that other types of + # connections may be exempted from the tunnel using the same mechanism. + # Then, we enable src_valid_mark so that the restored fwmark is taken into account for the reverse path lookup. + # If the rpfilter netfilter module is used instead, it must be invoked with --validmark in the mangle.PREROUTING chain or later. + printf -v restore '%s*mangle\n' "$restore" + printf -v restore '%s-I POSTROUTING -m mark --mark %d -p udp -j CONNMARK --save-mark %s\n' "$restore" "$table" "$marker" + printf -v restore '%s-I PREROUTING -m connmark --mark %d -j CONNMARK --restore-mark %s\n' "$restore" "$table" "$marker" + printf -v restore '%sCOMMIT\n' "$restore" + printf -v nftcmd '%sadd rule %s %s postmangle meta l4proto udp mark %d ct mark set mark \n' "$nftcmd" "$pf" "$nftable" "$table" + printf -v nftcmd '%sadd rule %s %s premangle ct mark %d meta mark set ct mark \n' "$nftcmd" "$pf" "$nftable" "$table" [[ $proto == -4 ]] && cmd sysctl -q net.ipv4.conf.all.src_valid_mark=1 + if type -p nft >/dev/null; then cmd nft -f <(echo -n "$nftcmd") else -- 2.37.2 From david at dbell.dev Tue Oct 18 00:32:21 2022 From: david at dbell.dev (David Bell) Date: Mon, 17 Oct 2022 17:32:21 -0700 Subject: Improving Wireguard Throughput Message-ID: Hey all, I'm investigating Wireguard as an alternative to mTLS in our infrastructure and I'm trying to understand what's achievable in terms of throughput and incremental cpu overhead, though at the moment I'm focused on throughput. We run 48 core machines with 25gbit NICs using the 5.10.113 kernel. Benchmarking unencrypted traffic between hosts in the same dc achieves line rate, and initially Wireguard was hitting ~2gbit. However, After bumping up Wireguard's default MTU to 8920 I'm able to get to ~12gbit throughput consistently over repeated benchmarks. At this point I'm at a bit of a loss where to look for increasing throughput. Independent results seem to be sparse for this kind of workload, but looking at cilium's benchmark they were able to get around 17.5gbit[0] with the following setup[1]. One thing I'm wondering (probably incorrectly?) is whether we're being impacted by Wireguard's lack of cpu affinity. Our NIC has 8 queues which are bound to explicit cores, and my understanding with Wireguard is that encrypt/decrypt handling is going to get splayed across all 48. [0] https://docs.cilium.io/en/latest/operations/performance/benchmark/#wireguard-vs-ipsec [1] https://docs.cilium.io/en/latest/operations/performance/benchmark/#test-hardware From rick at openfortress.nl Wed Oct 19 09:29:54 2022 From: rick at openfortress.nl (Rick van Rein) Date: Wed, 19 Oct 2022 09:29:54 +0000 Subject: Wireguard setup with SIP Message-ID: <20221019092954.GA22238@openfortress.nl> Hello all, I would like to tell you about some work I'm doing to allow Wireguard sites to negotiate their setup over SIP. This can even be used to spontaneously setup VPNs with new parties, to the level that their SIP server is open to such requests. The standard session setup and teardown is used, INVITE and BYE. Given the right SDP formulation, these can exchange the params for the tunnel; this is what I am sending in the current version, v=0 o=- 4124031101 285260646 IN IP6 2001:db8:666::666 s=- c=IN IP6 2001:db8:666::666 t=0 0 m=application 57660 udp vnd.wireguard a=fmtp:vnd.wireguard pubkey=YWl42m1t56sMAYKwGZUQZNuYG+AbdW9eE7KLj3KBT1M=;prefix=2001:db8:456:1::/64;pskmth=none a=sendrecv The traffic should be authenticated; for that I want to validate the From: and To: SIP headers using SASL, possibly with mutual authentication and possibly with key derivation (then set pskmth to a suitable value). I'm curious how you feel about this! In the SDP fragment above, I mentioned application/vnd.wireguard as a Media Type; these are best registered with IANA. In this application (and probably any other) this could represent the message flow as it is encapsulated into UDP. Would you agree on registering such a Media Type with IANA? I don't care who does it, but it would be the proper course of action. Code, SIP achieves Wireguard setup within localhost: https://gitlab.com/0cpm/subliminal/-/blob/master/src/wgsip.c Man page: https://gitlab.com/0cpm/subliminal/-/blob/master/doc/man/wgsip.1 SASL for SIP and HTTP: https://www.ietf.org/archive/id/draft-vanrein-sipauth-sasl-01.html https://www.ietf.org/archive/id/draft-vanrein-httpauth-sasl-07.html Context: The code arose as part of a project "Subliminal Messaging" that injects digital data into a POTS/VoIP call mixture. The idea is that phone calls would be *one* possible method for Wireguard setup, but the same idea would also work over Thanks, -Rick RFC 6838 says: The "application" top-level type is to be used for discrete data that do not fit under any of the other type names, and particularly for data to be processed by some type of application program. This is information that must be processed by an application before it is viewable or usable by a user. ... The vendor tree is used for media types associated with publicly available products. "Vendor" and "producer" are construed very broadly in this context and are considered equivalent. Note that industry consortia as well as non-commercial entities that do not qualify as recognized standards-related organizations can quite appropriately register media types in the vendor tree. ... Vendor-tree registrations will be distinguished by the leading facet "vnd.". That may be followed, at the discretion of the registrant, by either a media subtype name from a well-known producer (e.g., "vnd.mudpie") or by an IANA-approved designation of the producer's name that is followed by a media type or product designation (e.g., vnd.bigcompany.funnypictures). While public exposure and review of media types to be registered in the vendor tree are not required, using the media-types at iana.org mailing list for review is encouraged, to improve the quality of those specifications. Registrations in the vendor tree may be submitted directly to the IANA, where they will undergo Expert Review [RFC5226] prior to approval. From kevans at FreeBSD.org Sat Oct 29 01:45:05 2022 From: kevans at FreeBSD.org (kevans at FreeBSD.org) Date: Fri, 28 Oct 2022 20:45:05 -0500 Subject: [PATCH] wg: freebsd: move if_wg path to reflect new in-tree location Message-ID: <20221029014505.2679-1-kevans@FreeBSD.org> From: Kyle Evans When we re-added if_wg to the tree, we changed directories in dev to strip the if_ (we don't use this prefix for other interfaces' directories). Adjust it here as a convenience, so that when we import wireguard-tools to FreeBSD the path will just work as-is with our usual build. Signed-off-by: Kyle Evans --- src/ipc-freebsd.h | 2 +- src/uapi/freebsd/dev/{if_wg => wg}/if_wg.h | 0 2 files changed, 1 insertion(+), 1 deletion(-) rename src/uapi/freebsd/dev/{if_wg => wg}/if_wg.h (100%) diff --git a/src/ipc-freebsd.h b/src/ipc-freebsd.h index 2c10c10..b5be15b 100644 --- a/src/ipc-freebsd.h +++ b/src/ipc-freebsd.h @@ -6,7 +6,7 @@ #include #include -#include +#include #define IPC_SUPPORTS_KERNEL_INTERFACE diff --git a/src/uapi/freebsd/dev/if_wg/if_wg.h b/src/uapi/freebsd/dev/wg/if_wg.h similarity index 100% rename from src/uapi/freebsd/dev/if_wg/if_wg.h rename to src/uapi/freebsd/dev/wg/if_wg.h -- 2.36.1 From Jason at zx2c4.com Sat Oct 29 01:52:26 2022 From: Jason at zx2c4.com (Jason A. Donenfeld) Date: Sat, 29 Oct 2022 03:52:26 +0200 Subject: [PATCH] wg: freebsd: move if_wg path to reflect new in-tree location In-Reply-To: <20221029014505.2679-1-kevans@FreeBSD.org> References: <20221029014505.2679-1-kevans@FreeBSD.org> Message-ID: https://git.zx2c4.com/wireguard-tools/commit/?id=7b2ae7aa2f52fbac65874a641cbfbb0182d0ba46 Applied, thanks! (Very excited about all this...) Jason From jirislaby at kernel.org Mon Oct 31 11:44:24 2022 From: jirislaby at kernel.org (Jiri Slaby (SUSE)) Date: Mon, 31 Oct 2022 12:44:24 +0100 Subject: [PATCH] wireguard (gcc13): cast enum limits members to int in prints Message-ID: <20221031114424.10438-1-jirislaby@kernel.org> Since gcc13, each member of an enum has the same type as the enum [1]. And that is inherited from its members. Provided "REKEY_AFTER_MESSAGES = 1ULL << 60", the named type is unsigned long. This generates warnings with gcc-13: error: format '%d' expects argument of type 'int', but argument 6 has type 'long unsigned int' Cast the enum members to int when printing them. Alternatively, we can cast it to ulong (to silence gcc < 12) and use %lu. Alternatively, we can move REKEY_AFTER_MESSAGES away from the enum. [1] https://gcc.gnu.org/bugzilla/show_bug.cgi?id=36113 Cc: Martin Liska Cc: "Jason A. Donenfeld" Cc: "David S. Miller" Cc: Eric Dumazet Cc: Jakub Kicinski Cc: Paolo Abeni Cc: wireguard at lists.zx2c4.com Cc: netdev at vger.kernel.org Signed-off-by: Jiri Slaby (SUSE) --- drivers/net/wireguard/timers.c | 9 +++++---- 1 file changed, 5 insertions(+), 4 deletions(-) diff --git a/drivers/net/wireguard/timers.c b/drivers/net/wireguard/timers.c index b5706b6718b1..51081ba93609 100644 --- a/drivers/net/wireguard/timers.c +++ b/drivers/net/wireguard/timers.c @@ -46,7 +46,7 @@ static void wg_expired_retransmit_handshake(struct timer_list *timer) if (peer->timer_handshake_attempts > MAX_TIMER_HANDSHAKES) { pr_debug("%s: Handshake for peer %llu (%pISpfsc) did not complete after %d attempts, giving up\n", peer->device->dev->name, peer->internal_id, - &peer->endpoint.addr, MAX_TIMER_HANDSHAKES + 2); + &peer->endpoint.addr, (int)MAX_TIMER_HANDSHAKES + 2); del_timer(&peer->timer_send_keepalive); /* We drop all packets without a keypair and don't try again, @@ -64,7 +64,7 @@ static void wg_expired_retransmit_handshake(struct timer_list *timer) ++peer->timer_handshake_attempts; pr_debug("%s: Handshake for peer %llu (%pISpfsc) did not complete after %d seconds, retrying (try %d)\n", peer->device->dev->name, peer->internal_id, - &peer->endpoint.addr, REKEY_TIMEOUT, + &peer->endpoint.addr, (int)REKEY_TIMEOUT, peer->timer_handshake_attempts + 1); /* We clear the endpoint address src address, in case this is @@ -94,7 +94,8 @@ static void wg_expired_new_handshake(struct timer_list *timer) pr_debug("%s: Retrying handshake with peer %llu (%pISpfsc) because we stopped hearing back after %d seconds\n", peer->device->dev->name, peer->internal_id, - &peer->endpoint.addr, KEEPALIVE_TIMEOUT + REKEY_TIMEOUT); + &peer->endpoint.addr, + (int)(KEEPALIVE_TIMEOUT + REKEY_TIMEOUT)); /* We clear the endpoint address src address, in case this is the cause * of trouble. */ @@ -126,7 +127,7 @@ static void wg_queued_expired_zero_key_material(struct work_struct *work) pr_debug("%s: Zeroing out all keys for peer %llu (%pISpfsc), since we haven't received a new one in %d seconds\n", peer->device->dev->name, peer->internal_id, - &peer->endpoint.addr, REJECT_AFTER_TIME * 3); + &peer->endpoint.addr, (int)REJECT_AFTER_TIME * 3); wg_noise_handshake_clear(&peer->handshake); wg_noise_keypairs_clear(&peer->keypairs); wg_peer_put(peer); -- 2.38.1 From Jason at zx2c4.com Mon Oct 31 13:07:45 2022 From: Jason at zx2c4.com (Jason A. Donenfeld) Date: Mon, 31 Oct 2022 14:07:45 +0100 Subject: [PATCH] wireguard (gcc13): cast enum limits members to int in prints In-Reply-To: <20221031114424.10438-1-jirislaby@kernel.org> References: <20221031114424.10438-1-jirislaby@kernel.org> Message-ID: Hi Jiri, On Mon, Oct 31, 2022 at 12:44:24PM +0100, Jiri Slaby (SUSE) wrote: > Since gcc13, each member of an enum has the same type as the enum [1]. And > that is inherited from its members. Provided "REKEY_AFTER_MESSAGES = 1ULL > << 60", the named type is unsigned long. > > This generates warnings with gcc-13: > error: format '%d' expects argument of type 'int', but argument 6 has type 'long unsigned int' > > Cast the enum members to int when printing them. > > Alternatively, we can cast it to ulong (to silence gcc < 12) and use %lu. > Alternatively, we can move REKEY_AFTER_MESSAGES away from the enum. > > [1] https://gcc.gnu.org/bugzilla/show_bug.cgi?id=36113 Huh, interesting situation. It's interesting that 1<<60 even works at all on old gccs. I guess that in this case, it just takes the type of the actual constant, rather than of the enum type? Either way, I'll apply (a version of) your patch and push it back out on the next wireguard fixup series. Thanks for this. Jason From madars at gmail.com Mon Oct 31 03:26:57 2022 From: madars at gmail.com (Madars Virza) Date: Sun, 30 Oct 2022 23:26:57 -0400 Subject: WireGuard namespacing/isolation on Windows Message-ID: Hi! Consider the following use case: preventing accidental WebRTC-style information leaks. These leaks used to happen because WebRTC JS API exposes IP enumeration even if no packets get sent over the corresponding interfaces (i.e., even though the default route is the VPN endpoint, WebRTC API would "betray" information about other interfaces visible to the browser.) In Linux, an elegant way around such leakage is to run your application in a separate network namespace a la https://www.wireguard.com/netns/ . For example, you can launch your browser/BitTorrent client/etc in a separate netns that only sees wgN so that even if there were WebRTC-style leaks, the application would not immediately see interfaces outside its network namespace. What would one do to achieve a similar result for WireGuard clients on Windows? I'd be happy to write a little bit of code / accept solutions that are not production-grade (this is all meant for a developer workstation). Madars From kyle.leet at gmail.com Tue Oct 25 06:58:33 2022 From: kyle.leet at gmail.com (Kyle Sanderson) Date: Mon, 24 Oct 2022 23:58:33 -0700 Subject: absolutely terrible wireguard performance on a 4ghz box - need tuning help Message-ID: hi wireguard, APU2C4 - AMD Embedded G series GX-412TC, 1 GHz quad Jaguar core with 64 bit and AES-NI support There has to be a way to improve this. I'm getting 20-40MB/s (causing a system load of 6 on this poor box) where SSHFS in comparison is still able to fly. Traffic not traversing the tunnel is also, strangely, impacted as well. I have the Intel i210AT controller, which doesn't appear to have any offloading to it when wireguard is present. ksoftirqd/0 spins at around 50%, ksoftirqd/1 spins around 20%, and the other 12(!) wg-crypt threads hover around 20% respectively. If I disable wg I'm able to get line-rate through acceleration. The other 2 ksoftirqds sit around 5% respectively. kworker events jumping between 10-25%. Regrettably openwrt doesn't appear to have perf available... K.