Poor performance under high load

[Maximilian Pudelko]

Hello WireGuard list,

as part of my research I (try to) evaluate the performance of
WireGuard and found a curious pattern under increasing load: Until
~0.55 Mpps WireGuard can keep up with encrypting and forwarding, but
after that the rate decreases while the CPU load keeps increasing
until all cores are 100% utilized and _no_ packets get send.

Is that expected behavior due to the unoptimized implementation that
uses a ring buffer with spinlocks? perf shows that a lot of time is
spend waiting for those locks.

Is there a simple way to get better performance? Multiple connections?

Measurement graph:
https://gist.github.com/pudelkoM/2f216e7eb820fc5dc898eaea119448e5

Test configuration for reference:

- WireGuard 0.0.20180531
- Point-to-point setup with 2 hosts
- Linux 4.4.0-78-generic
- Intel(R) Xeon(R) CPU E5-2620 v3 @ 2.40GHz
- 2x Intel XL710 NICs
- Traffic: 64 byte UDP packets, 1000 parallel flows by src port randomization


Regards
Max