T-Mobile 4G/5G CGNAT vs WireGuard tunnel jitter

[Mo Balaa]

Thanks for sharing, I have also been running WG tunnels over T-Mobile home internet and haven’t seen any of the jitter you are reporting. 

Did you try the same tests (outbound) without running them via WG?
Which modem do you have? How many signal bars are you getting? Also, what does an non-tunneled speed test report?

Cheers 


> On Apr 10, 2021, at 10:31, Lonnie Abelbeck <lists at lonnie.abelbeck.com> wrote:
> 
> Greetings,
> 
> I have been testing the T-Mobile Home Internet (4G/5G fixed wireless) service to a Linode VM via WireGuard.
> 
> The TMHI service uses CGNAT plus an additional NAT in their modem/gateway with a MTU of 1420, so WireGuard is configured with a 1340 MTU.
> 
> Everything works, but I thought I would share some jitter results that readers here might find interesting.
> 
> [gw-lan WGIP:10.4.1.1] -- [TMHI modem/gateway] -- 4G/5G/CGNAT -- [linode WGIP:10.4.1.10]
> 
> gw-lan ~ # mtr -wn -c 30 -s 1340 10.4.1.10
> ...
> HOST: gw-lan      Loss%   Snt   Last   Avg  Best  Wrst StDev
>  1.|-- 10.4.1.10  0.0%    30   88.7  88.9  77.2  99.2   5.4
> 
> Looks to be as expected, in the direction of the CGNAT, now the other direction, against the grain of the CGNAT ...
> 
> linode ~ # mtr -wn -c 30 -s 1340 10.4.1.1
> ...
> HOST: linode     Loss%   Snt   Last   Avg  Best  Wrst StDev
>  1.|-- 10.4.1.1  0.0%    30  206.1 243.5  73.8 393.9  97.9
> 
> Huge jitter, and is very reproducible.  But no packet loss.
> 
> Further investigation shows for low traffic rates (linode->gw-lan) the jitter over WireGuard is huge, here are some UDP iperf3 tests showing how the jitter goes down as the traffic rate is increased.
> 
> linode ~ # iperf3 -c 10.4.1.1 -u -b 5k -t 30
> ...
> [ ID] Interval           Transfer     Bitrate         Jitter    Lost/Total Datagrams
> [  5]   0.00-30.25  sec  18.9 KBytes  5.11 Kbits/sec  68.428 ms  0/15 (0%)  receiver
> 
> linode ~ # iperf3 -c 10.4.1.1 -u -b 10k -t 30
> ...
> [ ID] Interval           Transfer     Bitrate         Jitter    Lost/Total Datagrams
> [  5]   0.00-30.30  sec  37.7 KBytes  10.2 Kbits/sec  82.411 ms  0/30 (0%)  receiver
> 
> linode ~ # iperf3 -c 10.4.1.1 -u -b 50k -t 30
> ...
> [ ID] Interval           Transfer     Bitrate         Jitter    Lost/Total Datagrams
> [  5]   0.00-30.14  sec   184 KBytes  49.9 Kbits/sec  7.532 ms  0/146 (0%)  receiver
> 
> linode ~ # iperf3 -c 10.4.1.1 -u -b 100k -t 30
> ...
> [ ID] Interval           Transfer     Bitrate         Jitter    Lost/Total Datagrams
> [  5]   0.00-30.10  sec   367 KBytes   100 Kbits/sec  4.182 ms  0/292 (0%)  receiver
> 
> linode ~ # iperf3 -c 10.4.1.1 -u -b 500k -t 30
> ...
> [ ID] Interval           Transfer     Bitrate         Jitter    Lost/Total Datagrams
> [  5]   0.00-30.11  sec  1.79 MBytes   498 Kbits/sec  1.308 ms  0/1456 (0%)  receiver
> 
> 
> So using VoIP a higher bitrate CODEC is actually better w.r.t jitter.
> 
> Hope others find this interesting.
> 
> Lonnie
>