T-Mobile 4G/5G CGNAT vs WireGuard tunnel jitter

[Lonnie Abelbeck]

Greetings,

I have been testing the T-Mobile Home Internet (4G/5G fixed wireless) service to a Linode VM via WireGuard.

The TMHI service uses CGNAT plus an additional NAT in their modem/gateway with a MTU of 1420, so WireGuard is configured with a 1340 MTU.

Everything works, but I thought I would share some jitter results that readers here might find interesting.

[gw-lan WGIP:10.4.1.1] -- [TMHI modem/gateway] -- 4G/5G/CGNAT -- [linode WGIP:10.4.1.10]

gw-lan ~ # mtr -wn -c 30 -s 1340 10.4.1.10
...
HOST: gw-lan      Loss%   Snt   Last   Avg  Best  Wrst StDev
  1.|-- 10.4.1.10  0.0%    30   88.7  88.9  77.2  99.2   5.4

Looks to be as expected, in the direction of the CGNAT, now the other direction, against the grain of the CGNAT ...

linode ~ # mtr -wn -c 30 -s 1340 10.4.1.1
...
HOST: linode     Loss%   Snt   Last   Avg  Best  Wrst StDev
  1.|-- 10.4.1.1  0.0%    30  206.1 243.5  73.8 393.9  97.9

Huge jitter, and is very reproducible.  But no packet loss.

Further investigation shows for low traffic rates (linode->gw-lan) the jitter over WireGuard is huge, here are some UDP iperf3 tests showing how the jitter goes down as the traffic rate is increased.

linode ~ # iperf3 -c 10.4.1.1 -u -b 5k -t 30
...
[ ID] Interval           Transfer     Bitrate         Jitter    Lost/Total Datagrams
[  5]   0.00-30.25  sec  18.9 KBytes  5.11 Kbits/sec  68.428 ms  0/15 (0%)  receiver

linode ~ # iperf3 -c 10.4.1.1 -u -b 10k -t 30
...
[ ID] Interval           Transfer     Bitrate         Jitter    Lost/Total Datagrams
[  5]   0.00-30.30  sec  37.7 KBytes  10.2 Kbits/sec  82.411 ms  0/30 (0%)  receiver

linode ~ # iperf3 -c 10.4.1.1 -u -b 50k -t 30
...
[ ID] Interval           Transfer     Bitrate         Jitter    Lost/Total Datagrams
[  5]   0.00-30.14  sec   184 KBytes  49.9 Kbits/sec  7.532 ms  0/146 (0%)  receiver

linode ~ # iperf3 -c 10.4.1.1 -u -b 100k -t 30
...
[ ID] Interval           Transfer     Bitrate         Jitter    Lost/Total Datagrams
[  5]   0.00-30.10  sec   367 KBytes   100 Kbits/sec  4.182 ms  0/292 (0%)  receiver

linode ~ # iperf3 -c 10.4.1.1 -u -b 500k -t 30
...
[ ID] Interval           Transfer     Bitrate         Jitter    Lost/Total Datagrams
[  5]   0.00-30.11  sec  1.79 MBytes   498 Kbits/sec  1.308 ms  0/1456 (0%)  receiver


So using VoIP a higher bitrate CODEC is actually better w.r.t jitter.

Hope others find this interesting.

Lonnie