I had to edit some existing configs in my android UI. Namely, to change the DNS entries. Updated with new IPs hit save and all looked good. Going back to the config screen the old IP was still present. Had to kill the app via the android task manager, reload, and then the new value is visually present and also being used.