If v6 is disabled, then no packets will make it to that socket, and it won't be used. Nothing to worry about.