The domainname is just fine as long as there is no IPv6 IP address assigned to it (Mattermost Calls does not support IPv6 at the moment). Can you verify that there’s no AAAA record for the domain in your ICEHostOverride config?
I’m not familiar with the Azure firewall configuration, but the error indicates that there are connection problems. Could you verify that the connection goes through? If you stop the Mattermost Calls plugin, the UDP port 8443 should be free again on your server, you could then run nc (netcat) there and try to connect to it from the outside to see if the connection actually gets established (to rule out some firewall misconfiguration).