Calls plugin contacting client when joining call

We have a self-hosted Mattermost instance at our workplace, and I’ve been trying to set up the Calls plugin. I’ve got it working on our internal networks, but not remotely.

Our instance is on Mattermost version 9.5.6, running in Docker using nginx, and we have another reverse proxy in front of the host the service runs on. In order to get around having to route UDP traffic through the reverse proxy, we have a public IP that forwards all TCP and UDP traffic on port 8443 straight to the host, where said ports are exposed from the Mattermost container. This public IP is set as the ICE host override in the Calls plugin settings.

Trying to start a call on a device connected to the workplace network or VPN works perfectly fine, and running a tcpdump shows constant requests back and forth between the Mattermost container and the client device. However, on a device outside the network, the call times out after a few seconds. Checking the container logs gives me a lot of this following error:

{"timestamp":"2024-06-25 15:06:27.585 +02:00","level":"info","msg":"Failed to send packet: write udp4 127.0.0.1:8443->x.x.x.x:40201: sendto: invalid argument","caller":"app/plugin_api.go:1000","plugin_id":"com.mattermost.calls","origin":"main.(*logger).Info log.go:104","origin":"ice/v2.(*candidateBase).writeTo github.com/pion/ice/v2@v2.3.24/candidate_base.go:340"}
{"timestamp":"2024-06-25 15:06:27.586 +02:00","level":"info","msg":"Failed to send packet: write udp4 127.0.0.1:8443->y.y.y.y:61186: sendto: invalid argument","caller":"app/plugin_api.go:1000","plugin_id":"com.mattermost.calls","origin":"main.(*logger).Info log.go:104","origin":"ice/v2.(*candidateBase).writeTo github.com/pion/ice/v2@v2.3.24/candidate_base.go:340"}
{"timestamp":"2024-06-25 15:06:27.788 +02:00","level":"info","msg":"Failed to send packet: write udp4 127.0.0.1:8443->x.x.x.x:40201: sendto: invalid argument","caller":"app/plugin_api.go:1000","plugin_id":"com.mattermost.calls","origin":"main.(*logger).Info log.go:104","origin":"ice/v2.(*candidateBase).writeTo github.com/pion/ice/v2@v2.3.24/candidate_base.go:340"}
{"timestamp":"2024-06-25 15:06:27.789 +02:00","level":"info","msg":"Failed to send packet: write udp4 127.0.0.1:8443->y.y.y.y:61186: sendto: invalid argument","caller":"app/plugin_api.go:1000","plugin_id":"com.mattermost.calls","origin":"main.(*logger).Info log.go:104","origin":"ice/v2.(*candidateBase).writeTo github.com/pion/ice/v2@v2.3.24/candidate_base.go:340"}
{"timestamp":"2024-06-25 15:06:27.991 +02:00","level":"info","msg":"Failed to send packet: write udp4 127.0.0.1:8443->x.x.x.x:40201: sendto: invalid argument","caller":"app/plugin_api.go:1000","plugin_id":"com.mattermost.calls","origin":"main.(*logger).Info log.go:104","origin":"ice/v2.(*candidateBase).writeTo github.com/pion/ice/v2@v2.3.24/candidate_base.go:340"}
{"timestamp":"2024-06-25 15:06:27.991 +02:00","level":"info","msg":"Failed to send packet: write udp4 127.0.0.1:8443->y.y.y.y:61186: sendto: invalid argument","caller":"app/plugin_api.go:1000","plugin_id":"com.mattermost.calls","origin":"main.(*logger).Info log.go:104","origin":"ice/v2.(*candidateBase).writeTo github.com/pion/ice/v2@v2.3.24/candidate_base.go:340"}

Running a tcpdump gave much the same results, repeated requests from the Mattermost container to both my local and public IP addresses (replaced with x.x.x.x and y.y.y.y here for anonymity).

From this I’ve gathered that, when starting a call, the Calls plugin tries to contact the client, instead of the other way around. What I’m really wondering here is if this is intended behavior or not. If so, this makes it unnecessarily difficult for our employees who work remotely to make use of this feature. We’d really like to not force them to use a VPN or change the settings on their private network setup to forward this kind of traffic to their devices.

Regards,
Thomas

Hi @thoesp,

To help, I’d need some more information.

Could you share your plugin config and client logs (sanitized)?

You can also find me at @claudio.costa on our Community server, where it may be quicker to investigate this.

Hi Claudio,

Here’s the plugin config. I’ve redacted the ICE host override setting, but as per my initial post, that is set as the public IP we use to forward traffic straight to the Mattermost container.

"com.mattermost.calls": {
    "allowscreensharing": true,
    "defaultenabled": false,
    "enableipv6": false,
    "enablelivecaptions": false,
    "enablerecordings": false,
    "enableringing": false,
    "enablesimulcast": false,
    "enabletranscriptions": false,
    "icehostoverride": "x.x.x.x",
    "icehostportoverride": 8443,
    "iceserversconfigs": "[{\"urls\":[\"stun:stun.global.calls.mattermost.com:3478\"]}]",
    "jobserviceurl": null,
    "livecaptionslanguage": "en",
    "livecaptionsmodelsize": "tiny",
    "livecaptionsnumthreadspertranscriber": 2,
    "livecaptionsnumtranscribers": 1,
    "maxcallparticipants": 0,
    "maxrecordingduration": 60,
    "recordingquality": "medium",
    "rtcdserviceurl": null,
    "serversideturn": false,
    "tcpserveraddress": "",
    "tcpserverport": 8443,
    "transcribermodelsize": "base",
    "transcribernumthreads": 2,
    "turncredentialsexpirationminutes": 1440,
    "turnstaticauthsecret": "",
    "udpserveraddress": "",
    "udpserverport": 8443
}

As for the console logs, I’m assuming you meant the logs as I try to join a call. Though I must admit, it wasn’t particularly illuminating.

Let me know if there’s anything else you need! Thanks for taking the time :smile:

@thoesp

Since you are using an ice host override, I recommend emptying the iceserversconfigs field, as it complicates things. Then restart the plugin.

I also see you are using our Desktop app. While we debug this, it may be easier to use a Chrome browser so you can more easily access the full logs.

Bear in mind that the client logs will contain IP addresses.

In any case, I recommend going through our documented connectivity checks to ensure the network is configured as expected.