Mattermost Calls Plugin not working

Summary
I created a Bitnami Mattermost Application on Azure. The application works and I have initialized SSL. But when using the calls plugins, it’s not working. In the logs i get the following error message:

{
  "caller": "app/plugin_api.go:976",
  "level": "error",
  "msg": "callback failed: call state is missing from channel state",
  "origin": "main.(*Plugin).handleLeave websocket.go:454",
  "plugin_id": "com.mattermost.calls",
  "timestamp": "2023-06-13 13:00:28.899 Z"
}

I already tried to forward the port 8443 through iptables as mentioned in other threads. Still not working. In Azure, I set the rules for incoming and out coming request as demanded by the documentation.
Incoming: 80, 443, 8443 (UDP), 8045
Outgoing: 3478 (UDP), 8443 (UDP)

Steps to reproduce
Version: Mattermost Team Edition packaged by Bitnami 7.10.2-5

Expected behavior
Calls should work.

Observed behavior
What did you see happen? Please include relevant error messages and/or screenshots.

Hi @Lennart-Franck and welcome to the Mattermost forums!

What’s the ICEHostOverride set to in your plugin settings? It should point to the server which has the udp port 8443.

Hey @agriesser,

thanks for your reply!

Currently, I have set the ICEHostOverride to the domain (not the IP-Address) where the Mattermost server is running. On this server, I enabled an outgoing rule for 8443 UDP via the azure Interface.

The domainname is just fine as long as there is no IPv6 IP address assigned to it (Mattermost Calls does not support IPv6 at the moment). Can you verify that there’s no AAAA record for the domain in your ICEHostOverride config?

I’m not familiar with the Azure firewall configuration, but the error indicates that there are connection problems. Could you verify that the connection goes through? If you stop the Mattermost Calls plugin, the UDP port 8443 should be free again on your server, you could then run nc (netcat) there and try to connect to it from the outside to see if the connection actually gets established (to rule out some firewall misconfiguration).

I checked, and the Domain has no IPv6 address assigned to it. I also changed the ICEHostOverride to my ipv4 address to be sure.

So, I disabled the calls plugin and then listened on the port 8443 via nc -u -l -p 8443 and on my client I did echo "Test" | nc -u <IP-ADRESS> 8443 but nothing was coming through. So, I think it’s an azure port problem, but as far as I can tell, everything is set up correctly. Are there some more ports in and outgoing that need to be configured? The following image shows the rules for incoming ports.

At this point, I’m not quite sure how to further investigate on this issue.

Unfortunately I do not have any experience with Azure FW, but could it be that there’s also additionally a host based firewall on your linux server which is preventing access? Can you post the output of the following commands, please?

lsof -n -i :8443
ss -n | grep 8443
iptables -L -n -v

I think the Azure Rules are actually working because when I stopped the rules for 80 and 443 the server was indeed not reachable anymore. So, I think the rules are getting applied.

lsof -n -i :8443: No output

ss -n | grep 8443: No output

iptables -L -n -v:

Chain INPUT (policy ACCEPT 0 packets, 0 bytes)
 pkts bytes target     prot opt in     out     source               destination

Chain FORWARD (policy ACCEPT 0 packets, 0 bytes)
 pkts bytes target     prot opt in     out     source               destination

Chain OUTPUT (policy ACCEPT 0 packets, 0 bytes)
 pkts bytes target     prot opt in     out     source               destination

I think that means that no FW are applied by the VM itself. So, not sure which thing is actually blocking it.

If it helps, this is the DevTools output:

On a standard Mattermost server with the calls plugin enabled, the lsof command will generate the following output:

# lsof -n -i :8443
COMMAND       PID       USER   FD   TYPE   DEVICE SIZE/OFF NODE NAME
plugin-li 1150777 mattermost   12u  IPv4 23749319      0t0  UDP *:8443
plugin-li 1150777 mattermost   14u  IPv4 23749320      0t0  UDP *:8443

this indicates that the calls plugin is running and listening on udp *:8443. If this is not the case on your system, then the calls plugin is not running or you did configure a different binding port for it. Can you please verify your settings and also make sure that the calls plugin is running?

# ps ax | grep calls
1150777 ?        Sl     4:00 plugins/com.mattermost.calls/server/dist/plugin-linux-amd64

Sorry, I was not using sudo with the command … My fault. The correct output is:

# sudo lsof -n -i :8443
COMMAND    PID       USER   FD   TYPE DEVICE SIZE/OFF NODE NAME
plugin-li 5032 mattermost   12u  IPv4 154061      0t0  UDP 127.0.0.1:8443
plugin-li 5032 mattermost   13u  IPv4 154062      0t0  UDP 10.0.0.4:8443
# sudo ps ax | grep calls
   5032 ?        Sl     0:00 plugins/com.mattermost.calls/server/dist/plugin-linux-amd64
   5268 pts/0    S+     0:00 grep calls

OK, so the plugin is listening on this port and there’s not local linux firewall, but you still cannot connect to the port from the outside, which is strange.
Could it be that there is another IP address active on the system? Maybe even a public one? Your output looks different from mine (mine has the * character which means “listen on all interfaces”, yours shows the IP addresses one by one as if you had configured it to do so).
Can you maybe also share the full calls plugin configuration?

mmctl --local config get PluginSettings.Plugins.com.mattermost.calls

I’m still thinking that the Azure Firewall is to blame, but I’m not sure.

To start a call, select Start call in the channel or message header . When you start a call in a channel, you’re muted by default. In a direct or group message you’re unmuted by default. Select the call widget to move it to a different area of your screen.

Here is my Calls Plugin Configuration:

{
  "allowscreensharing": true,
  "defaultenabled": true,
  "enablerecordings": false,
  "enablesimulcast": false,
  "icehostoverride": "<MY_HOST_IP_WITHOUT_PORT>",
  "iceserversconfigs": "[{\"urls\":[\"stun:stun.global.calls.mattermost.com:3478\"]}]",
  "jobserviceurl": null,
  "maxcallparticipants": 0,
  "maxrecordingduration": 60,
  "recordingquality": "medium",
  "rtcdserviceurl": null,
  "serversideturn": false,
  "turncredentialsexpirationminutes": 1440,
  "turnstaticauthsecret": "",
  "udpserveraddress": "",
  "udpserverport": 8443
}

I checked, and the Domain has no IPv6 address assigned to it. I also changed the ICEHostOverride to my ipv4 address to be sure.

@Lennart-Franck 's configuration looks good, not sure what exactly is the problem here to be honest, but it most likely has something to do with the infrastructure before Mattermost and I’m not sure how to debug that further without having access to the systems (or knowledge about the Azure thinggy).

@miikebross welcome to the Mattermost forums!
Does your configuration look similar? Did you follow the troubleshooting steps in this topic so far? What does your infrastructure look like?