How to start WebRTC?

Troubleshooting
1

Hello!
I am trying to start MM + WebRTC service.

I am using AWS instance with all ports opened.

My docker-compose.yml is:

version: “2”

services:
webrtc:
image: mattermost/webrtc
container_name: mattermost-webrtc
restart: unless-stopped
ports:
- “7088:7088”
- “7089:7089”
- “8188:8188”
- “8189:8189”

messaging:
image: mattermost/mattermost-preview
container_name: mattermost
restart: unless-stopped
ports:
- “8065:8065”
volumes:
- /home/ubuntu/webrtc/mattermost/app/config:/mattermost/config:rw
- /home/ubuntu/webrtc/mattermost/app/data:/mattermost/data:rw
- /home/ubuntu/webrtc/mattermost/app/logs:/mattermost/logs:rw
- /etc/localtime:/etc/localtime:ro
environment:
- MM_USERNAME=mattermost_user
- MM_PASSWORD=xxxxxxxxx
- MM_DBNAME=mattermost
links:
- webrtc

When I start it at all, i can register and chat in text mode with another one user.

These are MM for WebRTC settings:
image

http://prntscr.com/jo1q9u

There is no any ssl for this test.

WebRTC logs:
VideoCall watchdog started
Loading transport plugin ‘libjanus_pfunix.so’…
NoSIP watchdog started
Configuring SOCK_SEQPACKET Unix Sockets server (Janus API)
[WARN] Unix Sockets server disabled (Admin API)
JANUS Unix Sockets transport plugin initialized!
Unix Sockets thread started
Loading transport plugin ‘libjanus_websockets.so’…
WebSockets server started (port 8188)…
Secure WebSockets server started (port 8189)…
[WARN] Admin WebSockets server disabled
[WARN] Secure Admin WebSockets server disabled
JANUS WebSockets transport plugin initialized!
Loading transport plugin ‘libjanus_http.so’…
HTTP webserver started (port 8088, /janus path listener)…
HTTPS webserver started (port 8089, /janus path listener)…
Admin/monitor HTTP webserver started (port 7088, /admin path listener)…
Admin/monitor HTTPS webserver started (port 7089, /admin path listener)…
JANUS REST (HTTP/HTTPS) transport plugin initialized!
HTTP/Janus sessions watchdog started
WebSockets thread started

But when I click to user icon, i could not see a videochat button.

http://prntscr.com/jo1qij

So tell me please, where I was walking a wrong way?

2

Hi @noizzzzy.boy! Thank you for reaching out.

Can you confirm what Mattermost server version you are using, and whether you followed any specific documentation for this?

Looking forward to further troubleshooting this.

3

Hi, @amy.blais!

Thanks for looking!

I made some changes:

  1. Add an extra hosts to dockers:
    extra_hosts:

    • “testmm:***.***.***.***”

  2. change MM image to
    image: mattermost/platform

  3. add to /etc/hosts on host machine
    .***.***. testmm
    (and on my PC too)

  4. Enable this: “User” -> Accaunt Settings -> “Advanced Settings” -> “Preview pre-release features” -> Enable the ability to make and receive one-on-one WebRTC calls

  5. Change IP in settings to hostname image

The result is:

I can initiate a video call to another user, BUT I have a message “Unable to access camera or microphone.”

In JS console I saw a sign: getUserMedia() no longer works on insecure origins. To use this feature, you should consider switching your application to a secure origin, such as HTTPS. See https://goo.gl/rStTGz for more details.

MM version probably is : mattermost-team-4.9.4-rc1-linux-amd64

WebRTC logs:

Creating new handle in session 660641304239317: 7221613578758599
Detaching handle from JANUS VideoCall plugin
No WebRTC media anymore
Cleaning up handle 7221613578758599…
[7221613578758599] WebRTC resources freed
[7221613578758599] Handle and related resources freed

It seems to me that I need to have a SSL connection here ?..

4

Hi @noizzzzy.boy - thank you for the additional information!

As you mentioned, I believe this is probably an issue with SSL connection. Can you help verify your SSL certificates are correct? You can use https://www.ssllabs.com/ssltest/ to verify them.

Let us know any questions!

5

Hi @amy.blais!

I’ve made several changes.

I started AWS instance with static IP <- load balancer with port routing + SSL <- 3d level domain.

So from outside we have
https://mydomain:443 -> h t t p://instance:8065 (Mattermost)
wss://mydomain:8189 -> h t t p://instance:8188 (WebSocket Janus WebRTC)
https://mydomain:7089/admin -> h t t p://instance:7088(http Janus WebRTC)

First of all - with the latest build of mattermost (4.10) I did not find “User” -> Accaunt Settings -> “Advanced Settings” -> “Preview pre-release features” -> Enable the ability to make and receive one-on-one WebRTC calls

But button (WebRTC) and Video Calls were on their places.

Is it a feature or a forgotten bug? =))

Nevermid, I rolled back to MM 4.9

Other settings (like STUN URI (stun:stun.l.google.com:19302), Gateway Admin Secret (janusoverlord), Enable Insecure Outgoing Connections (true)) are still previous

=================

So, after SSL implementation:

I can call to another user via Mattermost (Mattermonstr!)
I can agree to let access to mic/webcam from both abonents
Another user can accept call
No any voice/video is recieved
After 10 seconds call is finished, there is a sign: Call with user ended

At that time I can see a Janus log:
[ERR] [transports/janus_http.c:janus_http_admin_handler:1563] Invalid url /
[ERR] [transports/janus_http.c:janus_http_admin_handler:1563] Invalid url /
[ERR] [transports/janus_http.c:janus_http_admin_handler:1563] Invalid url /
[ERR] [transports/janus_http.c:janus_http_admin_handler:1563] Invalid url /
Creating new handle in session 2114300204279197: 544213068337465
Creating new handle in session 8287556257129988: 585051823977465
[585051823977465] Creating ICE agent (ICE Full mode, controlled)
[544213068337465] Creating ICE agent (ICE Full mode, controlling)
[WARN] [585051823977465] ICE failed for component 1 in stream 1, but let’s give it some time… (trickle received, answer received, alert not set)
[WARN] [544213068337465] ICE failed for component 1 in stream 1, but let’s give it some time… (trickle pending, answer received, alert not set)
[ERR] [ice.c:janus_ice_check_failed:1606] [585051823977465] ICE failed for component 1 in stream 1…
No WebRTC media anymore
No WebRTC media anymore
[544213068337465] WebRTC resources freed
[WARN] No call to hangup
Detaching handle from JANUS VideoCall plugin
No WebRTC media anymore
[WARN] No call to hangup
Detaching handle from JANUS VideoCall plugin
No WebRTC media anymore
Cleaning up handle 544213068337465…
[544213068337465] WebRTC resources freed
[544213068337465] Handle and related resources freed
Cleaning up handle 585051823977465…
[585051823977465] WebRTC resources freed
[585051823977465] Handle and related resources freed

Still have no any video/audio connection…

What is “Invalid url /” ?

inside Janus Docker I found janus.cfg file
There is a [nat] section
“In case you’re deploying Janus on a server which is configured with a 1:1 NAT (e.g., Amazon EC2), you might want to also specify the public address of the machine using the setting below.”

So i made a correction nat_1_1_mapping = ..**.***

Do not know howto restart Janus server and just restart docker with it inside.

Nothing changed…

6

Hi @noizzzzy.boy!

Which browser and app are you having this issue on (apologies beforehand if you already mentioned and I missed this in your previous notes)?

Regarding Enable the ability to make and receive one-on-one WebRTC calls, this setting was removed in v4.10.

7

Hello, @amy.blais!

I was trying to connect via latest Chrome and Firefox browsers from Windows 7,8 and Android smartphone.

The connection brakes after few seconds. No video/voice data is being transferred. I can see a picture only from my webcam, but not from user whom I making a call.

[WARN] [585051823977465] ICE failed for component 1 in stream 1, but let’s give it some time… (trickle received, answer received, alert not set)
[WARN] [544213068337465] ICE failed for component 1 in stream 1, but let’s give it some time… (trickle pending, answer received, alert not set)
[ERR] [ice.c:janus_ice_check_failed:1606] [585051823977465] ICE failed for component 1 in stream 1…

The log is same as before.

I changed several STUN servers, placed nat_1_1_mapping with IP of an EC2 instance - but still have no success.

What am I doing wrong?

8

Hi @noizzzzy.boy There was a quite a long thread here a while ago to troubleshoot WebRTC issues, I’ll summarize suggestions / tips that people gave throughout the thread:

  • Confirm you have allowed Mattermost to access your camera and microphone. See instructions on how to give permissions on Chrome and Firefox.
  • Do you see any JavaScript errors in the Chrome developer console?
  • This reply here has some tips.
  • You need to set a STUN server in the janus config.
  • You can do additional testing on our nightly build server
  • If you’re using the Docker preview image to test the service, make sure you have the following settings configured:

Enable Mattermost WebRTC: true
Gateway Websocket URL: wss://dockerhost:8189
Gateway Admin URL: https://dockerhost:7089/admin 6
Gateway Admin Secret: janusoverlord

9

@amy.blais I appreciate you for this information.

  • Of course I allowed MM to access my camera and microphone.

  • No, there no any errors in Chrom Dev Console

  • I have all ports opened. WebRTC & MM are in running dockers.
    SSL is provided by Amazone
    But I have no coturn.

  • Inside docker janus.cfg includes
    [nat]
    stun_server = stun.l.google.com
    stun_port = 19302

  • I have configured MM to WebRTC, but I do not use “dockerhost”, because of using true domain name & SSL on it. All these URLs are include real domain name instead “dockerhost”, I think this is right.

10

Hi @noizzzzy.boy! I created a Jira ticket as I’m able to repro the issue: https://mattermost.atlassian.net/browse/MM-10779.

However, I will give a heads-up that WebRTC is known to be unreliable, so it is possible that this is a known issue and something that won’t be fixed in the upcoming release version. All in all, you can follow the progress of the issue in the Jira ticket I linked above.

11

Thank you, @amy.blais!

Also I was trying without Amazon SSL Certificates:

  1. Amazon EC2 instance with webrtc + mattermost dockers
  2. Generated SSL with https://letsencrypt.org/ using https://certbot.eff.org
  3. Certificates were copied inside docker containers of MM & WebRTC, MM was reconfigured to use SSL on port 443. Dockers were restrarted
  4. SSL connection is avaliable to MM and https://mywebsite:7089/admin (webrtc).
  5. All MM settings were configured according to instruction.
  6. Inside WebRTC container was enabled option “nat_1_1_mapping”
  7. While making a video-call the same issue is appear.

This are WebRTC Logs:

I want to try run Janus WebRTC without docker.
Do you know any instructions to this way?

12

Hi @noizzzzy.boy Update on this - we just discussed the Jira ticket as a team and it seems that we will need to update the Janus on our side - the ticket is now queued to be fixed in 5.0 (releasing June 15th), so you should see an improvement on WebRTC once we release.

13

@amy.blais thanks for keeping an eye on this problem .

As for me, looks like WebRTC could not handle a connection because of some external reasons. This probably does not depend on the Mattermost platform. May be it is a NAT or STUN issue. At the same time even a browsers could get some restrictions to provide video/voice stream for these 6 months after last WebRTC release.

So I am very waiting for this issue to be solved.

Thanx again.

14

Hi, @amy.blais!

I made an update of mattermost docker to version 5.0, but still have an issue with video calls. It is very sad fact for me…
Symptoms are similar to the previous

here is log of webrtc
https://pastebin.com/digHwn4x

Which way I should dig to solve it?

15

Hi @noizzzzy.boy - v5.0 hasn’t been officially released yet, I’ll let you know when v5.0 is available (will be by EOD).

16

@amy.blais thanks a lot for your attention!

17

Hi @noizzzzy.boy - v5.0 is now available to download if you’d like to test WebRTC on latest version: https://about.mattermost.com/download/ (it includes a fix for this issue).

18

hi, @amy.blais
I still do not have progress in the video call, even with the new version of Mattermost.

Mattermost version:

{“level”:“info”,“ts”:1529309168.0687737,“caller”:“commands/server.go:81”,“msg”:“Current version is 5.0.0 (5.0.0-rc1/Tue Jun 5 21:38:10 UTC 2018/ffb1d5d8e3a40c007b8faa152ddc3ae281f2a63a/none)”}

WebRTC version:

root@7a0aec31498f:/janus-gateway# /opt/janus/bin/janus -V
Janus commit: 7a42432c2683879e124f3f99251fb21d8cc37a4e
Compiled on: Thu Nov 30 15:37:15 UTC 2017
janus 0.2.6

some other issue in mattermost log:

------------------------------------ ERROR ------------------------------------------------
The platform binary has been deprecated, please switch to using the new mattermost binary.
The platform binary will be removed in a future version.

19

Hi @noizzzzy.boy I’m noticing that your current version is 5.0.0-rc1, which is not the final official version (and doesn’t include WebRTC fixes). I’m wondering why it says that? Can you help double-check that you followed all the upgrade steps: https://docs.mattermost.com/administration/upgrade.html.

Can you also help review these important upgrade notes - e.g. there is one note that mentions that platform binary is renamed to mattermost: https://docs.mattermost.com/administration/important-upgrade-notes.html.

I just tested WebRTC on 5.0 final and it’s working for me, so we can start by double-checking if the upgrade process was successful, and then consider next steps for troubleshooting.

20

Hi, @amy.blais

I changed docker-compose file to get version 5.0.0 of MM.

image

I do not need an upgrade - MM + WebRTC are running from scratch every time to check an ability of Video Call.

Today I was trying to deploy in different zones and on two hosters. The result is the same.

I’m stumped

The error is same:

mwebrtc | [Tue Jun 19 11:11:01 2018] [WARN] [8693122465677053] ICE failed for component 1 in stream 1, but let’s give it some time… (trickle received, answer received, alert not set)
mwebrtc | [Tue Jun 19 11:11:01 2018] [8693122465677053] Creating ICE state check timer with ID 5
mwebrtc | [Tue Jun 19 11:11:01 2018] [61645027977677] Component state changed for component 1 in stream 1: 5 (failed)
mwebrtc | [Tue Jun 19 11:11:01 2018] [WARN] [61645027977677] ICE failed for component 1 in stream 1, but let’s give it some time… (trickle pending, answer received, alert not set)
mwebrtc | [Tue Jun 19 11:11:01 2018] [61645027977677] Creating ICE state check timer with ID 5
mmattermost | {“level”:“debug”,“ts”:1529406661.7588344,“caller”:“migrations/scheduler.go:50”,“msg”:“Scheduling Job”,“scheduler”:“MigrationsScheduler”}
mmattermost | {“level”:“debug”,“ts”:1529406661.7594457,“caller”:“migrations/scheduler.go:89”,“msg”:“All migrations are complete.”,“scheduler”:“MigrationsScheduler”}
mmattermost | {“level”:“debug”,“ts”:1529406661.7600842,“caller”:“jobs/schedulers.go:153”,“msg”:“Next run time for scheduler MigrationsScheduler: ”}
mwebrtc | [Tue Jun 19 11:11:06 2018] [ERR] [ice.c:janus_ice_check_failed:1606] [8693122465677053] ICE failed for component 1 in stream 1…

Here is a full log

My configs are described in How to start WebRTC? - #18 by noizzzzy.boy