MM Server Subpath Issue

Troubleshooting
1

we have discussed about implementing subpath support for MM to allow it to work with apache and proxypass.
It seems to me that the implementation deployed with v5.1 doesn’t do the job.
I see that the port assign to the server affects he site url. and I didnt find a documentation how to set it up.
Can we have a second look at it.

In a real scenario. apache handles the webserver port 80 and proxy pass to different applications using subpaths.
apache port 80 http://myserver
apache port 80 http://myserver/mattermost proxypass to MM http://myserver:8065

The second step is to get SSL working. Manage primarily by apache.

Cheers

2

Hi @RbDev,

even without the subpath Mattermost has worked with Apache just fine (I guess once you reach a certain amount of users you will memory wise be happier with nginx or something even lighter).

After the 5.1 release I made a few tests with the Univention app I’m maintaining and it’s default apache configuration (you can find the template at https://stash.z-hub.io/projects/K4U/repos/mattermost/browse/configure_host#94-171, the stuff that proxies to 8778 you can ignore, that is our Web Meetings server). There it worked out of the box. This in the other hand means to proxy /mattermost, you still need to proxy /api as well.

3

Hi @fbartels,

Your script seems to be the same from mattermost documentation website.

You are not using subpaths instead you are using a subdomain. Subdomains works fine.

I can access the website but I keep getting websocket errors.

Can we have some documentation to how properly setup subpaths.

In this case apache is handling SSL.

Cheers

4

Yes, that vhost was primarily designed to be run as a subdomain, but it works just as well when the mattermost ui is running from a subpath.

I already told you above that you need to add a proxy for /api ontop of your /mattermost proxy.

5

“Yes, that vhost was primarily designed to be run as a subdomain, but it works just as well when the mattermost ui is running from a subpath.”

no it doesn’t. it requires a very different setup.

“I already told you above that you need to add a proxy for /api ontop of your /mattermost proxy.”

I have proxy sockets setup for http and ws but it doesn’t work. I think it’s something related to websockets and webapis conflicting to each other. I’m still looking into it.

6

In my test setup it worked. I could access Mattermost running at mydomain/mattermost

7

that’s good news. You should send what you’ve done to the MM team and ask them to document the process.

8

Hey, @RbDev! Sorry to hear that subpath didn’t work the way you expected.

There’s a brief note on how to configure for a subpath in our SiteURL documentation:

In Mattermost v5.1 and later, the URL may contain a subpath such as https://example.com/company/mattermost.

There’s also some more technical details on our developer blog (the blog itself still being under development).

Can you test with an updated SiteURL (and restarting your server) and let me know if that works for you?

9

Hi @Jesse,

Thanks for the support.

The big trouble is to setup apache to get subpath working.

MM implementation is a bit out of course but leave it for later.

I have subpath set to http://myserver/mattermost

[Client] - SSL - [Apache] - port 8065 - [Mattermost]

Good news is I got HTTP to work now I want HTTPS to work as well.

Notice that Apache is handling SSL, not mattermost.

but it fails with websocket incorrect http response code 400 bad request.

{“level”:“error”,“ts”:1532105907.0583162,“caller”:“api4/websocket.go:28”,“msg”:“websocket connect err: websocket: the client is not using the websocket protocol: ‘upgrade’ token not found in ‘Connection’ header”}
{“level”:“error”,“ts”:1532105907.0584223,“caller”:“web/handlers.go:142”,“msg”:“Failed to upgrade websocket connection”,“path”:"/mattermost/api/v4/websocket",“request_id”:“75mwaqp5mtr79cbiycamf896ec”,“ip_addr”:“XX.XX.XX.XX”,“user_id”:“ia9bwacoqpgaxyk8qyp7d1ox5y”,“method”:“GET”,“err_where”:“connect”,“http_code”:500,“err_details”:""}
{“level”:“info”,“ts”:1532105907.0584779,“caller”:“web/handlers.go:163”,“msg”:“http: multiple response.WriteHeader calls”,“source”:“httpserver”}

I had a quick look to those pages and they seem quite shallow.

I think you should have a guide with steps how to setup these important processes.

10

Hey @RbDev, are you able to post any of your configurations for review?

There’s some detailed Apache2 documentation at https://docs.mattermost.com/install/config-apache2.html?highlight=apache? Is that the guide you’re following? Have you updated your Location* directives to reflect the correct subpath?

Also, does your Apache forwarding work without a subpath configuration, and it’s specifically subpath that breaks things?

11

Hi Jesse,

My current config is based on that website. I’ve been making changes trying to get subpath to work. HTTP is working but HTTPS isn’t.

I haven’t tried without subpath but that is a different scenario. Even if it does work. it doesn’t mean it will with subpaths.

MM server:

Site URL: http://myserver.com/mattermost
Port: 8065
80To433: false

Apache conf:

ProxyPreserveHost on
ProxyRequests off

RewriteEngine On
RewriteCond %{REQUEST_URI} /mattermost/api/v[0-9]+/(users/)?websocket [NC,OR]
RewriteCond %{HTTP:UPGRADE} ^WebSocket$ [NC,OR]
RewriteCond %{HTTP:CONNECTION} ^Upgrade$ [NC]
RewriteRule .* ws://localhost:8065%{REQUEST_URI} [P,QSA,L]

<LocationMatch “^/mattermost/api/v(?[0-9]+)/(?users/)?websocket”>
Require all granted
ProxyPass ws://localhost:8065/mattermost/api/v%{env:MATCH_APIVERSION}/%{env:MATCH_APIUSERS}websocket
ProxyPassReverse ws://localhost:8065/mattermost/api/v%{env:MATCH_APIVERSION}/%{env:MATCH_APIUSERS}websocket
< /LocationMatch>

<Location “/mattermost”>
Require all granted
ProxyPass http://localhost:8065/mattermost
ProxyPassReverse http://localhost:8065/mattermost
< /Location>

12

Understood. It doesn’t sound like subpaths are (yet) an issue here, but a problem with our documentation re: Apache2 and SSL.

One thing that seems odd in the documentation is that the <VirtualHost *:80> section has wss:// in the RewriteRule, but the <VirtualHost *:443> section has a ws://. This seems backwards, since wss is the analogue to https.

Can you try updating your RewriteRule in the SSL configuration to use wss instead?

13

I see your point. I actually tried both and I remember having issues with wss.

I don’t see why wss is required when sending it to mattermost.

14

Ah, of course, if you’re proxying wss shouldn’t be required. Let me see if I can make our own install instructions work here locally.

15

Hi Jesse,

Good news. I got HTTPS working. Now I can get some sleep.

Please review with your team, it’s possible something is missing or can be done in a better way.

Please create a document on the document.mattermost.org explaining how to achieve this so that people know what to do instead of trying to guess.

I’m not using virtualhosts because now MM is using subpath. It could be done, if needed, I leave this to you.

{Client} - HTTP/HTTPS - {Server : [Apache] - Port 8065 - [Mattermost]}

  • MM server settings:

Site URL: http://myserver.com/mattermost
Port: :8065
80To433: false
Connection Security: None

  • Apache.conf:

ProxyPreserveHost on
ProxyRequests off

<LocationMatch “^/mattermost/api/v(?[0-9]+)/(?users/)?websocket”>
     RewriteEngine On
     RewriteCond %{REQUEST_URI} /mattermost/api/v[0-9]+/(users/)?websocket [NC,OR]
     RewriteCond %{HTTP:UPGRADE} ^WebSocket$ [NC,OR]
     RewriteCond %{HTTP:CONNECTION} ^Upgrade$ [NC]
     RewriteRule .* ws://localhost:8065%{REQUEST_URI} [P,QSA,L]

     Require all granted
     ProxyPass              ws://localhost:8065/mattermost/api/v%{env:MATCH_APIVERSION}/%{env:MATCH_APIUSERS}websocket
     ProxyPassReverse ws://localhost:8065/mattermost/api/v%{env:MATCH_APIVERSION}/%{env:MATCH_APIUSERS}websocket
< /LocationMatch>

<Location “/mattermost”>
     Require all granted
     ProxyPass              http://localhost:8065/mattermost
     ProxyPassReverse http://localhost:8065/mattermost
< /Location>

16

Great to hear, @RbDev! Our Apache2 proxy guide is community maintained by people with experience such as yourself. Perhaps you might consider making a pull request with some proposed changes? See https://github.com/mattermost/docs/issues/1295 and https://github.com/mattermost/docs/blob/master/source/install/config-ssl-http2-apache2.rst.