Mattermost Installed, but not loading in browser

Hi,

New here and I’ve just installed mattermost on a Dreamhost Cloud Instance (VM). I installed form tarball not docker (still leanring docker).

So, I followed all the instructions, and it’s up and running, and I can cutl into it from locahost to loclahost:8065 successfully.

However, when I go to the URL of my server http://chat.kozakis.net:8065 - nothing loads and I get a “The site can’t be reached” in Chrome.

Same with trying to curl http://chat.kozakis.net:8065 - it times out.

Something that wasn’t clear in the requirements and install instructions is whether or not Apache is needed to for Mattermost to work.

IN any case, can anyone help?

Hello?
Bueller? Bueller? Bueller?
Can anyone provide assistance?

Hi there, @Anestis

Since you mentioned that you are able to cURL to localhost:8065, can you confirm if you are able to access Mattermost on the web browser when you use http://localhost:8065?

Which documentation did you refer to for the instructions and how does your your SiteURL look like? You can run the command below on the server terminal to pull your current siteURL configuration as well.

cat /opt/mattermost/config/config.json | grep "SiteURL"

Additionally, did you configure any proxy based on the Configuring NGINX as a proxy for Mattermost Server documentation here?

The VM does not have X11 or a windowmanager installed, it’s just a base install of Debian with MySQL and everything needed to run Mattermost, so I can’t reallt use a web browser to browse to localhost.

I followed these instructions: Installing Mattermost on Debian Buster — Mattermost 5.32 documentation

There is no SiteURL set - the instructions I linked don’t say to set it.

I haven’t configured a proxy yet. The instructions I linked said after installing Mattermost it shoudl be available on port 8065.

Besides, I prefer to use Apache2 over NGNIX as a web service as it’s what I’m more familiar with. However, as I mentioned, no web server is installed yet.

Hello! I can jump in here and provide some clarification then, to this one! If you are running your Mattermost instance without a reverse proxy, you would have to directly access it by the port 8065 in order to load it to your browser, and you would need to set the SiteURL value of the config.json file as the IP address of the server you are accessing it from, in order to make the instance bind to the interface - otherwise, you would only be able to access it locally from the command line.

If you’d like assistance with configuring Apache as the reverse proxy, I would be more than happy to help, as I use Mattermost with Apache personally, and use Apache for everything I do for both my clients as well as my personal websites.

I changed the SiteURL value to the public IP address of the server, but the site still does not load on port 8065.
I woiuld appreciate any help you can give, including help with Apache.

1 Like

Yes, I can certainly help you with your Apache setup - and based on the information gathering that I have conducted on your domain name (that you provided in the initial post) as well as open-source intelligence that I was able to gather, I believe that you are working with a misconfigured Apache server and that this is what is preventing you from accessing your Mattermost instance.

Here is what I have found, and how I interpret this information:
First, I simply opened the original URL that you provided and attempted to access it, which of course failed.

Second, I used my Kali Linux server installation to run a quick reverse DNS query to the domain name you provided, which gave me the origin IP address of your server, as you can see from the below output of the reverse DNS query:

root@kali:~# dig chat.kozakis.net

; <<>> DiG 9.16.11-Debian <<>> chat.kozakis.net
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 31441
;; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 3, ADDITIONAL: 4

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 4096
;; QUESTION SECTION:
;chat.kozakis.net.              IN      A

;; ANSWER SECTION:
chat.kozakis.net.       248     IN      A       208.113.166.163

;; AUTHORITY SECTION:
kozakis.net.            172351  IN      NS      ns3.dreamhost.com.
kozakis.net.            172351  IN      NS      ns2.dreamhost.com.
kozakis.net.            172351  IN      NS      ns1.dreamhost.com.

;; ADDITIONAL SECTION:
ns1.dreamhost.com.      284     IN      A       162.159.26.14
ns2.dreamhost.com.      284     IN      A       162.159.26.81
ns3.dreamhost.com.      284     IN      A       162.159.27.84

;; Query time: 23 msec
;; SERVER: 72.240.13.5#53(72.240.13.5)
;; WHEN: Tue Feb 23 16:23:21 EST 2021
;; MSG SIZE  rcvd: 176

The line below is the line that contains the origin IP address of the webserver that you are hosting your Mattermost instance from. (Correct me if that is not the right IP address, because then there are even bigger issues than those I have identified here)

chat.kozakis.net.       248     IN      A       208.113.166.163

To make sure that this IP address is indeed the origin IP address and not a web application firewall’s IP address, such as a Cloudflare IP address, I used a service (can’t specify which one for client confidentiality reasons) to return the following JSON array, confirming that it is the real origin IP address:

{"as":"AS26347 New Dream Network, LLC","city":"Brea","country":"United States","countryCode":"US","isp":"New Dream Network, LLC","lat":33.9119,"lon":-117.881,"org":"New Dream Network, LLC","query":"208.113.166.163","region":"CA","regionName":"California","status":"success","timezone":"America/Los_Angeles","zip":"92821"}

Now that I confirmed to myself that the IP address is indeed the correct address to investigate, I ran a port scan (again from my Kali Linux machine) to determine what services are open on your server. Here is the result of my first scan:
Host is up (0.062s latency).
Not shown: 65532 filtered ports
PORT    STATE  SERVICE
22/tcp  open   ssh
80/tcp  closed http
443/tcp closed https

First, this indicates that the port 8065 which you are attempting to connect to by appending :8065 to the end of your URL is not open, and therefore cannot be connected to regardless.
Second, unless there is an enhanced security feature that is blocking direct requests to the IP address, your firewall is currently blocking all external requests to this domain name, which is obviously not going to allow you to connect to the website. This can be changed by modifying your firewall access rules to allow traffic on both the HTTP port 80 and the HTTPS port 443.

Third, as a note on the configuration, you need to use Apache as a reverse proxy with Mattermost. This allows you to access Mattermost from chat.example.com instead of having to navigate to chat.example.com:8065, and also significantly improves the security of your Mattermost instance. This is going to require rewriting your Apache VirtualHost configuration file, which I am more than willing and capable of assisting you with.

Finally, you are going to want to use HTTPS SSL encryption on your chat service and as such, it would be useful to install CertBot which is free, on your server.

Does this make sense to you, and would you be able to further elaborate on the Arch and operating system of the webserver you are currently using?

Thanks! :slight_smile:

Makes sense if Dreamhost has a firewall blocking 8065, as there is no firewall on my VM and Apache isn’t installed yet.

Linux Debian Buster (10) 64 bit.

1 Like

The thing is, port 8065 will never be exposed to the public internet when you configure Apache properly, it should only ever be able to be accessed from your localhost. And since you are on Debian (my personal preference, 1000% support it! :grin: ) I’ll be able to easily help you set it up correctly!

Do you know how to install Apache and do the basic configuration or shall I start from the very beginning and walk you through the installation?

Also, as a side note, you should have UFW (uncomplicated firewall) installed by default on your system, as it comes with Debian builds by default, it just may not be enabled, however, we will change that as well in the coming steps, just to be safe, if that is okay with you :slight_smile:

I have Apache installed. Do you have an Apache config file for Mattermost?

I did find this - Configuring Apache2 (Unofficial) — Mattermost 5.32 documentation - but yes, I will want to run it over SSL (https).

So I’ve installed certbot and generated an SSL Certificate.

Current config file is as follow:

# The ServerName directive sets the request scheme, hostname and port that # the server uses to identify itself. This is used when creating # redirection URLs. In the context of virtual hosts, the ServerName # specifies what hostname must appear in the request's Host: header to # match this virtual host. For the default virtual host (this file) this # value is not decisive as it is used as a last resort host regardless. # However, you must set it for any further virtual host explicitly. #ServerName www.example.com
ServerAdmin webmaster@localhost
DocumentRoot /var/www/html

# Available loglevels: trace8, ..., trace1, debug, info, notice, warn,
# error, crit, alert, emerg.
# It is also possible to configure the loglevel for particular
# modules, e.g.
#LogLevel info ssl:warn

ErrorLog ${APACHE_LOG_DIR}/error.log
CustomLog ${APACHE_LOG_DIR}/access.log combined

# For most configuration files from conf-available/, which are
# enabled or disabled at a global level, it is possible to
# include a line for only one particular virtual host. For example the
# following line enables the CGI configuration for this host only
# after it has been globally disabled with "a2disconf".
#Include conf-available/serve-cgi-bin.conf


ServerName chat.kozakis.net
Include /etc/letsencrypt/options-ssl-apache.conf
ServerAlias www.[domain.name]
SSLCertificateFile /etc/letsencrypt/live/chat.kozakis.net/fullchain.pem
SSLCertificateKeyFile /etc/letsencrypt/live/chat.kozakis.net/privkey.pem
</VirtualHost>
</IfModule>

I had tio put [domain name] in the place of the website domain name as the system won’t let me post more than two URLs in one post.

Hello again! Here is my SSL Apache Configuration file, slightly redacted though.

<IfModule mod_ssl.c>
<VirtualHost *:443>
  # If you're not using a subdomain you may need to set a ServerAlias to:
  # ServerAlias www.mydomain.com
  ServerName <REDACTED>
  ServerAdmin [REMOVED]@<REDACTED>
  ProxyPreserveHost On
  DocumentRoot /opt/mattermost
  # For the IP logging we run on all domains/subdomains, header 
  # provides real user IP when behind Cloudflare
  RemoteIPHeader CF-Connecting-IP


# Mattermost Config

        # Set web sockets
        ProxyRequests Off
        RewriteEngine On
        RewriteCond %{REQUEST_URI} ^/api/v3/users/websocket [NC,OR]
        RewriteCond %{HTTP:UPGRADE} ^WebSocket$ [NC,OR]
        RewriteCond %{HTTP:CONNECTION} ^Upgrade$ [NC]
        RewriteRule .* ws://127.0.0.1:8065%{REQUEST_URI} [P,QSA,L]
        RewriteCond %{DOCUMENT_ROOT}/%{REQUEST_FILENAME} !-f
        RewriteRule .* http://127.0.0.1:8065%{REQUEST_URI} [P,QSA,L]
        RequestHeader set X-Forwarded-Proto "https"

  <Location /api/v4/users/websocket>
    Require all granted
    ProxyPassReverse ws://127.0.0.1:8065/api/v4/users/websocket
    ProxyPassReverseCookieDomain 127.0.0.1 <REDACTED>
  </Location>

  <Location />
    Require all granted
    ProxyPassReverse https://127.0.0.1:8065/
    ProxyPassReverseCookieDomain 127.0.0.1 <REDACTED>
  </Location>

# End Mattermost Config

        #Custom Error Pages 
        ErrorDocument 401 /srv/<REDACTED>/error/401
        ErrorDocument 403 /srv/<REDACTED>/error/403
        ErrorDocument 404 /srv/<REDACTED>/error/404
        ErrorDocument 500 /srv/<REDACTED>/error/500

        #Block access to the .git folder from the webserver (Security Enhancement)
        RedirectMatch 404 /\.git

        Include /etc/letsencrypt/options-ssl-apache.conf
        SSLEngine on
        SSLVerifyClient require
        SSLVerifyDepth 1
   
        #Cloudflare Authenticated Origin Pull Certificate
        SSLCACertificateFile /root/origin-pull-ca.pem 
   
        #Domain name SSL Certs. configuration
        SSLCertificateFile /etc/letsencrypt/live/<REDACTED>/fullchain.pem
        SSLCertificateKeyFile /etc/letsencrypt/live/<REDACTED>/privkey.pem
</VirtualHost>
</IfModule>

I know there is a bit of extra stuff in there but I wanted to provide what I know is fully functional for me personally and my team. Does this help?

If you have any questions or encounter any issues, please don’t hesitate to ask them, I’m more than happy to help however I can!

Also on a second note, I’m happy to let you know the SSL configuration seems to be working!

All we should need to get taken care of at this point is setting up your reverse proxy and you should be 100% good to go!

OK, so it’s working (yay), and I have logged in and created my admin account.
The only thing is I don’t know (and I don’t think I have) setup postfix correctly for e-mail notifications.

And one plugin failed: Incident Collaboration (com.mattermost.plugin-incident-management - 1.3.2)
This plugin allows users to coordinate and manage incidents within Mattermost.
Disable - Remove
This plugin failed to start. Check your system logs for errors.

What am I looking for and where? Apache or Mattermost logs?

Further bad news, I am now getting " Please check connection, Mattermost unreachable. If issue persists, ask administrator to check WebSocket port." error,’

From Chrome cosnole:

websocket_client.tsx:51 WebSocket connection to ‘wss://chat.kozakis.net/api/v4/websocket’ failed: Error during WebSocket handshake: Unexpected response code: 500

I’m glad to hear it’s working!

As far as this issue

websocket_client.tsx:51 WebSocket connection to ‘wss://chat.kozakis.net/api/v4/websocket’ failed: Error during WebSocket handshake: Unexpected response code: 500

it appears that you are using wss:// for your WebSocket connection. While this is a correct way to implement the websocket protocol, you should only use wss if you are using the TLS encryption features built into Mattermost itself. If you are not using the TLS built into Mattermost, it should just be ws://. I hope that helps, let me know :slight_smile:

As far as Mattermost logs, you should be able to find those in the /opt/mattermost/logs/mattermost.log file, or in the location that you set in your config.json file in the "FileLocation": "/opt/mattermost/logs/" line. If you can change your logging level to DEBUG and then attempt to replicate the events that are giving you errors, and then upload the mattermost.log file, this should help us. In addition, it would be helpful if you could provide the contents of the Apache error log, which can be found at /var/log/apache2/error.log. Thanks!

1 Like

Didn’t get an e-mail notification for this reply. Hmmm.

I found out that that plugin requires an Enterprise License of Mattermost, so I disabled it.

As for the was error, my apache config only has ws:// as shown by your apache config above. I took from yours what I needed for my config.

After some digging it seems I needed to have proxy_ajp enabled for Apache.

Mattermost is now auto-updating in the browser when new messages are posted.

Thanks for your help.

Anya advice on how to setup Postfix and configure Mattermost to properly enable e-mail notifications?

I’m glad it’s now auto-updating, that is great to hear! I personally use the free tier of SendGrid for my instance’s email automation, however, I can indeed help you with the setup of Postfix and configuring it to your Mattermost instance. The one thing that I suggest, however, is not running your email server from the same server as your Mattermost instance if you are utilizing any form of a web application firewall, such as Cloudflare, Akami, etc, due to the fact that your origin IP address will be exposed in the headers of sent emails, and there is no way to prevent this.

It’s a private server for only a few people for a sepcific purpose, so I’m not so worried about the IP address issue.