Summary
Mattermost sites don’t come up on browser, iRedMail’s do, even though Mattermost’s subdomain is specified.
Steps to reproduce
Mattermost-9.5.2-linux-amd64 using Postgresql 13.14, on iRedMail 1.6.8 with nginx 1.18.0 and MariaDB 15.1
Expected behavior
Mattermost sites come up when URL with correct Mattermost subdomain is given to browser.
Observed behavior
iRedMail’s login page comes up instead of Mattermost page.
Couldn’t solve with this info: Locations of configuration and log files of major components
Interestingly, curl http://mm.example.com yields:
-bash: curl http://mm.example.com:
and curl https://mm.example.com yields: (arrows deleted, parentheses instead)
‘(html)(head)(meta HTTP-EQUIV=“REFRESH” content=“0; url=/mail/”)(/head)(/html)’
but curl http://#.#.#.#:8065 yields html appearing to be the correct Mattermost login page,
as does
curl http://mm.example.com:8065 and
curl https://#.#.#.#:8065
yet curl https://mm.example.com:8065 yields:
curl: (35) error:1408F10B:SSL routines:ssl3_get_record:wrong version number
My first guess is this “iRedMail” is capturing and redirecting traffic. If you want to give the actual URL, someone might be able to figure out something else. But you may need to take this up with iRedMail support.
Thank you, John Oliver, I think this is true - I had Mattermost working for awhile before starting again with iRedMail. I thought it was standard practice to anonymize websites in examples and questions. Also at this point one can’t surf to the website - I’ve attempted so many wrong config file changes that it has become unreachable. I hope to have the site working soon, then hardened before drawing attention to it. Is that wise? I’m asking similar questions at nginx and iRedMail fora too, and studying docs as well. Thanks again,
If you think your config is borked beyond all recovery, you can just copy over a default config.json and start from the beginning. Just remember it starts on a default port of 8065/TCP with no TLS.
Thanks, John, I hired Zhang Huangbin, a iRedMail guy, to set things up. He’s gotten iRedMail up and running again and Mattermost’s login page up. But I can’t access the next Mattermost page, so I will fuss with the MM config file, hopefully while using GNU rcs, an ancient revisioning program that a friend knows who’s willing to help me. With rcs I plan to be able to revert after failed config tinkerings. Should I mark this as solved? I guess I could post files worked with to get this far, which is not like my previously posted situation.
Update: Also will tinker with other config and firewall settings. By the way, rcs looks unhelpful with config files, as between edits rcs files are stored in a central folder away from where config files need to be.
Trying again fails; with error: “error”:“dial tcp 127.0.1.1:5432: connect: connection refused”
About rcs: A friend pointed out that I can store the old config file in RCS folder, then modify the file and place in original working location - If the new one fails, I can use the old one. So rcs could work well.
It looks like you need to whitelist the mattermost binary to access port 8065.
John, What’s the mattermost binary? 127.0.1.1?
/opt/mattermost/bin/mattermost is usually where the binary lives. Look at ExecStart in /lib/systemd/system/mattermost.service
I can whitelist an ip address in fail2ban, but I’m not seeing an ip address in the mattermost log. I see a lot of ‘failed to ping DB’ - looks like two days worth.
On trying ‘systemctl start mattermost’ it fails and says check ‘systemctl status mattermost.service’. There I get “error”:“dial tcp 127.0.1.1:5432: connect: connection refused” I’ve whitelisted 1270.1.1 once in fail2ban. Just whitelisted it again and got the same error message. Not sure what to do.
You’d need to whitelist the IP(s) of the hosts connecting to Mattermost.
As for 'failed to ping DB"… look at DataSource in /opt/mattermost/config.config.json It should look something like:
"DataSource": "postgres://mmuser:mostest@192.168.1.2/mattermost_test?sslmode=disable\u0026connect_timeout=10\u0026binary_parameters=yes",
You need to be able to:
psql -h 192.168.1.2 -d mattermost_test -U mmuser -W and enter the password mostest and get a postgres prompt. I’m guessing you’ll get an error… you need to make sure all of that info is correct and works in the psql command.
The dataSource line looks like this, where .snip. was the password:
“DataSource”: “postgres://mmuser:.snip.@green-rainbow.party:5432/mattermost?sslmode=disable\u0026connect_timeout=10\u0026binary_parameters=yes”,
How’d I do? I tried to follow: Install Mattermost Server on Ubuntu - Mattermost documentation
After entering systemctl start postgresql, and the following:
root@green-rainbow:/opt/mattermost/config# psql -h green-rainbow.party -d mattermost -U mmuser -W
Password:
I get:
psql: error: could not connect to server: Connection refused
Is the server running on host “green-rainbow.party” (127.0.1.1) and accepting
TCP/IP connections on port 5432?
I unbanned/put on ignore list: 127.0.1.1 by:fail2ban-client set nginx-http-auth addignoreip 127.0.1.1
This seems to have resolved that problem. Now on to the next problem.
Well, the problem was not solved as I mis-thought: after ‘systemctl start mattermost’ i get:
“error”:“dial tcp 127.0.1.1:5432: connect: connection refused”
ufw is inactive. There’s an external firewall provided by the vps service Hostinger, but I expect that’s not relevant. It now contains an allow port 5432, but this changes nothing.
Your Postgres server isn’t accepting the username / password you’re providing. It’s possible that it’s set up for localhost and not for remote hosts.
Allowing Remote Connections in PostgreSQL: 2 Methods of Configuring Database should help.
John, thank you for considering this situation, but this is all taking place on just one machine. There is not communication between a remote mattermost computer and a local postgresql one, or vice versa. I ssh into that one vps machine in the cloud, but neither postgresql nor mattermost are away from that machine.
I can log into the postgresql mattermost database using user mmuser and the password:
root@green-rainbow:~# psql -d mattermost -U mmuser -W
Password:
psql (12.18 (Ubuntu 12.18-0ubuntu0.20.04.1))
Type “help” for help.
mattermost=> \q
root@green-rainbow:~#
I’ve whitelisted 127.0.1.1 in all six fail2ban jails. Same behavior.
I wonder what else this could be caused by?
OK, we’re conflating two issues here then. What, exactly, does your DataSource say in config.json? If you have 127.0.1.1 in there, try 127.0.0.1
Once you can get the mattermost service to start successfully, then you should be able to connect to port 8065 with a browser and see the login page.
The dataSource line looks like this, where .snip. was the password:
“DataSource”: “postgres://mmuser:.snip.@green-rainbow.party:5432/mattermost?sslmode=disable\u0026connect_timeout=10\u0026binary_parameters=yes”,
How’d I do? I tried to follow: Install Mattermost Server on Ubuntu - Mattermost documentation
systemctl start mattermost.service
yields:
…“error”:“dial tcp 127.0.1.1:5432: connect: connection refused”
I change ‘green-rainbow.party’ to ‘127.0.1.1’: no change - same error.
I changed 127.0.1.1 to ‘localhost’: then systemctl start mattemost.service returned no errors.
But browsing to neither (IP#s):8065 nor to green-rainbow.party:8065 works.