Hi. I’ve been running Mattermost in a jelastic-based docker instance for about a year now – I’m a complete docker neophyte, but things have gone well and typically upgrades have been easy.
A community that I’m not the host of updated to 5.33 today, and so I went into my instance to do the same. All appeared fine, but now I’m throwing nginx 502 bad gateway errors that I can’t resolve. When I check the Mattermost version at the command line, I get a huge pile of errors, including the info that my software version (5.30) and my db schema version (5.32) don’t match. I’m not sure if that’s causing the current problem, but it seems like it needs resolution in any case. Any ideas would be welcome!
Next question: buried in among the pile of messages were a couple of errors indicating that SiteURL and SMTP info were not set. And in exploring /mattermost-docker/volumes/app/mattermost/config/config.json, it indeed appears that the file has been overwritten with a default – no SiteURL, no SMTP info, etc. Is there any chance of reverting to a pre-update version of that file…?
More info: I’ve managed to get in and begin reconstructing the config settings, including SiteURL, SMTP, etc. Those settings require a server restart to take effect. However, every time I restart the server those settings disappear again. I’m… a little beside myself.
Can you confirm that you have write permissions to the configuration file that you are making changes to? @ahmaddanial I think you would be better off providing suggestions related to the docker side of things, I’m not as experienced with Docker, however I can help with the 502 code part of things.
Thanks – I do have write permissions, and I’ve confirmed that the changes have been written before the restart. I’m in touch with the hosting provider at this point and am asking for things to be restored to a pre-update state. Fingers crossed!
Basically th2 502 HTTP error means its a bad gateway, which could be at any point in your connection to the server, not specifically related to the configuration of the Mattermost installation itself. Its possible that it is caused from an upstream issue such as an authoritative domain name server going offline for your domain, a failure of one of your current DNS servers to resolve everything to your domain/server properly, etc. I would suggest checking your error.log file for Nginx in /var/log/ or /var/log/nginx/ to see if there are more details that can be determined from there.
Thanks for this, @ahmaddanial – it definitely explains the config.json resets I was seeing!
@XxLilBoPeepsxX I’ve looked into the logs and am not finding anything at all helpful, alas.
This morning I cloned the environment and in the clone updated to 5.33.1 – I think: running docker exec -it mattermost-docker_app_1 mattermost version produces a HIGHLY verbose set of messages (all the way down to “info”), but buried in there is
Build Number: 5.31.0
Build Date: Fri Jan 15 02:36:16 UTC 2021
Build Hash: 5c6f37ee6db78c8440145a27320baebdcc6e8311
Build Enterprise Ready: true
DB Version: 5.32.0
The 502s didn’t go away, so in the cloned environment I took down the load balancer entirely… and am still getting nginx 502s. So… I’m at a loss.
My next thought is to try starting over with a clean environment and a fresh installation, and then importing the data from the old environment, but running the CLI bulk exporter doesn’t seem to be working. I get the same crazy verbose overflow of messages, but no output file despite there not being errors indicating that the process has failed. I’m just flummoxed.
Thanks for your help so far, all – I really appreciate it!