Upgrade MM 5.3.1 to latest version fails

Troubleshooting
46

I copied and pasted your command sequences you provided into a code formatting below, just because it is much easier to read and process for me, and as such probably is for others too.

#systemctl start mattermost

Job for mattermost.service failed because the control process exited with error code. See "systemctl status mattermost.service" and "journalctl -xe" for details.

#systemctl status mattermost

● mattermost.service - Mattermost
   Loaded: loaded (/lib/systemd/system/mattermost.service; enabled; vendor preset: enabled)
   Active: activating (auto-restart) (Result: exit-code) since Fre 2021-02-05 13:10:24 CET; 2s ago
  Process: 27369 ExecStart=/opt/mattermost/bin/mattermost (code=exited, status=119)
 Main PID: 27369 (code=exited, status=119)

Feb 05 13:10:24 SERVER systemd[1]: Failed to start Mattermost.
Feb 05 13:10:24 SERVER systemd[1]: mattermost.service: Unit entered failed state.
Feb 05 13:10:24 SERVER systemd[1]: mattermost.service: Failed with result 'exit-code'.



root@SERVER:/opt# systemctl status mattermost
● mattermost.service - Mattermost
   Loaded: loaded (/lib/systemd/system/mattermost.service; enabled; vendor preset: enabled)
   Active: activating (start) since Fre 2021-02-05 13:10:45 CET; 937ms ago
 Main PID: 27407 (mattermost)
    Tasks: 11 (limit: 65000)
   Memory: 27.7M
      CPU: 353ms
   CGroup: /system.slice/mattermost.service
           └─27407 /opt/mattermost/bin/mattermost

Feb 05 13:10:45 SERVER systemd[1]: mattermost.service: Service hold-off time over, scheduling restart.
Feb 05 13:10:45 SERVER systemd[1]: Stopped Mattermost.
Feb 05 13:10:45 SERVER systemd[1]: Starting Mattermost...
Feb 05 13:10:45 SERVER mattermost[27407]: {"level":"info","ts":1612527045.976185,"caller":"utils/i18n.go:83","msg":"Loaded system
Feb 05 13:10:45 SERVER mattermost[27407]: {"level":"info","ts":1612527045.9765236,"caller":"app/server_app_adapters.go:58","msg":
Feb 05 13:10:45 SERVER mattermost[27407]: {"level":"info","ts":1612527045.9883635,"caller":"sqlstore/supplier.go:215","msg":"Ping
Feb 05 13:10:46 SERVER mattermost[27407]: {"level":"error","ts":1612527046.0502627,"caller":"sqlstore/supplier.go:807","msg":"Fai

root@SERVER:/opt# ps -ef | grep -i mattermost
matterm+ 27425     1 31 13:10 ?        00:00:00 /opt/mattermost/bin/mattermost
root     27439 21018  0 13:10 pts/0    00:00:00 grep --color=auto -i mattermost

root@SERVER:/opt# ps -ef | grep -i mattermost
root     27459 21018  0 13:11 pts/0    00:00:00 grep --color=auto -i mattermost

Ok, return to v5.3.1 ... :frowning: 
> rm -r mattermost
> mv old.mattermost mattermost
> mysql> drop database mattermost;
> mysql> create database mattermost;
> mysql -u root -p mattermost < mm.sql
> systemctl start mattermost
> systemctl status mattermost
    ● mattermost.service - Mattermost
       Loaded: loaded (/lib/systemd/system/mattermost.service; enabled; vendor preset: enabled)
       Active: active (running) since Fre 2021-02-05 13:39:48 CET; 17min ago
     Main PID: 30413 (mattermost)
        Tasks: 23 (limit: 65000)
       Memory: 78.7M
          CPU: 2.676s
       CGroup: /system.slice/mattermost.service
               ├─30413 /opt/mattermost/bin/mattermost




I noticed a few things based on the commands you were running here that could be causing potential conflicts when you are attempting to upgrade, mainly being that (unless you haven’t included it in the commands here) you aren’t stopping the Mattermost program before making upgrades, which is something you must do, especially when upgrading the binary itself. To do this, simply run the command sudo systemctl stop mattermost && systemctl status mattermost and then you can verify that it is stopped with the resulting output (which came from one of my own mattermost dev instances that I just stopped and pulled the output from)

root@ubuntu:~# systemctl stop mattermost && systemctl status mattermost
● mattermost.service - Mattermost
     Loaded: loaded (/lib/systemd/system/mattermost.service; enabled; vendor preset: enabled)
     Active: inactive (dead) since Sun 2021-02-07 13:56:35 EST; 40ms ago
    Process: 3190695 ExecStart=/opt/mattermost/bin/mattermost (code=exited, status=0/SUCCESS)
   Main PID: 3190695 (code=exited, status=0/SUCCESS)

Feb 07 07:08:07 ubuntu mattermost[3190695]: {"level":"info","ts":1612699686.6868248,"caller":"app/server.go:817","msg":>Feb 07 07:08:07 ubuntu mattermost[3190695]: {"level":"info","ts":1612699687.291837,"caller":"app/server.go:860","msg":">Feb 07 07:08:07 ubuntu mattermost[3190695]: {"level":"info","ts":1612699687.521241,"caller":"commands/server.go:106","m>Feb 07 07:08:08 ubuntu systemd[1]: Started Mattermost.
Feb 07 13:56:35 ubuntu systemd[1]: Stopping Mattermost...
Feb 07 13:56:35 ubuntu mattermost[3190695]: {"level":"info","ts":1612724195.3124132,"caller":"app/server.go:643","msg":>Feb 07 13:56:35 ubuntu mattermost[3190695]: {"level":"info","ts":1612724195.4578426,"caller":"app/web_hub.go:115","msg">Feb 07 13:56:35 ubuntu mattermost[3190695]: {"level":"info","ts":1612724195.7608464,"caller":"app/server.go:721","msg":>Feb 07 13:56:35 ubuntu systemd[1]: mattermost.service: Succeeded.
Feb 07 13:56:35 ubuntu systemd[1]: Stopped Mattermost.

Then once upgrading your instance, you must start the Mattermost server again with sudo systemctl start mattermost and hopefully it will get a bit further.

As a side note, I also noticed that while you’re providing the output of systemctl status mattermost the output of journalctl -xe contains much more detailed and specific error logs and would be a great resource to check immediately after encountering an error on start, for example. (checking immediately after encountering an error ensures that the most recent error log messages are shown first, which will be the ones your looking for)

1 Like