Summary
Mattermost community edition fails to start after it has been running fine for a month
Steps to reproduce
Service fails to start on Proxmox Ubuntu VM
Expected behavior
Mattermost service starts
Observed behavior
Mattermost service fails to start
Distributor ID: Ubuntu
Description: Ubuntu 24.04.1 LTS
Release: 24.04
Version: 10.4.2
Build Number: 12918017578
Build Date: Wed Jan 22 22:20:52 UTC 2025
Build Hash: dabbe427c192a73527734aec5449a71acbe03535
Build Enterprise Ready: true
info [2025-02-07 16:03:16.307 Z] Pinging SQL caller=“sql/sql_utils.go:68” database=master dataSource=“postgres://%2A%2A%2A%2A:%2A%2A%2A%2A@localhost/mattermost?binary_parameters=yes&connect_timeout=10&sslmode=disable”
Current database schema version is: 128
â—Ź mattermost.service - Mattermost
Loaded: loaded (/usr/lib/systemd/system/mattermost.service; enabled; preset: enabled)
Active: activating (auto-restart) (Result: exit-code) since Fri 2025-02-07 15:38:48 UTC; 8s ago
Process: 7713 ExecStart=/opt/mattermost/bin/mattermost (code=exited, status=1/FAILURE)
Main PID: 7713 (code=exited, status=1/FAILURE)
CPU: 6.470s
Feb 07 15:38:48 mattermost systemd[1]: Failed to start mattermost.service - Mattermost.
Feb 07 15:38:48 mattermost systemd[1]: mattermost.service: Consumed 6.470s CPU time.
From syslog:
2025-02-07T15:35:02.069257+00:00 mattermost mattermost[7239]: {“timestamp”:“2025-02-07 15:35:02.067 Z”,“level”:“info”,“msg”:“Current version is 10.4.2 (12918017578/Wed Jan 22 22:20:52 UTC 2025/dabbe427c192a73527734aec5449a71acbe03535/9ea56374f1e6bb532564a0ab18784e22815f2b9c)”,“caller”:“app/server.go:409”,“current_version”:“10.4.2”,“build_number”:“12918017578”,“build_date”:“Wed Jan 22 22:20:52 UTC 2025”,“build_hash”:“dabbe427c192a73527734aec5449a71acbe03535”,“build_hash_enterprise”:“9ea56374f1e6bb532564a0ab18784e22815f2b9c”,“service_environment”:“production”}
2025-02-07T15:35:02.069274+00:00 mattermost mattermost[7239]: {“timestamp”:“2025-02-07 15:35:02.067 Z”,“level”:“info”,“msg”:“Enterprise Build”,“caller”:“app/server.go:419”,“enterprise_build”:true,“is_trial”:false}
2025-02-07T15:35:02.069285+00:00 mattermost mattermost[7239]: {“timestamp”:“2025-02-07 15:35:02.067 Z”,“level”:“info”,“msg”:“Printing current working”,“caller”:“app/server.go:429”,“directory”:“/opt/mattermost”}
2025-02-07T15:35:02.069295+00:00 mattermost mattermost[7239]: {“timestamp”:“2025-02-07 15:35:02.067 Z”,“level”:“info”,“msg”:“Loaded config”,“caller”:“app/server.go:430”,“source”:“file:///opt/mattermost/config/config.json”}
2025-02-07T15:35:02.072932+00:00 mattermost mattermost[7239]: {“timestamp”:“2025-02-07 15:35:02.069 Z”,“level”:“info”,“msg”:“License key has not been uploaded. Loading license key from disk.”,“caller”:“utils/license.go:107”,“filename”:“/opt/mattermost/config/mattermost.mattermost-license”}
2025-02-07T15:35:02.072967+00:00 mattermost mattermost[7239]: {“timestamp”:“2025-02-07 15:35:02.069 Z”,“level”:“warn”,“msg”:“Failed to get license from disk”,“caller”:“platform/license.go:93”,“error”:“We could not find the license key on disk at /opt/mattermost/config/mattermost.mattermost-license: stat /opt/mattermost/config/mattermost.mattermost-license: no such file or directory”}
2025-02-07T15:35:02.072979+00:00 mattermost mattermost[7239]: {“timestamp”:“2025-02-07 15:35:02.069 Z”,“level”:“warn”,“msg”:“License key from https://mattermost.com required to unlock enterprise features.”,“caller”:“platform/license.go:105”,“error”:"resource "License" not found, id: "}
2025-02-07T15:35:02.072989+00:00 mattermost mattermost[7239]: {“timestamp”:“2025-02-07 15:35:02.070 Z”,“level”:“info”,“msg”:“Starting workers”,“caller”:“jobs/workers.go:48”}
2025-02-07T15:35:02.073000+00:00 mattermost mattermost[7239]: {“timestamp”:“2025-02-07 15:35:02.071 Z”,“level”:“info”,“msg”:“Starting schedulers.”,“caller”:“jobs/schedulers.go:54”}
2025-02-07T15:35:02.131974+00:00 mattermost mattermost[7239]: {“timestamp”:“2025-02-07 15:35:02.104 Z”,“level”:“info”,“msg”:“Starting up plugins”,“caller”:“app/plugin.go:191”}
2025-02-07T15:35:02.132076+00:00 mattermost mattermost[7239]: {“timestamp”:“2025-02-07 15:35:02.105 Z”,“level”:“info”,“msg”:“Syncing plugins from the file store”,“caller”:“app/plugin.go:268”}
2025-02-07T15:35:02.132112+00:00 mattermost mattermost[7239]: {“timestamp”:“2025-02-07 15:35:02.105 Z”,“level”:“info”,“msg”:“Removing local installation of managed plugin before sync”,“caller”:“app/plugin.go:295”,“plugin_id”:“playbooks”}
2025-02-07T15:35:02.153615+00:00 mattermost mattermost[7239]: {“timestamp”:“2025-02-07 15:35:02.109 Z”,“level”:“info”,“msg”:“Removing local installation of managed plugin before sync”,“caller”:“app/plugin.go:295”,“plugin_id”:“com.mattermost.calls”}
2025-02-07T15:35:02.153657+00:00 mattermost mattermost[7239]: {“timestamp”:“2025-02-07 15:35:02.113 Z”,“level”:“info”,“msg”:“Removing local installation of managed plugin before sync”,“caller”:“app/plugin.go:295”,“plugin_id”:“com.mattermost.nps”}
2025-02-07T15:35:02.153667+00:00 mattermost mattermost[7239]: {“timestamp”:“2025-02-07 15:35:02.115 Z”,“level”:“info”,“msg”:“Removing local installation of managed plugin before sync”,“caller”:“app/plugin.go:295”,“plugin_id”:“mattermost-ai”}
2025-02-07T15:35:02.153679+00:00 mattermost mattermost[7239]: {“timestamp”:“2025-02-07 15:35:02.119 Z”,“level”:“info”,“msg”:“Syncing plugin from file store”,“caller”:“app/plugin.go:342”,“plugin_id”:“playbooks”,“bundle_path”:“plugins/playbooks.tar.gz”}
2025-02-07T15:35:02.153689+00:00 mattermost mattermost[7239]: {“timestamp”:“2025-02-07 15:35:02.132 Z”,“level”:“info”,“msg”:“Syncing plugin from file store”,“caller”:“app/plugin.go:342”,“plugin_id”:“com.mattermost.calls”,“bundle_path”:“plugins/com.mattermost.calls.tar.gz”}
2025-02-07T15:35:02.816302+00:00 mattermost mattermost[7239]: {“timestamp”:“2025-02-07 15:35:02.811 Z”,“level”:“info”,“msg”:“Installing extracted plugin”,“caller”:“app/plugin_install.go:429”,“plugin_id”:“playbooks”,“version”:“1.40.0”}
2025-02-07T15:35:02.902844+00:00 mattermost mattermost[7239]: {“timestamp”:“2025-02-07 15:35:02.900 Z”,“level”:“info”,“msg”:“Installing extracted plugin”,“caller”:“app/plugin_install.go:429”,“plugin_id”:“com.mattermost.calls”,“version”:“1.4.0”}
2025-02-07T15:35:03.689279+00:00 mattermost mattermost[7239]: {“timestamp”:“2025-02-07 15:35:03.688 Z”,“level”:“warn”,“msg”:“failed to get public IP address for local interface”,“caller”:“app/plugin_api.go:1014”,“plugin_id”:“com.mattermost.calls”,“origin”:“main.(*logger).Warn log.go:108”,“localAddr”:“127.0.0.1”,“error”:“failed to get public address: write udp4 127.0.0.1:8443->52.72.139.62:3478: sendto: invalid argument”}
I think this is related to a failure in the upgrade to 10.4.2. When I re-run dpkg --configure mattermost, it fails:
Setting up mattermost (10.4.2-0) …
dpkg: error processing package mattermost (–configure):
installed mattermost package post-installation script subprocess returned error exit status 1
Errors were encountered while processing:
mattermost
There is also this from mattermost.log:
{“timestamp”:“2025-02-07 15:56:28.757 Z”,“level”:“error”,“msg”:“Failed to sync plugin from file store”,“caller”:“app/plugin.go:344”,“plugin_id”:“com.mattermost.calls”,“bundle_path”:“plugins/com.mattermost.calls.tar.gz”,“error”:“installExtractedPlugin: Unable to move plugin from temporary directory to final destination. Another plugin may be using the same directory name., destination already exists”}
I’m new to mattermost, but the server had been running nicely for several weeks. Any help would be much appreciated!