Mattermost deployement fails when using docker stack deploy

Summary
Mattermost automaticly shuts down after a minute when using docker stack deploy but not when using docker-compose. It only happends when using Mattermost 6 or above.

Steps to reproduce
Docker version: 20.10.16, build aa7e414
OS: Ubuntu 20.04.4 LTS
Mattermost version: Team Edition 6.3 (but it behaves the same way on all team edition versions for version 6 and above)

Docker-compose file:

version: "3.4"

volumes:
  app-client-plugins:
    driver: local
  app-config:
    driver: local
  app-data:
    driver: local
  app-logs:
    driver: local
  app-plugins:
    driver: local
  db-data:
    driver: local

services:
  db:
    image: postgres:13-alpine
    tmpfs:
      - /tmp
      - /var/run/postgresql
    volumes:
      - db-data:/var/lib/postgresql/data
    environment:
      - POSTGRES_USER=mmuser
      - POSTGRES_PASSWORD=somestrongpassword
      - POSTGRES_DB=mmdb
    deploy:
      restart_policy:
        condition: on-failure

  app:
    image: mattermost/mattermost-team-edition:6.3
    volumes:
      - app-config:/mattermost/config:rw
      - app-data:/mattermost/data:rw
      - app-logs:/mattermost/logs:rw
      - app-plugins:/mattermost/plugins:rw
      - app-client-plugins:/mattermost/client/plugins:rw
    tmpfs:
      - /tmp
    ports:
      - 8065:8000
    environment:
      - MM_SQLSETTINGS_DRIVERNAME=postgres
      - MM_SQLSETTINGS_DATASOURCE=postgres://mmuser:somestrongpassword@db:5432/mmdb?sslmode=disable&connect_timeout=10
      - MM_SERVICESETTINGS_LISTENADDRESS=:8000
      - MM_SERVICESETTINGS_CONNECTIONSECURITY=
    deploy:
      restart_policy:
        condition: on-failure

Deploy with:

  • docker-compose --file docker-compose.yml up
  • docker stack deploy --compose-file docker-compose.yml mattermost

Expected behavior
Both deployment options should have equivalent behaviour.

Observed behavior
When using docker stack deploy, Mattermost starts as it should, runs for a couple minutes and then shuts down, seemingly from nowhere. When using the exact same docker-compose-file with docker-compose everything works as it should. When using an old version of Mattermost (5.16.3) both methods work as intended. Before it shuts down, the web UI is accessible. The docker status goes directly from starting to unhealthy before it shuts down.

It behaves the exact same way on two different machines with different overall configurations.

Log:

{"timestamp":"2022-06-06 17:12:43.622 Z","level":"info","msg":"Server is initializing...","caller":"app/server.go:237","go_version":"go1.16.7"}
{"timestamp":"2022-06-06 17:12:43.622 Z","level":"info","msg":"Starting websocket hubs","caller":"app/web_hub.go:93","number_of_hubs":8}
{"timestamp":"2022-06-06 17:12:43.622 Z","level":"info","msg":"Pinging SQL","caller":"sqlstore/store.go:262","database":"master"}
{"timestamp":"2022-06-06 17:12:43.640 Z","level":"info","msg":"Pinging SQL","caller":"sqlstore/store.go:262","database":"migrations"}
{"timestamp":"2022-06-06 17:12:44.288 Z","level":"info","msg":"The database schema version has been set","caller":"sqlstore/upgrade.go:135","version":"6.3.8"}
{"timestamp":"2022-06-06 17:12:44.815 Z","level":"info","msg":"Loaded system translations","caller":"i18n/i18n.go:93","for locale":"en","from locale":"/mattermost/i18n/en.json"}
{"timestamp":"2022-06-06 17:12:44.826 Z","level":"error","msg":"SiteURL must be set. Some features will operate incorrectly if the SiteURL is not set. See documentation for details: http://about.mattermost.com/default-site-url","caller":"app/server.go:505"}
{"timestamp":"2022-06-06 17:12:44.826 Z","level":"info","msg":"Current version is 6.3.8 (6.3.8/Thu Apr 28 16:38:41 UTC 2022/a030ae538b9cd373b4e0aa808fc426b82ed24937/none)","caller":"app/server.go:514","current_version":"6.3.8","build_number":"6.3.8","build_date":"Thu Apr 28 16:38:41 UTC 2022","build_hash":"a030ae538b9cd373b4e0aa808fc426b82ed24937","build_hash_enterprise":"none"}
{"timestamp":"2022-06-06 17:12:44.826 Z","level":"info","msg":"Team Edition Build","caller":"app/server.go:525","enterprise_build":false}
{"timestamp":"2022-06-06 17:12:44.826 Z","level":"info","msg":"Printing current working","caller":"app/server.go:529","directory":"/mattermost"}
{"timestamp":"2022-06-06 17:12:44.826 Z","level":"info","msg":"Loaded config","caller":"app/server.go:530","source":"file:///mattermost/config/config.json"}
{"timestamp":"2022-06-06 17:12:44.829 Z","level":"info","msg":"Starting workers","caller":"jobs/workers.go:140"}
{"timestamp":"2022-06-06 17:12:44.830 Z","level":"info","msg":"Migrating roles to database.","caller":"app/migrations.go:32"}
{"timestamp":"2022-06-06 17:12:44.831 Z","level":"info","msg":"Starting schedulers.","caller":"jobs/schedulers.go:116"}
{"timestamp":"2022-06-06 17:12:44.844 Z","level":"info","msg":"License key from https://mattermost.com required to unlock enterprise features.","caller":"app/license.go:90"}
{"timestamp":"2022-06-06 17:12:44.853 Z","level":"warn","msg":"Fetching time of first server run failed. Setting to 'now'.","caller":"app/server.go:667"}
{"timestamp":"2022-06-06 17:12:44.903 Z","level":"info","msg":"Migrating emojis config to database.","caller":"app/migrations.go:109"}
{"timestamp":"2022-06-06 17:12:45.975 Z","level":"info","msg":"Starting up plugins","caller":"app/plugin.go:185"}
{"timestamp":"2022-06-06 17:12:45.975 Z","level":"info","msg":"Syncing plugins from the file store","caller":"app/plugin.go:259"}
{"timestamp":"2022-06-06 17:12:47.101 Z","level":"info","msg":"Ensuring Surveybot exists","caller":"app/plugin_api.go:925","plugin_id":"com.mattermost.nps"}
{"timestamp":"2022-06-06 17:12:47.239 Z","level":"info","msg":"Surveybot created","caller":"app/plugin_api.go:925","plugin_id":"com.mattermost.nps"}
{"timestamp":"2022-06-06 17:12:47.249 Z","level":"info","msg":"Upgrade detected. Checking if a survey should be scheduled.","caller":"app/plugin_api.go:925","plugin_id":"com.mattermost.nps"}
{"timestamp":"2022-06-06 17:12:47.257 Z","level":"info","msg":"Scheduling next survey for Jun 27, 2022","caller":"app/plugin_api.go:925","plugin_id":"com.mattermost.nps"}
{"timestamp":"2022-06-06 17:12:47.654 Z","level":"info","msg":"Post.Message has size restrictions","caller":"sqlstore/post_store.go:2207","max_characters":16383,"max_bytes":65535}
{"timestamp":"2022-06-06 17:12:47.815 Z","level":"info","msg":"connectDatabase","caller":"app/plugin_api.go:925","plugin_id":"focalboard","dbType":"postgres"}
{"timestamp":"2022-06-06 17:12:48.364 Z","level":"info","msg":"Initialized notification backend","caller":"app/plugin_api.go:925","plugin_id":"focalboard","name":"notifyMentions"}
{"timestamp":"2022-06-06 17:12:48.365 Z","level":"info","msg":"Initialized notification backend","caller":"app/plugin_api.go:925","plugin_id":"focalboard","name":"notifySubscriptions"}
{"timestamp":"2022-06-06 17:12:48.365 Z","level":"info","msg":"Initialized notification backend","caller":"app/plugin_api.go:925","plugin_id":"focalboard","name":"notifyLogger"}
{"timestamp":"2022-06-06 17:12:48.368 Z","level":"info","msg":"Server.Start","caller":"app/plugin_api.go:925","plugin_id":"focalboard"}
{"timestamp":"2022-06-06 17:12:48.422 Z","level":"error","msg":"Mail server connection test is failed","caller":"app/server.go:1141","error":"SendEmailNotifications is not true"}
{"timestamp":"2022-06-06 17:12:48.424 Z","level":"info","msg":"Starting Server...","caller":"app/server.go:1163"}
{"timestamp":"2022-06-06 17:12:48.424 Z","level":"info","msg":"Server is listening on [::]:8000","caller":"app/server.go:1236","address":"[::]:8000"}
{"timestamp":"2022-06-06 17:13:04.854 Z","level":"error","msg":"Failed to get security update information from Mattermost.","caller":"app/security_update_check.go:86"}
{"timestamp":"2022-06-06 17:13:04.857 Z","level":"warn","msg":"Failied to perform initial product notices fetch","caller":"app/server.go:582","error":"UpdateProductNotices: Fetching product notices failed, Get \"https://notices.mattermost.com/\": dial tcp: lookup notices.mattermost.com on 127.0.0.11:53: read udp 127.0.0.1:36933->127.0.0.11:53: i/o timeout"}
{"timestamp":"2022-06-06 17:13:04.921 Z","level":"info","msg":"Failed to fetch marketplace plugins for telemetry. Using predefined list.","caller":"telemetry/telemetry.go:1327","error":"Get \"https://api.integrations.mattermost.com/api/v1/plugins?build_enterprise_ready=false&cloud=false&enterprise_plugins=false&filter=&local_only=false&page=0&platform=&plugin_id=&return_all_versions=false&server_version=6.3.8\": lookup api.integrations.mattermost.com on 127.0.0.11:53: read udp 127.0.0.1:48117->127.0.0.11:53: i/o timeout"}
{"timestamp":"2022-06-06 17:13:15.976 Z","level":"warn","msg":"Health check failed for plugin","caller":"plugin/health_check.go:59","id":"com.mattermost.nps","error":"plugin RPC connection is not responding"}
{"timestamp":"2022-06-06 17:13:15.976 Z","level":"error","msg":"RPC call OnDeactivate to plugin failed.","caller":"plugin/client_rpc_generated.go:33","plugin_id":"com.mattermost.nps","error":"connection is shut down"}
{"timestamp":"2022-06-06 17:13:15.976 Z","level":"warn","msg":"error closing client during Kill","caller":"plugin/hclog_adapter.go:70","plugin_id":"com.mattermost.nps","wrapped_extras":"errconnection is shut down"}
{"timestamp":"2022-06-06 17:13:15.976 Z","level":"warn","msg":"plugin failed to exit gracefully","caller":"plugin/hclog_adapter.go:72","plugin_id":"com.mattermost.nps"}
{"timestamp":"2022-06-06 17:13:15.999 Z","level":"info","msg":"Ensuring Surveybot exists","caller":"app/plugin_api.go:925","plugin_id":"com.mattermost.nps"}
{"timestamp":"2022-06-06 17:13:49.883 Z","level":"info","msg":"Worker: Job is complete","caller":"migrations/worker.go:109","worker":"Migrations","job_id":"ffu6sdu8wjgptmycqkry46amae"}
{"timestamp":"2022-06-06 17:14:15.605 Z","level":"info","msg":"Stopping Server...","caller":"app/server.go:922"}
{"timestamp":"2022-06-06 17:14:15.605 Z","level":"info","msg":"stopping websocket hub connections","caller":"app/web_hub.go:113"}
{"timestamp":"2022-06-06 17:14:22.031 Z","level":"info","msg":"Stopped workers","caller":"jobs/workers.go:353"}
{"timestamp":"2022-06-06 17:14:22.031 Z","level":"info","msg":"Stopping schedulers.","caller":"jobs/schedulers.go:187"}
{"timestamp":"2022-06-06 17:14:22.031 Z","level":"info","msg":"Schedulers stopped.","caller":"jobs/schedulers.go:119"}
{"timestamp":"2022-06-06 17:14:22.032 Z","level":"info","msg":"Server stopped","caller":"app/server.go:1002"}
{"timestamp":"2022-06-06 17:14:22.032 Z","level":"info","msg":"Shutting down plugins","caller":"app/plugin.go:339"}
{"timestamp":"2022-06-06 17:14:22.034 Z","level":"info","msg":"2022/06/06 17:14:22 [ERR] plugin: plugin server: accept unix /tmp/plugin020812118: use of closed network connection\n","caller":"io/io.go:425","plugin_id":"com.mattermost.nps","source":"plugin_stderr"}
{"timestamp":"2022-06-06 17:14:22.302 Z","level":"info","msg":"Server.Shutdown","caller":"app/plugin_api.go:925","plugin_id":"focalboard"}

I can’t find any cause of the shutdown in the logs, but someone more experienced maybe can. I am also relatively new to docker and docker stack, so it is entirely possible that I have made a stupid mistake there.

Thanks in advance!