Mattermost service not starting

Troubleshooting
1

after upgrading mattermost package from 10.something to 11.6 the service fails to start.

journalctl -xeu mattermost.service reveals error:


May 07 18:18:39 mma mattermost[10231]: Error: failed to load configuration: failed to create store: unable to load on store creation: invalid config: LogSettings.isValid: Invalid format: invalid log level id 100, invalid log level id 100
May 07 18:18:39 mma mattermost[10231]: Usage:
May 07 18:18:39 mma mattermost[10231]:   mattermost [flags]
May 07 18:18:39 mma mattermost[10231]:   mattermost [command]
May 07 18:18:39 mma mattermost[10231]: Available Commands:
May 07 18:18:39 mma mattermost[10231]:   completion  Generate the autocompletion script for the specified shell
May 07 18:18:39 mma mattermost[10231]:   db          Commands related to the database
May 07 18:18:39 mma mattermost[10231]:   export      Export data from Mattermost
May 07 18:18:39 mma mattermost[10231]:   help        Help about any command
May 07 18:18:39 mma mattermost[10231]:   import      Import data.
May 07 18:18:39 mma mattermost[10231]:   jobserver   Start the Mattermost job server
May 07 18:18:39 mma mattermost[10231]:   server      Run the Mattermost server
May 07 18:18:39 mma mattermost[10231]:   version     Display version information
May 07 18:18:39 mma mattermost[10231]: Flags:
May 07 18:18:39 mma mattermost[10231]:   -c, --config string   Configuration file to use.
May 07 18:18:39 mma mattermost[10231]:   -h, --help            help for mattermost
May 07 18:18:39 mma mattermost[10231]: Use “mattermost [command] --help” for more information about a command.
May 07 18:18:39 mma systemd[1]: mattermost.service: Main process exited, code=exited, status=1/FAILURE

How to fix it? What are valid log levels? This 100 with previous version worked just fine… in fact it itself put this level 100 there into the configuration file.

$ grep -n '"LogSettings"' -A45 /opt/mattermost/config/config.json
170:    "LogSettings": {
171-        "EnableConsole": false,
172-        "ConsoleLevel": "INFO",
173-        "ConsoleJson": false,
174-        "EnableColor": false,
175-        "EnableFile": true,
176-        "FileLevel": "INFO",
177-        "FileJson": true,
178-        "FileLocation": "",
179-        "EnableWebhookDebugging": true,
180-        "EnableDiagnostics": true,
181-        "EnableSentry": true,
182-        "AdvancedLoggingJSON": {
183-            "file_1": {
184-                "format": "plain",
185-                "levels": [
186-                    {
187-                        "id": 100,
188-                        "name": "audit-api"
189-                    },
190-                    {
191-                        "id": 101,
192-                        "name": "audit-content"
193-                    },
194-                    {
195-                        "id": 102,
196-                        "name": "audit-permissions"
197-                    },
198-                    {
199-                        "id": 103,
200-                        "name": "audit-cli"
201-                    }
202-                ],
203-                "maxqueuesize": 1000,
204-                "options": {
205-                    "compress": true,
206-                    "filename": "/opt/mattermost/logs/audit.log",
207-                    "max_age": 1,
208-                    "max_backups": 10,
209-                    "max_size": 500
210-                },
211-                "type": "file"
212-            }
213-        },
214-        "MaxFieldSize": 2048
215-    },

the advanced logging config part was probably from this webpage:

which previously worked or at least didn’t give any errors. now it just dies.

2

removed the content from AdvancedLoggingJSON and it started…

"AdvancedLoggingJSON": {}

still really confusing what I am supposed to really put there and what will it actually log.

3

Still… when I try to enable AdvancedLogging under Audit logging from this page:

the sample for v11 is:

"AdvancedLoggingJSON": {
    "file_1": {
        "type": "file",
        "format": "plain",
        "format_options": {
            "disable_level": false
        },
        "levels": [
            { "id": 100, "name": "audit-api" },
            { "id": 101, "name": "audit-content" },
            { "id": 102, "name": "audit-permissions" },
            { "id": 103, "name": "audit-cli" }
        ],
        "options": {
            "compress": true,
            "filename": "./logs/audit.log",
            "max_age": 1,
            "max_backups": 10,
            "max_size": 500
        },
        "maxqueuesize": 1000
    },
    "file_notifications": {
        "type": "file",
        "format": "json",
        "levels": [
            { "id": 300, "name": "NotificationError" },
            { "id": 301, "name": "NotificationWarn" },
            { "id": 302, "name": "NotificationInfo" },
            { "id": 303, "name": "NotificationDebug" },
            { "id": 304, "name": "NotificationTrace" }
        ],
        "options": {
            "compress": true,
            "filename": "./logs/notifications.log",
            "max_age": 7,
            "max_backups": 5,
            "max_size": 200
        },
        "maxqueuesize": 1000
    }
}

and when I try to insert it into the config via UI I get this error

(!) Invalid format: invalid log level id 300:

image
image1505×1252 81.3 KB