Unresponsive Mattermost

Troubleshooting
1

Summary
Mattermost is repsonding extremely slow, and a lot of actions don’t even come through resulting in an error, or just nothing.

Steps to reproduce
Mattermost Version: 9.9.2
Database Schema Version: 124
Build Number: 9962179521
Database: postgres

Expected behavior
A normal responding Mattermost

Observed behavior
Channels aren’t loaded, unable to create DM’s, unable to create sidebar, unable to get drafts. Besides these problems which occur all the time, every now and then no one is able to login and receives the error: “Unable to save the token”

"error": "createDirectChannelWithUser: Unable to save direct channel., default_channel_roles_select: context deadline exceeded",
"error": "createInitialSidebarCategories: Unable to create initial sidebar categories for user., CreateInitialSidebarCategories: GetTeamsForUser: failed to find TeamMembers with userId=j1n7rfm3wj8h5non8684fchkwc: context deadline exceeded",
"error": "UpdateSidebarCategories: Failed to insert record to database., failed to find SidebarCategories: resource \"SidebarCategories\" not found, id: channels_u16m6rek7pbp5ridkx61ne1ija_o3etsg38gf8ijcua4xziemjaaw, error: context deadline exceeded",
"error": "GetDraftsForUser: Unable to get user's Drafts., failed to get user drafts: pq: canceling statement due to user request",
"error": "GetChannelsForUser: Unable to get the channels., failed to get channels with UserId=wxq69h3bqtyntc43g7ox61t44w: context deadline exceeded",

It seems to boil down to the context deadline exceeded error, but we can’t find the real issue behind it.
We already executed a VACUUM ANALYZE on the database, and extended the memory from 8gb to 16gb.

2

Mattermost is having difficulty with the database. Is the postgres service running? What do its logs say? Can you execute a manual query with psql? What’s the CPU / memory / disk / network IO like? I have a feeling you’re running into some kind of resource limitation.

Also, 9.9.3 is out. I don’t believe that’s the issue, but there’s always a possibility that you’ve encountered some odd corner case. v9 changelog - Mattermost documentation has details of changes between versions.