Unable to save the team member

Troubleshooting
1

Summary

I cannot invite/add a user to a team.

Steps to reproduce

The user already exists on the Mattermost instance and already is in a team. I tried adding them to a second team using the invite feature, but the link didn’t seem to work. So I went and tried with the command line:

$ mattermost team add team-name user@example.org
Unable to add 'user@example.org' to team-name

So I tried using the api:

curl -X POST -i -H 'Authorization: Bearer myadmintoken' -d '{"team_id":"***redacted***","user_id":"***redacted***"}' http://localhost:19065/api/v4/teams/***redacted***/members

which gives me:

{"id":"store.sql_team.save_member.save.app_error","message":"Unable to save the team member.","detailed_error":"","request_id":"fitsy5npn3gemempy93txeojrw","status_code":500}

$ mattermost version
Version: 5.22.0
Build Number: nixpkgs-5.22.0
Build Date: 
Build Hash: 
Build Enterprise Ready: 
DB Version: 5.22.0

Postgresql version: 11.7

Thanks in advance for you help

2

Hi there, @risson

Can you share the log entry on mattermost.log when you ran the API & command line?

tail -f /opt/mattermost/log/mattermost.log

I just want to understand if this could be related to the maximum number of users per team that we can confirm by running the following:

ahmaddanial@mattermost:/opt/mattermost$ cat /opt/mattermost/config/config.json | grep "MaxUsersPerTeam"
        "MaxUsersPerTeam": 50,

Keep me posted. Thanks!

3

Hi!

First, thanks for take the time to answer me :smiley:

Here’s what happens in the logs when I try to add the user using the command line:

{"level":"info","ts":1587660811.8833458,"caller":"utils/i18n.go:83","msg":"Loaded system translations","for locale":"en","from locale":"/srv/mattermost/i18n/en.json"}
{"level":"info","ts":1587660811.8959937,"caller":"app/server_app_adapters.go:58","msg":"Server is initializing..."}
{"level":"debug","ts":1587660811.8960571,"caller":"utils/html.go:30","msg":"Parsing server templates","templates_directory":"/srv/mattermost/templates"}
{"level":"info","ts":1587660811.8976185,"caller":"sqlstore/supplier.go:224","msg":"Pinging SQL","database":"master"}
{"level":"debug","ts":1587660811.954607,"caller":"sqlstore/preference_store.go:41","msg":"Deleting any unused pre-release features"}
{"level":"debug","ts":1587660812.22013,"caller":"filesstore/s3store.go:82","msg":"Connection to S3 or minio is good. Bucket exists."}
{"level":"info","ts":1587660812.2241628,"caller":"mlog/log.go:167","msg":"Starting up plugins"}
{"level":"info","ts":1587660812.224219,"caller":"app/plugin.go:202","msg":"Syncing plugins from the file store"}
{"level":"debug","ts":1587660812.2402098,"caller":"plugin/environment.go:506","msg":"Enabling plugin health check job","interval_s":30}
{"level":"info","ts":1587660812.2402604,"caller":"app/server.go:252","msg":"Current version is 5.22.0 (nixpkgs-5.22.0///)","current_version":"5.22.0","build_number":"nixpkgs-5.22.0","build_date":"","build_hash":"","build_hash_enterprise":""}
{"level":"info","ts":1587660812.240295,"caller":"app/server.go:263","msg":"Team Edition Build","enterprise_build":false}
{"level":"info","ts":1587660812.2403145,"caller":"app/server.go:267","msg":"Printing current working","directory":"/srv/mattermost"}
{"level":"info","ts":1587660812.2403274,"caller":"app/server.go:268","msg":"Loaded config","source":"postgres://mattermost:@localhost:5432/mattermost?sslmode=disable&connect_timeout=10"}
{"level":"debug","ts":1587660812.2403467,"caller":"plugin/health_check.go:31","msg":"Plugin health check job starting."}
{"level":"info","ts":1587660812.3798344,"caller":"sqlstore/post_store.go:1518","msg":"Post.Message has size restrictions","max_characters":4000,"max_bytes":0}
{"level":"debug","ts":1587660812.4565775,"caller":"jobs/schedulers.go:30","msg":"Initialising schedulers."}
Unable to add 'user@example.org to team-name
{"level":"info","ts":1587660812.463866,"caller":"app/server.go:399","msg":"Stopping Server..."}
{"level":"info","ts":1587660812.4639013,"caller":"app/web_hub.go:130","msg":"stopping websocket hub connections"}
{"level":"warn","ts":1587660812.4639175,"caller":"app/web_hub.go:135","msg":"We appear to have already sent the stop checking for deadlocks command"}
{"level":"info","ts":1587660812.4641974,"caller":"app/plugin.go:279","msg":"Shutting down plugins"}
{"level":"info","ts":1587660812.464493,"caller":"app/server.go:450","msg":"Server stopped"}

And with the API:

Apr 23 18:56:07 duck hmdhx8x473fi3fvivb84dps7nfsz7bs4-unit-script-mattermost-start[10708]: {"level":"debug","ts":1587660967.5912883,"caller":"web/handlers.go:85","msg":"Received HTTP request","method":"POST","url":"/api/v4/users/status/ids","request_id":"j1qqifgyp7ni5x7qpjtf3texzr"}
Apr 23 18:56:10 duck hmdhx8x473fi3fvivb84dps7nfsz7bs4-unit-script-mattermost-start[10708]: {"level":"debug","ts":1587660970.6370542,"caller":"web/handlers.go:85","msg":"Received HTTP request","method":"POST","url":"/api/v4/teams/***team_id***/members","request_id":"3hzg4hz94iyxfn151z77p1y4ge"}
Apr 23 18:56:10 duck hmdhx8x473fi3fvivb84dps7nfsz7bs4-unit-script-mattermost-start[10708]: {"level":"error","ts":1587660970.642299,"caller":"mlog/log.go:175","msg":"Unable to save the team member.","path":"/api/v4/teams/**team_id***/members","request_id":"3hzg4hz94iyxfn151z77p1y4ge","ip_addr":"::1","user_id":"***user_id***","method":"POST","err_where":"SqlTeamStore.SaveMember","http_code":500,"err_details":"team_id=***team_id***, user_id=***user_id***, pq: invalid input syntax for integer: \"true\""}

MaxUsersPerTeam is 50, and we currently are 3 in the team.

Thanks again!

4

@risson, this appears to be the crux of the problem. Let me investigate further.

1 Like
5

@risson, I can’t reproduce this locally, but I’m wondering if there’s a schema mismatch at play.

Can you share the schema you have for TeamMembers in your Postgres database?

1 Like
6

Here it is:

mattermost=# \d teammembers 
                      Table "public.teammembers"
   Column    |         Type          | Collation | Nullable | Default 
-------------+-----------------------+-----------+----------+---------
 teamid      | character varying(26) |           | not null | 
 userid      | character varying(26) |           | not null | 
 roles       | character varying(64) |           |          | 
 deleteat    | numeric               |           |          | 
 schemeguest | smallint              |           |          | 
 schemeuser  | smallint              |           |          | 
 schemeadmin | smallint              |           |          | 
Indexes:
    "idx_20793_primary" PRIMARY KEY, btree (teamid, userid)
    "idx_20793_idx_teammembers_delete_at" btree (deleteat)
    "idx_20793_idx_teammembers_team_id" btree (teamid)
    "idx_20793_idx_teammembers_user_id" btree (userid)
    "idx_teammembers_delete_at" btree (deleteat)
    "idx_teammembers_team_id" btree (teamid)
    "idx_teammembers_user_id" btree (userid)
7

Any news on this? @jesse @ahmaddanial

8

The schema for the scheme* columns is unexpected from my perspective: these should be booleans, not smallints. I note we attempted to converge on a uniform schema definition as part of https://mattermost.atlassian.net/browse/MM-16888, but something was missed. It’s likely the canonical definition of these columns doesn’t match the “upgraded” version, but is being missed by the current schema tests.

@amy.blais, could you promote this issue to Jira ticket? It likely needs to be investigated by our server team.

@risson, you can fix this manually by altering the schema appropriately. Please note that we do not usually recommend manually modifying the schema, and ask that you make a backup before effecting any changes should issues arise.

ALTER TABLE teammembers ALTER COLUMN schemeguest TYPE BOOLEAN
ALTER TABLE teammembers ALTER COLUMN schemeuser TYPE BOOLEAN
ALTER TABLE teammembers ALTER COLUMN schemeadmin TYPE BOOLEAN

If you have a large number of team members, I would recommend doing this during non-peak hours.

9

@risson, one other question: do you have issues adding team members via the webapp vs. just the CLI? This would be unexpected as well, so it would be good to note.

10

@jesse I’m getting the error using the CLI, the API and the webapp, even after the schema change

Here are the logs from the webapp (after the schema change):

Apr 27 19:25:59 duck hmdhx8x473fi3fvivb84dps7nfsz7bs4-unit-script-mattermost-start[7968]: {"level":"error","ts":1588008359.5073829,"caller":"mlog/log.go:175","msg":"Unable to get the channel member count.","path":"/api/v4/channels/khex1dtzxid1drfktze3r9gqeh/stats","request_id":"np8ktq38widybygqztykcjhc1o","ip_addr":"90.126.230.161","user_id":"bp93hwzseidwumkx9skj48qbgc","method":"GET","err_where":"SqlChannelStore.GetGuestCount","http_code":500,"err_details":"channel_id=khex1dtzxid1drfktze3r9gqeh, pq: operator does not exist: smallint = boolean"}

From the CLI, I’m still getting the same error.

Here is what I’m getting when making a call to the API, which is different from before:

Apr 27 19:29:12 duck hmdhx8x473fi3fvivb84dps7nfsz7bs4-unit-script-mattermost-start[7968]: {"level":"error","ts":1588008552.0222657,"caller":"mlog/log.go:175","msg":"Unable to save the team member.","path":"/api/v4/teams/pctwzbq83trxzfzym1jibrt4pw/members","request_id":"cdxfrmjgs3r8bk7icsgb1q597y","ip_addr":"::1","user_id":"bp93hwzseidwumkx9skj48qbgc","method":"POST","err_where":"SqlTeamStore.SaveMember","http_code":500,"err_details":"team_id=pctwzbq83trxzfzym1jibrt4pw, user_id=d639bk9tk3dfxq81fcbmdundha, pq: invalid input syntax for integer: \"true\""}
11

Hold on my bad, it seems the schema wasn’t changed.

mattermost=# \d teammembers
                      Table "public.teammembers"
   Column    |         Type          | Collation | Nullable | Default 
-------------+-----------------------+-----------+----------+---------
 teamid      | character varying(26) |           | not null | 
 userid      | character varying(26) |           | not null | 
 roles       | character varying(64) |           |          | 
 deleteat    | numeric               |           |          | 
 schemeguest | smallint              |           |          | 
 schemeuser  | smallint              |           |          | 
 schemeadmin | smallint              |           |          | 
Indexes:
    "idx_20793_primary" PRIMARY KEY, btree (teamid, userid)
    "idx_20793_idx_teammembers_delete_at" btree (deleteat)
    "idx_20793_idx_teammembers_team_id" btree (teamid)
    "idx_20793_idx_teammembers_user_id" btree (userid)
    "idx_teammembers_delete_at" btree (deleteat)
    "idx_teammembers_team_id" btree (teamid)
    "idx_teammembers_user_id" btree (userid)
12 
ALTER TABLE teammembers ALTER schemeuser TYPE bool USING CASE WHEN schemeuser=0 THEN FALSE ELSE TRUE END;
ALTER TABLE teammembers ALTER schemeguest TYPE bool USING CASE WHEN schemeguest=0 THEN FALSE ELSE TRUE END;
ALTER TABLE teammembers ALTER schemeadmin TYPE bool USING CASE WHEN schemeadmin=0 THEN FALSE ELSE TRUE END;

Using this ^

13

Okay, after a proper conversion to booleans, I was able to add the user using the API, the CLI and the webapp. Thanks for your help!

14

Just a wonder, is it normal that the number of people in a channel is not displayed properly here:
2020-04-27_19-43

I’m guessing my schema is wrong here as well:

mattermost=# \d channelmembers;
                     Table "public.channelmembers"
    Column    |         Type          | Collation | Nullable | Default 
--------------+-----------------------+-----------+----------+---------
 channelid    | character varying(26) |           | not null | 
 userid       | character varying(26) |           | not null | 
 roles        | character varying(64) |           |          | 
 lastviewedat | numeric               |           |          | 
 msgcount     | numeric               |           |          | 
 mentioncount | numeric               |           |          | 
 notifyprops  | text                  |           |          | 
 lastupdateat | numeric               |           |          | 
 schemeguest  | smallint              |           |          | 
 schemeuser   | smallint              |           |          | 
 schemeadmin  | smallint              |           |          | 
Indexes:
    "idx_20617_primary" PRIMARY KEY, btree (channelid, userid)
    "idx_20617_idx_channelmembers_channel_id" btree (channelid)
    "idx_20617_idx_channelmembers_user_id" btree (userid)
    "idx_channelmembers_channel_id" btree (channelid)
    "idx_channelmembers_user_id" btree (userid)
15

Opened a ticket here: https://mattermost.atlassian.net/browse/MM-24573

1 Like