Failed to get channel members after upgrade

Troubleshooting
1

Hi everyone just updated to version 6.5. When login in mattermost says no available teams to join.
When looking at the log I found the following error:
imestamp":“2022-03-20 03:54:52.832 Z”,“level”:“error”,“msg”:“Unable to get the channels.”,“caller”:“web/context.go:105”,“path”:"/api/v4/teams/91xzx6efg7d9pm4iwmxpyoz97o/members/mx4w4iusfbnmfcwgwk9qgtamga",“request_id”:“1hryif37ciypjcnnnu4766obgy”,“ip_addr”:“127.0.0.1”,“user_id”:“mx4w4iusfbnmfcwgwk9qgtamga”,“method”:“DELETE”,“err_where”:“LeaveTeam”,“http_code”:500,“err_details”:“failed to get channels with TeamId=91xzx6efg7d9pm4iwmxpyoz97o and UserId=mx4w4iusfbnmfcwgwk9qgtamga: sql: Scan error on column index 17, name “TotalMsgCountRoot”: converting NULL to int64 is unsupported”}
{“timestamp”:“2022-03-20 03:54:55.258 Z”,“level”:“error”,“msg”:“Unable to get the teams unread messages.”,“caller”:“web/context.go:105”,“path”:"/api/v4/users/me/teams/unread",“request_id”:“jihasdok1bg5xr3khbb8iuirqa”,“ip_addr”:“127.0.0.1”,“user_id”:“mx4w4iusfbnmfcwgwk9qgtamga”,“method”:“GET”,“err_where”:“GetTeamsUnreadForUser”,“http_code”:500,“err_details”:“failed to find Channels with userId=mx4w4iusfbnmfcwgwk9qgtamga and teamId!=: sql: Scan error on column index 3, name “MsgCountRoot”: converting NULL to int64 is unsupported”}
{“timestamp”:“2022-03-20 03:54:55.267 Z”,“level”:“error”,“msg”:“Unable to get the channels.”,“caller”:“web/context.go:105”,“path”:"/api/v4/users/me/teams/91xzx6efg7d9pm4iwmxpyoz97o/channels",“request_id”:“m1gr6a5rojy4tea639uznmyj5e”,“ip_addr”:“127.0.0.1”,“user_id”:“mx4w4iusfbnmfcwgwk9qgtamga”,“method”:“GET”,“err_where”:“GetChannelsForUser”,“http_code”:500,“err_details”:“failed to get channels with TeamId=91xzx6efg7d9pm4iwmxpyoz97o and UserId=mx4w4iusfbnmfcwgwk9qgtamga: sql: Scan error on column index 17, name “TotalMsgCountRoot”: converting NULL to int64 is unsupported”}
{“timestamp”:“2022-03-20 03:54:55.274 Z”,“level”:“error”,“msg”:“Unable to get the channel members.”,“caller”:“web/context.go:105”,“path”:"/api/v4/users/me/teams/91xzx6efg7d9pm4iwmxpyoz97o/channels/members",“request_id”:“335oa5czup8e9nsxzf8gr9bece”,“ip_addr”:“127.0.0.1”,“user_id”:“mx4w4iusfbnmfcwgwk9qgtamga”,“method”:“GET”,“err_where”:“GetChannelMembersForUser”,“http_code”:500,“err_details”:“failed to find ChannelMembers data with teamId=91xzx6efg7d9pm4iwmxpyoz97o and userId=mx4w4iusfbnmfcwgwk9qgtamga: sql: Scan error on column index 12, name “MsgCountRoot”: converting NULL to int64 is unsupported”}
{“timestamp”:“2022-03-20 03:54:55.335 Z”,“level”:“error”,“msg”:“Unable to get the channels.”,“caller”:“web/context.go:105”,“path”:"/api/v4/users/me/channels",“request_id”:“5kcdjqyxebfcbj4s9xymaubc8c”,“ip_addr”:“127.0.0.1”,“user_id”:“mx4w4iusfbnmfcwgwk9qgtamga”,“method”:“GET”,“err_where”:“GetChannelsForUser”,“http_code”:500,“err_details”:“failed to get channels with UserId=mx4w4iusfbnmfcwgwk9qgtamga: sql: Scan error on column index 17, name “TotalMsgCountRoot”: converting NULL to int64 is unsupported”}
{“timestamp”:“2022-03-20 03:55:00.464 Z”,“level”:“error”,“msg”:“Unable to get the teams unread messages.”,“caller”:“web/context.go:105”,“path”:"/api/v4/users/me/teams/unread",“request_id”:“hha8h153rbye7g5tb5om1shrkh”,“ip_addr”:“127.0.0.1”,“user_id”:“mx4w4iusfbnmfcwgwk9qgtamga”,“method”:“GET”,“err_where”:“GetTeamsUnreadForUser”,“http_code”:500,“err_details”:“failed to find Channels with userId=mx4w4iusfbnmfcwgwk9qgtamga and teamId!=: sql: Scan error on column index 3, name “MsgCountRoot”: converting NULL to int64 is unsupported”}
{“timestamp”:“2022-03-20 03:55:00.522 Z”,“level”:“error”,“msg”:“Unable to get the channels.”,“caller”:“web/context.go:105”,“path”:"/api/v4/users/me/teams/91xzx6efg7d9pm4iwmxpyoz97o/channels",“request_id”:“s41aq191x3yttb1b7rhjwfgira”,“ip_addr”:“127.0.0.1”,“user_id”:“mx4w4iusfbnmfcwgwk9qgtamga”,“method”:“GET”,“err_where”:“GetChannelsForUser”,“http_code”:500,“err_details”:“failed to get channels with TeamId=91xzx6efg7d9pm4iwmxpyoz97o and UserId=mx4w4iusfbnmfcwgwk9qgtamga: sql: Scan error on column index 17, name “TotalMsgCountRoot”: converting NULL to int64 is unsupported”}
{“timestamp”:“2022-03-20 03:55:00.522 Z”,“level”:“error”,“msg”:“Unable to get the channel members.”,“caller”:“web/context.go:105”,“path”:"/api/v4/users/me/teams/91xzx6efg7d9pm4iwmxpyoz97o/channels/members",“request_id”:“j9sb89i6c7d18n51umkj3o4w5h”,“ip_addr”:“127.0.0.1”,“user_id”:“mx4w4iusfbnmfcwgwk9qgtamga”,“method”:“GET”,“err_where”:“GetChannelMembersForUser”,“http_code”:500,“err_details”:“failed to find ChannelMembers data with teamId=91xzx6efg7d9pm4iwmxpyoz97o and userId=mx4w4iusfbnmfcwgwk9qgtamga: sql: Scan error on column index 12, name “MsgCountRoot”: converting NULL to int64 is unsupported”}
{“timestamp”:“2022-03-20 03:55:28.974 Z”,“level”:“error”,“msg”:“Unable to get the teams unread messages.”,“caller”:“web/context.go:105”,“path”:"/api/v4/users/me/teams/unread",“request_id”:“htxpunraa3n77q7qe5uu9ee1co”,“ip_addr”:“127.0.0.1”,“user_id”:“mx4w4iusfbnmfcwgwk9qgtamga”,“method”:“GET”,“err_where”:“GetTeamsUnreadForUser”,“http_code”:500,“err_details”:“failed to find Channels with userId=mx4w4iusfbnmfcwgwk9qgtamga and teamId!=: sql: Scan error on column index 3, name “MsgCountRoot”: converting NULL to int64 is unsupported”}
{“timestamp”:“2022-03-20 03:55:29.030 Z”,“level”:“error”,“msg”:“Unable to get the channels.”,“caller”:“web/context.go:105”,“path”:"/api/v4/users/me/teams/91xzx6efg7d9pm4iwmxpyoz97o/channels",“request_id”:“58gp8iab1pbwbqs1z9u6a8j1ur”,“ip_addr”:“127.0.0.1”,“user_id”:“mx4w4iusfbnmfcwgwk9qgtamga”,“method”:“GET”,“err_where”:“GetChannelsForUser”,“http_code”:500,“err_details”:“failed to get channels with TeamId=91xzx6efg7d9pm4iwmxpyoz97o and UserId=mx4w4iusfbnmfcwgwk9qgtamga: sql: Scan error on column index 17, name “TotalMsgCountRoot”: converting NULL to int64 is unsupported”}
{“timestamp”:“2022-03-20 03:55:29.031 Z”,“level”:“error”,“msg”:“Unable to get the channel members.”,“caller”:“web/context.go:105”,“path”:"/api/v4/users/me/teams/91xzx6efg7d9pm4iwmxpyoz97o/channels/members",“request_id”:“ehgwsgbjiid75bn53b5wmff6ww”,“ip_addr”:“127.0.0.1”,“user_id”:“mx4w4iusfbnmfcwgwk9qgtamga”,“method”:“GET”,“err_where”:“GetChannelMembersForUser”,“http_code”:500,“err_details”:“failed to find ChannelMembers data with teamId=91xzx6efg7d9pm4iwmxpyoz97o and userId=mx4w4iusfbnmfcwgwk9qgtamga: sql: Scan error on column index 12, name “MsgCountRoot”: converting NULL to int64 is unsupported”}
{“timestamp”:“2022-03-20 03:55:37.891 Z”,“level”:“error”,“msg”:“Unable to get the channels.”,“caller”:“web/context.go:105”,“path”:"/api/v4/teams/91xzx6efg7d9pm4iwmxpyoz97o/members/mx4w4iusfbnmfcwgwk9qgtamga",“request_id”:“xk66bwhwgpg7imdk5eg7d7qdcy”,“ip_addr”:“127.0.0.1”,“user_id”:“mx4w4iusfbnmfcwgwk9qgtamga”,“method”:“DELETE”,“err_where”:“LeaveTeam”,“http_code”:500,“err_details”:“failed to get channels with TeamId=91xzx6efg7d9pm4iwmxpyoz97o and UserId=mx4w4iusfbnmfcwgwk9qgtamga: sql: Scan error on column index 17, name “TotalMsgCountRoot”: converting NULL to int64 is unsupported”}
{“timestamp”:“2022-03-20 03:56:01.512 Z”,“level”:“error”,“msg”:“Unable to get the channels.”,“caller”:“web/context.go:105”,“path”:"/api/v4/teams/91xzx6efg7d9pm4iwmxpyoz97o/members/mx4w4iusfbnmfcwgwk9qgtamga",“request_id”:“fctfg7yshbyrbgx1nkoyrecuey”,“ip_addr”:“127.0.0.1”,“user_id”:“mx4w4iusfbnmfcwgwk9qgtamga”,“method”:“DELETE”,“err_where”:“LeaveTeam”,“http_code”:500,“err_details”:“failed to get channels with TeamId=91xzx6efg7d9pm4iwmxpyoz97o and UserId=mx4w4iusfbnmfcwgwk9qgtamga: sql: Scan error on column index 17, name “TotalMsgCountRoot”: converting NULL to int64 is unsupported”}
{“timestamp”:“2022-03-20 03:56:13.872 Z”,“level”:“info”,“msg”:“api endpoint requires a license”,“caller”:“web/context.go:103”,“path”:"/api/v4/teams/91xzx6efg7d9pm4iwmxpyoz97o/groups",“request_id”:“3rs96wu5epbz9yybotmbuidfgo”,“ip_addr”:“127.0.0.1”,“user_id”:“mx4w4iusfbnmfcwgwk9qgtamga”,“method”:“GET”,“err_where”:"",“http_code”:501,“err_details”:""}
{“timestamp”:“2022-03-20 03:56:27.808 Z”,“level”:“info”,“msg”:“api endpoint requires a license”,“caller”:“web/context.go:103”,“path”:"/api/v4/teams/91xzx6efg7d9pm4iwmxpyoz97o/groups",“request_id”:“kfm665hcubrbiksptdiod6t1sy”,“ip_addr”:“127.0.0.1”,“user_id”:“mx4w4iusfbnmfcwgwk9qgtamga”,“method”:“GET”,“err_where”:"",“http_code”:501,“err_details”:""}
{“timestamp”:“2022-03-20 03:56:27.865 Z”,“level”:“error”,“msg”:“Unable to get the channels.”,“caller”:“web/context.go:105”,“path”:"/api/v4/teams/91xzx6efg7d9pm4iwmxpyoz97o/members/mx4w4iusfbnmfcwgwk9qgtamga",“request_id”:“jyzwu7nq5idwjcwayj6q86894a”,“ip_addr”:“127.0.0.1”,“user_id”:“mx4w4iusfbnmfcwgwk9qgtamga”,“method”:“DELETE”,“err_where”:“LeaveTeam”,“http_code”:500,“err_details”:“failed to get channels with TeamId=91xzx6efg7d9pm4iwmxpyoz97o and UserId=mx4w4iusfbnmfcwgwk9qgtamga: sql: Scan error on column index 17, name “TotalMsgCountRoot”: converting NULL to int64 is unsupported”}
{“timestamp”:“2022-03-20 03:58:15.137 Z”,“level”:“warn”,“msg”:“Unrecognized config permissions tag value.”,“caller”:“api4/config.go:379”,“tag_value”:“sysconsole_write_*_read”}
{“timestamp”:“2022-03-20 04:00:17.523 Z”,“level”:“error”,“msg”:“Unable to get the teams unread messages.”,“caller”:“web/context.go:105”,“path”:"/api/v4/users/me/teams/unread",“request_id”:“gdr5feom1p8fdcuwn7iaxxdkko”,“ip_addr”:“127.0.0.1”,“user_id”:“mx4w4iusfbnmfcwgwk9qgtamga”,“method”:“GET”,“err_where”:“GetTeamsUnreadForUser”,“http_code”:500,“err_details”:“failed to find Channels with userId=mx4w4iusfbnmfcwgwk9qgtamga and teamId!=: sql: Scan error on column index 3, name “MsgCountRoot”: converting NULL to int64 is unsupported”}

Any ideas on how to fix it?

Thanks!

1 Like
2

I’m having exaclty the same problem. Any feedback would be greatly appreciated. Thanks!

3

Hey, I fixed by following the upgrade chain. In my case the version was to old to upgrade directly to the latest one.

Hope that helps.

4

Hi! Thanks for the quick reply! I’m not sure what the official upgrade chain is. I followed Upgrading Mattermost Server — Mattermost documentation and didn’t get any schema upgrade errors. Only when launching mattermost I realized that while the chat history was preserved, the channel membership associations were lost (the side bar in the web interface is empty, existing users are asked to create a new team even though they’re already members of existing teams). Did you perform any specific Postgres schema updates in your installation?

5

What I meant by upgrade chain is follow all the versions (Version Archive — Mattermost documentation). So in my case I was using version 5.25 then I upgraded to 5.26, 5.27 … until 6.5. to avoid database issues, as each version can incorporate changes to the database. That way every change is made in between versions.

At least this was how I could make it work.
Cheers!

6

Hi! Thank you so much for the clarification - that was the pointer I needed. Turns out the database migration performed in 6.x works only for databases created with versions 5.37+ (I think). We had a 5.27 installation. When upgrading to 6.3 the database migration ran and completed “successfully” (no errors) but the schema was only partially upgraded resulting in post history etc. to be present but channel- and team-memberships were lost.

So for everyone stumbling upon this thread from a web-search: if you’re running a Mattermost instance prior version 5.37, first upgrade to 5.37, then to 6.3. Thanks again AlonzoG82816200 !

@Mattermost Devs: I think it might be helpful to explicitly mention this issue in the upgrade docs. The current section for builds older than 5.35 only mentions extended downtime.

Cheers!