After some hours of investigation, I was able to fix the problem thanks to this post: http://forum.mattermost.com/t/solved-cant-login-after-upgrade-from-3-2-0-to-3-4-0/2240
Steps to fix:
sudo -u gitlab-psql /opt/gitlab/embedded/bin/psql -h /var/opt/gitlab/postgresql/ mattermost_production
mattermost_production=# UPDATE Systems SET Value = '3.1.0' WHERE Name = 'Version'; mattermost_production-# \q
sudo gitlab-ctl reconfigure sudo gitlab-ctl restart