Gitlab omnibus 8.9.3 can not upgrade db to mattermost 3, panic no language found in config.json

So the crazy upgrade breakage that I see discussed back in the march timeframe is still making life difficult with the latest 8.9.3 omnibus gitlab packages.

Is it because we did the upgrade directly from 8.8 to 8.9.3? How do I fix this?

I can’t run this, while sudo’d as mattermost:

 /opt/gitlab/embedded/bin/mattermost -config=/var/opt/gitlab/mattermost/config.json -upgrade_db_30

If I do I just get this horror show:

mattermost@gitlab:~$ /opt/gitlab/embedded/bin/mattermost -config=/var/opt/gitlab/mattermost/config.json -upgrade_db_30
panic: no language found in "config.json"

goroutine 1 [running]:
panic(0xc91cc0, 0xc820200ef0)
        /usr/local/go/src/runtime/panic.go:481 +0x3e6

Googling around I tried copying the en.json file to various places but still get this error.

First, mattermost seems to be extremely primitively coded. Does it really need to find en.json in order to run the db upgrade? Does it need to panic when it can’t find a configuration entry?