Akito
March 12, 2022, 3:49am
1
Summary
Steps to reproduce
Install the, as of today, newest Mattermost Helm Chart .
Try to install any plugin.
Watch the logs.
Expected behavior
Observed behavior
The issue is of the following type:
Hello,
I try the last 3 version of jira plugins and read the doc related to, but the plugins is deleted on each starting of the mattermost server.
I have mattermost 5.1.0 installed.
log :
{“level”:“info”,“ts”:1534491808.960478,“caller”:“app/plugin.go:540”,“msg”:“Starting up plugins”}
{“level”:“error”,“ts”:1534491809.155748,“caller”:“app/plugin.go:592”,“msg”:“Failed to install prepackaged plugin”,“error”:“installPlugin: Unable to move plugin from temporary directory to final destination. An…
opened 03:16AM - 19 Feb 21 UTC
Bug Report/Open
Mattermost Version: [5.31.0](https://github.com/mattermost/mattermost-docker/blo… b/2b12534f900f51a485c47bcd880a61df1b20beb7/app/Dockerfile#L5)
#### Summary
Upgrading to Enterprise version from System Console causes loss of plugins
#### Long Description
I installed MM using [this](https://github.com/mattermost/mattermost-docker/blob/master/docker-compose.yml) `docker-compose.yml` file with PortainerCE on my UNRAID server. My configuration is identical except for the host volume locations and web port mappings.
I then installed a bunch of plugins (github, gif,meme, jitsi, todo etc) after configuring my team and inviting a few users.
I wanted to try the `Incident Collaboration` plugin but saw that it required an Enterprise Version
I opened the System Console and clicked on the `Upgrde to Enterprise button`:
I installed the plugin and that seemed to work. I created a playbook etc
Later, when I tried to install the `Channel Export` plugin which also requires Enterprise license, I got this error:
I checked the logs in docker's mattermost_app_1 container and found this:
```
{"level":"error","ts":1613664206.8584263,"caller":"mlog/log.go:229","msg":"Unable to move plugin from temporary directory to final destination. Another plugin may be using the same directory name.","path":"/api/v4/plugins/marketplace","request_id":"ncd9h619njf4dmobwsnbmwx8ey","ip_addr":"172.27.0.3","user_id":"46gef9hpofrb98q7pyp4u4nxpa","method":"POST","err_where":"installExtractedPlugin","http_code":500,"err_details":"destination already exists"}
```
I then checked the Plugins Marketplace and I no longer had any plugins:
However, if I list the plugins folder, they are all there but with empty contents:
```
/mattermost/plugins # ls -lasth
total 0
0 drwxr--r-- 1 mattermo mattermo 12 Feb 17 21:10 com.github.moussetc.mattermost.plugin.giphy
0 drwxr--r-- 1 mattermo mattermo 12 Feb 17 21:10 com.mattermost.nps
0 drwxr--r-- 1 mattermo mattermo 12 Feb 17 21:10 com.mattermost.plugin-channel-export
0 drwxr--r-- 1 mattermo mattermo 12 Feb 17 21:10 com.mattermost.plugin-incident-management
0 drwxr--r-- 1 mattermo mattermo 12 Feb 17 21:10 com.mattermost.plugin-todo
0 drwxr--r-- 1 mattermo mattermo 12 Feb 17 21:10 com.mattermost.welcomebot
0 drwxr--r-- 1 mattermo mattermo 12 Feb 17 21:10 com.nilsbrinkmann.icebreaker
0 drwxr--r-- 1 mattermo mattermo 12 Feb 17 21:10 github
0 drwxr--r-- 1 mattermo mattermo 12 Feb 17 21:10 jitsi
0 drwxr--r-- 1 mattermo mattermo 12 Feb 17 21:10 memes
0 drwxrwxrwx 1 99 users 466 Feb 17 21:01 .
0 drwxr-sr-x 1 mattermo mattermo 242 Feb 13 00:37 ..
/mattermost/plugins #
```
```
mattermost # tree plugins/
plugins/
├── com.github.moussetc.mattermost.plugin.giphy
│ └── server
│ └── dist
├── com.mattermost.nps
│ └── server
│ └── dist
├── com.mattermost.plugin-channel-export
│ └── server
│ └── dist
├── com.mattermost.plugin-incident-management
│ └── server
│ └── dist
├── com.mattermost.plugin-todo
│ └── server
│ └── dist
├── com.mattermost.welcomebot
│ └── server
│ └── dist
├── com.nilsbrinkmann.icebreaker
│ └── server
│ └── dist
├── github
│ └── server
│ └── dist
├── jitsi
│ └── server
│ └── dist
└── memes
└── server
└── dist
```
I switched back to Free version from within System Console and it still shows (0) plugins installed.
I restarted the server and ran ` chown -R mattermost:mattermost /mattermost` but still the Installed plugins tab shows as (0)
At this point no plugins worked:
1- I could non longer see any plugins on the WebUI or run them with slash commands
2- I could not install them from the UI because it says the folder already exists
Finally, I was able to recover by stopping the server, deleting the contents of `plugins` and `client/plugins` and starting the server up again. I was about to re-install all the plugins from scratch again.
However, the folders magically re-populated and work now (except for jitsi plugin which complains about a missing App ID)
#### Expected behavior
I expect to be able to upgrade to Enterprise and not lose all plugins.
#### Question:
What is the difference between the files located in `/mattermost/client/plugins` and `/mattermost/plugins`. They seem similar but not identical
```
/ # ls -lasth /mattermost/plugins/ /mattermost/client/plugins/
/mattermost/client/plugins/:
total 0
0 drwxr-xr-x 1 mattermo mattermo 272 Feb 18 20:48 .
0 drwxr--r-- 1 mattermo mattermo 64 Feb 18 20:48 jitsi
0 drwxr--r-- 1 mattermo mattermo 100 Feb 18 20:48 zoom
0 drwxr--r-- 1 mattermo mattermo 136 Feb 18 20:05 com.mattermost.plugin-incident-management
0 drwxr--r-- 1 mattermo mattermo 90 Feb 18 19:51 com.mattermost.nps
0 drwxr--r-- 1 mattermo mattermo 106 Feb 18 19:51 com.mattermost.plugin-todo
0 drwxr--r-- 1 mattermo mattermo 66 Feb 18 19:51 github
0 drwxr--r-- 1 mattermo mattermo 126 Feb 18 19:51 com.mattermost.plugin-channel-export
0 drwxr-xr-x 1 mattermo mattermo 4.3K Feb 13 00:37 ..
/mattermost/plugins/:
total 0
0 drwxr--r-- 1 mattermo mattermo 78 Feb 18 20:37 zoom
0 drwxr-xr-x 1 mattermo mattermo 392 Feb 18 20:37 .
0 drwxr--r-- 1 mattermo mattermo 78 Feb 18 20:11 jitsi
0 drwxr--r-- 1 mattermo mattermo 78 Feb 18 19:51 com.mattermost.nps
0 drwxr--r-- 1 mattermo mattermo 78 Feb 18 19:51 com.mattermost.plugin-todo
0 drwxr--r-- 1 mattermo mattermo 66 Feb 18 19:51 com.nilsbrinkmann.icebreaker
0 drwxr--r-- 1 mattermo mattermo 78 Feb 18 19:51 github
0 drwxr--r-- 1 mattermo mattermo 54 Feb 18 19:51 memes
0 drwxr--r-- 1 mattermo mattermo 66 Feb 18 19:51 com.github.moussetc.mattermost.plugin.giphy
0 drwxr--r-- 1 mattermo mattermo 90 Feb 18 19:51 com.mattermost.plugin-channel-export
0 drwxr--r-- 1 mattermo mattermo 54 Feb 18 19:51 com.mattermost.welcomebot
0 drwxr-sr-x 1 mattermo mattermo 242 Feb 13 00:37 ..
```
And many other issues just like that…
Here is the log excerpt, which is always the same across all issues, including mine:
"Unable to move plugin from temporary directory to final destination. Another plugin may be using the same directory name."
The difference between my issue and their is, that my permissions are actually correct and I’m on the newest Mattermost version, so I don’t have old bugs lingering, neither do I have actual permission errors on the OS.
The OS has chowned all the directories, including the plugin directory to the system user mattermost with the id 123. The securityContext in the Chart is set entirely to 123 and I checked the running pod and deployment specs. All show the expected and desired securityContext instances, entirely consisting of 123 owners and groups.
Everything on the OS and in the Kubernetes objects is set to use the user with the id 123 but the permission error still appears.
Of course, I already chowned the directories several times, I also set the permissions to 770 and not even that helped.
Somewhere in-between must be a place where the owner is not correctly applied or used. Because, it’s set everywhere correctly.
Akito
March 12, 2022, 3:16pm
2
Apparently, the user mattermost inside the Docker Image is hard-coded to user and group ID 2000. Even though, the mattermost process is clearly running as the user (ID) I desire, it still uses the mattermost user with the ID 2000 somewhere in-between.
This is clearly a bug, as it would mean, that this app can collide with any other map, which uses the same ID for its user.
Akito
March 12, 2022, 3:37pm
3
See the corresponding Github issue:
opened 03:23PM - 12 Mar 22 UTC
See the initial issue:
https://forum.mattermost.com/t/permission-error-but-pe… rmissions-are-permissive/13080
> Apparently, the user `mattermost` inside the Docker Image is hard-coded to user and group ID `2000`. Even though, the `mattermost` process is clearly running as the user (ID) I desire, it still uses the `mattermost` user with the ID `2000` somewhere in-between.
>
> This is clearly a bug, as it would mean, that this app can collide with any other map, which uses the same ID for its user.
It's necessary for the Chart setup to respect the provided `securityContext`. If I provide the user and group ID `123` and the `mattermost` process actually runs with those (it actually does, I checked), then the classic `mattermost` user with the ID `2000` shouldn't remain the user in charge.
Related Issues:
#128
#142