Summary
Recently (post upgrading to 10.8 I believe), uploading images has been very slow and is maxing out the CPU.
Steps to reproduce
Attempt to upload image, CPU spikes to 100%, and takes >10 seconds to process (progress far is entirely full, but has not yet gone away).
Expected behavior
Should be able to near instantly process images after uploading.
Observed behavior
I believe this happened post-upgrade from v10.5 → v10.8. The server is run on a very low resources server, 1 vCPU and 1GB of RAM on Digital Ocean. Historically, I’ve had no resource issues because of the very low user count (<5 people) and very rare number of simultaneous uploads. However now we’re seeing constant issues with uploading images.
After the server maxes out, they are additionally unable to send messages for like 30 seconds afterwards.
Tech specs:
- Ubuntu 22.04.4 LTS
- v10.8.0
- 1 vCPU and 1GB of RAM
- Reverse proxied with Caddy:
mydomain.com
reverse_proxy :8065
What I’ve tried:
- Rebooting the droplet
- Restarting the mattermost service
- Swapping from S3 → Local File Storage
- Bumping RAM from 1GB → 2GB
- Bumping from 1 vCPU → 2 vCPU (maxes out both CPUs and stalls)
Uploading small images (<100KB) is no problem. The issue seems to happen mostly with larger images which manifests quite often when users upload images from their phones (3-5MB). No discernible logs to look at either. Seems format agnostic, not a HEIC-specific issue. If I upload a 3MB JPG, I experience the same issue.
I do not see the issue on the community server.
Thanks in advance!