Size limit exceeded in mmctl import

Troubleshooting
1

Hi. I’ve been using Mattermost for many years and recently decided I would finally get away from MySQL and move to Postgres, so I installed the omnibus version and am using mmctl to export data from the older server. But I keep getting an error during the import about ‘size limit exceeded.’

I have adjusted nginx and the system console to 20gb to allow more than enough size for the export zip (17gb) so I don’t think it’s related to that.

The source server is running 9.4.2 on Mint 21.1 and MySQL 8.0.35. New server is Ubuntu 22.04.3 LTS with latest & greatest omnibus ver 9.4.2 & Postgres.

Everything runs perfectly until the import of what appears to be custom emojis. A log snippet:

{"timestamp":"2024-02-05 00:13:25.256 Z","level":"info","msg":"Importing emoji","caller":"app/import_functions.go:1949","emoji_name":"call"}
{"timestamp":"2024-02-05 00:13:25.263 Z","level":"info","msg":"Bulk import worker finished","caller":"app/import.go:121","bulk_import_worker_id":"dzc9ag5as7bwjknmooxqfkn9ro","processed_lines":16}
{"timestamp":"2024-02-05 00:13:25.273 Z","level":"info","msg":"Bulk import worker finished","caller":"app/import.go:121","bulk_import_worker_id":"pehdkmr543nspxhs4howi71dby","processed_lines":17}
{"timestamp":"2024-02-05 00:13:25.273 Z","level":"error","msg":"SimpleWorker: job execution error","caller":"jobs/base_workers.go:96","worker_name":"ImportProcess","job_id":"ykhpng9pjbf93eue6apnawx5re","job_type":"import_process","job_create_at":"Feb  5 00:13:01.146","error":"WriteFile: Unable to write the file., unable write the data in the file /var/opt/mattermost/data/emoji/1wirbaodqjn8fd9rsbg7szf94y/image: size limit exceeded"}

I do have a handful of emojis that are greater than 512kb so I’m wondering if that’s what’s causing this. And if so, should I delete them? I tried exploring the zip to find the files referenced but I don’t see the directories or files listed in the log so I am very confused.

I searched pretty extensively and only saw what appeared to be issues related to ‘client_max_body_size’ which I don’t believe is an issue. So I’m hoping someone has a suggestion as to how to work around this. Thanks in advance.

2

Hi Steven,

Thank you for your detailed information regarding the issue you’re experiencing and for the effort you’ve put into troubleshooting on your side.

As you deduced, it seems the import error may be connected to the file size of some emojis. Mattermost has a restriction for custom emoji uploads and it applies even during import processes. By default, image files for custom emojis must be no larger than 512KB. That’s probably the reason why you’re unable to see those files in the export as they might have been omitted due to their file size.

Nonetheless, here are the steps you can take to circumvent this problem:

  1. Removing Large Emojis: Since you’ve observed that some emojis are larger than the allowed size, consider deleting them and then try the import process again.

  2. Increasing the Max File Size: Another approach could be temporarily increasing the maximum file size for emoji uploads on your new Mattermost instance. Head to System Console > Environment > File Storage > Maximum File Size and set a higher value there. After the import is successful, revert the size to its original limitation.

  3. Manual Transfer: If the above 2 steps do not work, you might want to consider a manual approach where you manually copy your custom emojis over to the new instance. Be aware that this may be a more tedious process and should be considered as a last resort.

Remember to backup your data before making adjustments.

I hope these options helps. If you’re still experiencing difficulties, please don’t hesitate to let us know so we can continue to assist. Hang in there, we’ll figure this out together!

Best,
~Mattermost AI Assistant :robot:

[Disclaimer: This was an AI assisted response powered by Mattermost AI. Were any of the above suggestions inaccurate? Let us know by replying to this comment!]

3

Thanks for the reply, John! And Mattermost AI :grinning:

I started down the road of doing something like what you’re suggesting in item 1. What I discovered is that when I delete an offending emoji it leaves the image in the folder and just seems to rewrite it as ‘image_deleted.’ But that deleted image retains the original file properties, meaning it still kills the transfer process.

I know from my nearly eight years of Mattermost experience that it’s not a good idea to delete files from the server – even if those files have been deleted within the MM instance, they remain in the /data directories.

Are you confirming that it is safe to physically remove those larger emojis prior to the export on the old system? I have several hundred custom emojis so it’ll take some manual work – but if you think this won’t break MM then I will do it. Thanks again.

4

I worked through this issue and wanted to mark it as resolved for anyone that may experience this.

What I ultimately did was search all emojis above 500k and manually confirm what the emoji was (the /data/emoji/* directories use only the name ‘image’ so they have to be matched manually) Then I deleted the emoji itself from within the MM client on the source server (do NOT just delete or rename the files on the file system or it will cause consistency issues on import).

Once I got to the point where all of the offending emojis were showing as ‘image_deleted’ I was able to export and then import the zip export on the new server. It appears that the ‘image_deleted’ files, even though they remain on the file system as >512k files, are ignored by the export process.

Screenshot 2024-03-04 062701