Unable to bulk import

Summary
With a fresh Ubuntu container and Mattermost install, I try to bulk import data from Slack, but the bulk import command is targeting a nonexistent “mattermost_test” database, and when I create the database I get an error about a missing team.

Steps to reproduce

My setup:
Proxmox VE 8.1.3 (latest)
Ubuntu 20.04.6 LTS container template bundled with Proxmox (ubuntu-20.04-standard_20.04-1_amd64.tar.gz)
Mattermost Version: 9.2.3
Database Schema Version: 113
PostgreSQL 13.13 (Ubuntu 13.13-1.pgdg20.04+1)

Steps from container initialisation:

apt update && apt -y upgrade && apt -y install curl gnupg2 software-properties-common

(I create a local sudo user)

curl -o- https://deb.packages.mattermost.com/repo-setup.sh | sudo bash
sudo apt install mattermost-omnibus -y

(I copy .jsonl and data folder to /opt/mattermost/)

mattermost import bulk ./cihcb_mattermost_import.jsonl --import-path ./data --apply

This bulk import (1st run) fails:

pq: password authentication failed for user “mmuser”

I adjust the PostgreSQL conf:

nano /etc/postgresql/13/main/pg_hba.conf
# TYPE  DATABASE        USER            ADDRESS                 METHOD

# "local" is for Unix domain socket connections only
local   all             all                                     trust
# IPv4 local connections:
host    all             all             127.0.0.1/32            trust
# IPv6 local connections:
host    all             all             ::1/128                 trust

I then run the bulk import command again:

mattermost import bulk ./cihcb_mattermost_import.jsonl --import-path ./data --apply

Expected behavior
I expect the import to complete successfully, and for the imported data from Slack to appear in the Mattermost interface.

Observed behavior
1st run:

pq: password authentication failed for user “mmuser”

2nd run: (after modifying pg_hba.conf)

pq: database “mattermost_test” does not exist

What I don’t understand here is why the bulk importer is trying to target a database called “mattermost_test”. I don’t see it defined as a prerequisite in the documentation, and it is not created during installation, yet it seems to be required for the bulk importer to function.

3rd run: (after creating “mattermost_test” database in postgresql)

postgres=# CREATE DATABASE mattermost_test;

Error importing channel. Team with name “CihcB” could not be found., resource: Team id: name=CihcB

I think I have followed the steps in the documentation (after many attempts). I do have a team called “CihcB” in the new mattermost instance, however since the bulk import tool is targeting “mattermost_test”, which is just an empty database and not the production database, there are no teams.
I’m aware that the documentation says you should run the bulk import command as the mattermost user, however when I try “su mattermost” I am unable to change user. I can switch to any other user, but not mattermost. I tried resetting the password for the mattermost user and logging in to the terminal as mattermost, but it doesn’t work. Although I don’t think running the bulk import command as root is causing me issues so far.
I have attempted using the mmctl command to bulk upload, but it refused to upload the .zip saying it was too big (369MB).
I have attempted following the documentation to create the mattermost database before installing Mattermost, which setup PostgreSQL 16; but Mattermost Omnibus includes PostgreSQL 13 as part of the installation regardless of whether there is already a database running.
I am aware that there is a turnkey Linux container template for Mattermost in Proxmox, although when I tried it I ran into various undocumented problems.
I’m sorry for the verbosity of my post(!)

I have stumbled upon this going through the import process so let me tell you what I did to complete this:

Filesize issue
I have changed FileSettingsMaxFileSize to be able to import larger file. The value seems to be in bytes, see File storage system - Mattermost documentation

Import team name issue

  • when you setup a MM instance, go to the web interface and create the account and team.
  • then, on your server, login to the mmctl with
./mmctl auth login https://example.com --name my-mm-server --username=clayton

from there

./mmctl team list --json

you’ll notice the team name you need to use for the

./mmetl transform slack --team <TEAM-NAME>

command. I was also surprised when my name didn’t work, but it turned I was using Display Name and not the name the transformer command needed.

watch out for one thing: if your admin username AND imported user had same data (login/email) they will be overwritten during import and the

./mmctl import job show <JOB_ID> --json

will end up with:

Error: failed to get import job: You do not have the appropriate permissions.

Simply wait for a few minutes and try to login to your web interface/reset password from there. that’s a indicator the importer worked when it comes to users at least :slight_smile: