Some errors during migration from MySQL 8 to Postgres Database

Troubleshooting
1

Followed the instructions on Automated PostgreSQL migration - Mattermost documentation
Ubuntu 24.04 LTS with it’s defaults mostly, except migration-assist and pgloader (which needed to be rebuilt from source, because Ubuntu repo one is broken.):
mattermost 10.9.1-0 (repo install version)
MySQL Ver 8.0.42-0ubuntu0.24.04.1
psql (PostgreSQL) 16.9 (Ubuntu 16.9-0ubuntu0.24.04.1)
pgloader version “3.6.9” compiled with SBCL 2.2.9.debian ← built from source.

Is there something I should be worried about or I can ignore those errors?

2025-07-09T16:57:10.858053+03:00 ERROR pgloader failed to find target table for source "mattermost"."calls" with name "calls" in target catalog
2025-07-09T16:57:10.858053+03:00 ERROR pgloader failed to find target table for source "mattermost"."calls_channels" with name "calls_channels" in target catalog
2025-07-09T16:57:10.858053+03:00 ERROR pgloader failed to find target table for source "mattermost"."calls_jobs" with name "calls_jobs" in target catalog
2025-07-09T16:57:10.860053+03:00 ERROR pgloader failed to find target table for source "mattermost"."calls_sessions" with name "calls_sessions" in target catalog
2025-07-09T16:57:10.860053+03:00 ERROR pgloader failed to find target table for source "mattermost"."db_migrations_calls" with name "db_migrations_calls" in target catalog
2025-07-09T16:57:10.860053+03:00 LOG Skipping mattermost.calls
2025-07-09T16:57:10.860053+03:00 LOG Skipping mattermost.calls_channels
2025-07-09T16:57:10.860053+03:00 LOG Skipping mattermost.calls_jobs
2025-07-09T16:57:10.860053+03:00 LOG Skipping mattermost.calls_sessions
2025-07-09T16:57:10.860053+03:00 LOG Skipping mattermost.db_migrations_calls

otherwise it seemed to go ok?

2025-07-09T16:57:15.955146+03:00 LOG report summary reset
                           table name     errors       rows      bytes      total time
-------------------------------------  ---------  ---------  ---------  --------------
                          before load          0          4                     0.049s
                      fetch meta data          0         78                     0.201s
                    Drop Foreign Keys          0          4                     0.004s
-------------------------------------  ---------  ---------  ---------  --------------
                      mattermost.jobs          0     303365    49.4 MB          7.118s
                     mattermost.posts          0      17416     4.1 MB          1.973s
            mattermost.channelmembers          0        363    78.1 kB          2.121s
                    mattermost.audits          0        826   103.1 kB          0.981s
                 mattermost.reactions          0        476    55.1 kB          0.310s
           mattermost.sidebarchannels          0        263    29.0 kB          2.270s
         mattermost.threadmemberships          0        276    23.2 kB          0.612s
               mattermost.preferences          0        279    25.9 kB          1.280s
      mattermost.channelmemberhistory          0        108     7.5 kB          2.392s
                     mattermost.users          0         27    15.2 kB          2.623s
                  mattermost.channels          0        108    20.0 kB          0.877s
                   mattermost.threads          0        162    17.6 kB          1.658s
         mattermost.sidebarcategories          0         39     5.6 kB          1.941s
               mattermost.teammembers          0         21     1.6 kB          2.822s
    mattermost.productnoticeviewstate          0         31     1.8 kB          0.837s
                     mattermost.roles          0         23    22.6 kB          2.237s
            mattermost.recentsearches          0         10     1.5 kB          2.405s
                    mattermost.drafts          0          7     0.9 kB          2.961s
       mattermost.pluginkeyvaluestore          0         20     2.0 kB          1.221s
                      mattermost.bots          0          7     0.8 kB          1.176s
     mattermost.accesscontrolpolicies          0          0                     3.054s
          mattermost.incomingwebhooks          0          2     0.4 kB          2.342s
                     mattermost.teams          0          3     0.4 kB          1.549s
           mattermost.commandwebhooks          0          0                     3.052s
          mattermost.channelbookmarks          0          0                     2.423s
          mattermost.clusterdiscovery          0          0                     1.475s
                     mattermost.emoji          0          0                     3.176s
             mattermost.desktoptokens          0          0                     1.646s
                  mattermost.licenses          0          0                     3.104s
               mattermost.compliances          0          0                     2.509s
             mattermost.oauthauthdata          0          0                     3.216s
                mattermost.groupteams          0          0                     1.740s
              mattermost.groupmembers          0          0                     2.481s
      mattermost.postacknowledgements          0          0                     3.132s
           mattermost.oauthaccessdata          0          0                     2.620s
                 mattermost.oauthapps          0          0                     1.778s
            mattermost.propertygroups          0          0                     3.172s
          mattermost.outgoingwebhooks          0          0                     2.635s
   mattermost.persistentnotifications          0          0                     1.884s
         mattermost.retentionpolicies          0          0                     3.142s
             mattermost.postspriority          0          1     0.1 kB          2.501s
            mattermost.propertyfields          0          0                     1.857s
   mattermost.retentionidsfordeletion          0          0                     1.990s
            mattermost.scheduledposts          0          0                     1.908s
            mattermost.remoteclusters          0          0                     2.605s
                   mattermost.schemes          0          0                     3.139s
        mattermost.sharedchannelusers          0          0                     2.046s
    mattermost.retentionpoliciesteams          0          0                     2.636s
            mattermost.sharedchannels          0          0                     3.125s
      mattermost.sharedchannelremotes          0          0                     2.649s
            mattermost.uploadsessions          0          0                     1.983s
                    mattermost.tokens          0          0                     2.708s
          mattermost.useraccesstokens          0          0                     3.098s
        mattermost.usertermsofservice          0          0                     2.738s
              mattermost.linkmetadata          0        308    82.0 kB          0.111s
                  mattermost.fileinfo          0        294   521.7 kB          0.651s
                   mattermost.systems          0         67     2.8 kB          0.831s
                  mattermost.sessions          0         26     4.2 kB          0.636s
            mattermost.publicchannels          0         16     1.5 kB          0.779s
                    mattermost.status          0          7     0.4 kB          0.875s
mattermost.accesscontrolpolicyhistory          0          0                     0.938s
                  mattermost.commands          0          0                     1.036s
             mattermost.groupchannels          0          0                     1.087s
               mattermost.notifyadmin          0          0                     1.191s
  mattermost.outgoingoauthconnections          0          0                     1.266s
             mattermost.postreminders          0          0                     1.279s
            mattermost.propertyvalues          0          0                     1.462s
 mattermost.retentionpolicieschannels          0          0                     1.377s
  mattermost.sharedchannelattachments          0          0                     1.482s
            mattermost.termsofservice          0          0                     1.535s
                mattermost.usergroups          0          0                     1.604s
-------------------------------------  ---------  ---------  ---------  --------------
              COPY Threads Completion          0          8                     7.095s
                      Reset Sequences          0          0                     0.041s
                  Create Foreign Keys          0          2                     0.005s
                      Set Search Path          0          1                     0.001s
                     Install Comments          0          0                     0.000s
                           after load          0          4                     0.031s
-------------------------------------  ---------  ---------  ---------  --------------
                    Total import time          ✓     324551    54.5 MB          7.173s

so is it OK? :smile:

also Step 5 fails with:

./migration-assist postgres post-migrate --create-indexes 'postgres://mmuser:<SNIP>@localhost:5432/mattermost'
An Error Occurred: unknown flag: --create-indexes
4

also is it just me or the current migration-assist tool generates totally WRONG calls plugin migration calls.load file? from the migration guide:

./migration-assist pgloader calls \
--mysql="mmuser:SNIP@tcp(127.0.0.1:3306)/mattermost" \
--postgres="postgres://mmuser:SNIP@localhost:5432/mattermost" \
> calls.load
  1. it never creates tables for calls plugin. IT SHOULD! (when comparing to boards.load and playbooks.load)
  2. it destroys the posts and fileinfo indexes for some reason??? (boards.load and playbooks.load don’t)
LOAD DATABASE
    FROM       mysql://mmuser:SNIP@127.0.0.1:3306/mattermost
    INTO       pgsql://mmuser:SNIP@localhost:5432/mattermost

WITH data only,
    workers = 8, concurrency = 1,
    multiple readers per thread, rows per range = 10000,
    prefetch rows = 10000, batch rows = 2500,
    create no tables, create no indexes,
    preserve index names

SET PostgreSQL PARAMETERS
    maintenance_work_mem to '128MB',
    work_mem to '12MB'

SET MySQL PARAMETERS
    net_read_timeout  = '120',
    net_write_timeout = '120'

 CAST column Channels.Type to "channel_type" drop typemod,
    column Teams.Type to "team_type" drop typemod,
    column UploadSessions.Type to "upload_session_type" drop typemod,
    column ChannelBookmarks.Type to "channel_bookmark_type" drop typemod,
    column Drafts.Priority to text,
    type int when (= precision 11) to integer drop typemod,
    type bigint when (= precision 20) to bigint drop typemod,
    type text to varchar drop typemod using remove-null-characters,
    type tinyint when (<= precision 4) to boolean using tinyint-to-boolean,
    type json to jsonb drop typemod using remove-null-characters

EXCLUDING TABLE NAMES MATCHING ~<IR_>, ~<focalboard>, 'schema_migrations', 'db_migrations', 'db_lock',
    'configurations', 'configurationfiles', 'db_config_migrations'

BEFORE LOAD DO
    $$ ALTER SCHEMA public RENAME TO mattermost; $$,
    $$ TRUNCATE TABLE mattermost.systems; $$,
    $$ DROP INDEX IF EXISTS mattermost.idx_posts_message_txt; $$,
    $$ DROP INDEX IF EXISTS mattermost.idx_fileinfo_content_txt; $$

AFTER LOAD DO
    $$ UPDATE mattermost.db_migrations set name='add_createat_to_teamembers' where version=92; $$,
    $$ ALTER SCHEMA mattermost RENAME TO public; $$,
    $$ SELECT pg_catalog.set_config('search_path', '"$user", public', false); $$,
    $$ ALTER USER mmuser SET SEARCH_PATH TO 'public'; $$;

in fact… now to compare it to … it is identical to the default migration.load and nothing to do with calls plugin:

$ md5sum calls.load migration.load 
746f03dfbcc6c1d610efae401f1d01bb  calls.load
746f03dfbcc6c1d610efae401f1d01bb  migration.load
5

But found the good enough calls.load from Manual migration page.

Anyway. Migration success now. :partying_face:

Small question remains – is there some sort of method to test the structure of the migrated database? Like that it’s what you get also when you install directly for postgres? all the tables present and all the fields have correct types and stuff (I mean it gave a lot of warnings about the type differences when migrating. a la “varchar is not text” etc.)?