[Solved] Heap exhaustion during MySQL -> Postgres migration

Troubleshooting
1

Summary
During migration from MySQL (MariaDB) → Postgres using the migration-assist tool, we are seeing heap exhaustion, even while providing a lot of memory to the worker processes.

Observed behavior

2026-07-08T05:18:53.855728Z LOG Skipping public.ir_system
2026-07-08T05:18:53.855728Z LOG Skipping public.ir_timelineevent
2026-07-08T05:18:53.855728Z LOG Skipping public.ir_userinfo
2026-07-08T05:18:53.855728Z LOG Skipping public.ir_viewedchannel
Heap exhausted during garbage collection: 400 bytes available, 21088 requested.
| Immobile Objects |
Gen layout symbol   code  Boxed   Cons    Raw   Code  SmMix  Mixed  LgRaw LgCode  LgMix Waste%       Alloc        Trig   Dirty GCs Mem-age
1      0      0      0    620      3  10863      0      1     27      0      0      0    3.7   363309760   222218346   11514   1  1.3710
2      0      0      0    662      1  10893      0      1     11      0      0      0    3.4   366099296    10737418   11548   0  0.3793
3     56      3     79    361     56   4806      1      5    138      0      0   1800    4.3   224697984     2000000    5042   0  0.4556
4      0      0      0      0      0      0      0      0      0      0      0      0    0.0           0     2000000       0   0  0.0000
5      0      0      0      0      0      0      0      0      0      0      0      0    0.0           0     2000000       0   0  0.0000
6   2184  27442  35446    556    780    665     20     79     68    100      0    251    2.4    80593200     2000000      28   0  0.0000
Tot   2240  27445  35525   2199    840  27227     21     86    244    100      0   2051    3.6  1034699760 [96.4% of 1073741824 max]
GC control variables:
GC-INHIBIT = true
GC-PENDING = true
STOP-FOR-GC-PENDING = false
Collection trigger variables:
dynamic_space_size = 1073741824
bytes_allocated = 1034699760
auto_gc_trigger = 822651187
bytes_consed_between_gcs = 53687091
fatal error encountered in SBCL pid 660423 tid 660435:
Heap exhausted, game over.

Welcome to LDB, a low-level debugger for the Lisp runtime environment.
(GC in progress, oldspace=1, newspace=2)

The server has 64GB of physical memory and 12 cores dedicated to the process (nothing else on the host). During migration, it seems to have around 58-60GB free. We have allocated 16 workers to the job, and simple maths seem to indicate (could be wrong, please correct) that we should still then have enough free.

Here is our migration.load file:

LOAD DATABASE
FROM mysql://mmuser:<redacted>@localhost:3306/mattermost
INTO postgresql://mmuser:<redacted>@localhost:5432/mattermost
WITH include no drop,
create no tables,
preserve index names,
batch rows = 1000,
workers = 16,
concurrency = 8
SET work_mem to '1024MB',
maintenance_work_mem to '2048 MB'
ALTER SCHEMA 'mattermost' RENAME TO 'public'
CAST type datetime to timestamptz
drop default drop not null using zero-dates-to-null,
type date drop not null using zero-dates-to-null,
type decimal when (= precision 20) to numeric,
type longtext to text,
type mediumtext to text,
type tinyint when (= precision 1) to boolean
;

Our db is enormous, going back years. Should we be looking to `renice` the postgres parent process, or some other intervention on resource allocation?

Any help much appreciated. Looking forward to getting this done.

3

With 2048 work_men and 4096 maintenance_work_mem:

Heap exhausted during garbage collection: 16 bytes available, 32 requested.
| Immobile Objects |
Gen layout symbol   code  Boxed   Cons    Raw   Code  SmMix  Mixed  LgRaw LgCode  LgMix Waste%       Alloc        Trig   Dirty GCs Mem-age
1      0      0      0    500      3  11780      0      0     35      0      0      0    4.2   386571248   242104138   12318   1  1.3988
2      0      0      0    930      3  14340      0      1     45      0      0      0    3.1   486379504    10737418   15179   0  0.8102
3     56      3     79     94     54    535      1      4    124      0      0   1800    5.4    80951248     2000000     594   0  0.0000
4      0      0      0      0      0      0      0      0      0      0      0      0    0.0           0     2000000       0   0  0.0000
5      0      0      0      0      0      0      0      0      0      0      0      0    0.0           0     2000000       0   0  0.0000
6   2184  27442  35446    556    780    665     20     79     68    100      0    251    2.4    80593280     2000000      28   0  0.0000
Tot   2240  27445  35525   2080    840  27320     21     84    272    100      0   2051    3.7  1034494720 [96.3% of 1073741824 max]
GC control variables:
GC-INHIBIT = true
GC-PENDING = true
STOP-FOR-GC-PENDING = false
Collection trigger variables:
dynamic_space_size = 1073741824
bytes_allocated = 1034494720
auto_gc_trigger = 831676003
bytes_consed_between_gcs = 53687091
fatal error encountered in SBCL pid 660756 tid 660775:
Heap exhausted, game over.

Welcome to LDB, a low-level debugger for the Lisp runtime environment.
(GC in progress, oldspace=1, newspace=2)
4

Seems to be a known bug in pgloader supposedly fixed in v4 as of 2 weeks ago.

I’m going to try v4 and see how it goes.

5

No joy with pgloader version 4, which is a completely new implementation in Java, ugh. it doesn’t seem to see anything under the Public scope, for some reason. Did some more reading and have gone back to version 3, but with prefetch rows = 1000in the WITH block of the load file.

6

The prefetch above got past the heap issue. I’ve hit another, but will save that for another thread.