Mmctl `Unhandled panic in job` when importing

Troubleshooting Command Line Interface
1

I just set up a new Mattermost server using the omnibus installation method.

Following along with Migrate from Slack, I’ve managed to export the slack data.

Importing mattermost-bulk-import.zip using sudo mmctl --local import process --bypass-upload /home/ubuntu/mattermost-bulk-import.zip results in the server crashing, along with some errors:

  ID: zb7t4fbpnjnriyw73e6mm4nr3e
  Type: import_process
  Status: error
  Created: 2025-03-07 15:36:01 +0000 UTC
  Started: 2025-03-07 15:36:11 +0000 UTC
  Data: map[error:Unable to update the job. extract_content:true import_file:/home/ubuntu/mattermost-bulk-import.zip local_mode:true]

I was wondering if the file was too large (600+MB), so I used mmimportjs to break the file into small chunks (I modified the code to do 500 line chunks for diagnostics.

Running sudo mmctl --local import process --bypass-upload 000001-mattermost_import.jsonl results in Import process job successfully created, ID: wpoxomezkt84xjhn9s1aekxyoc

Using sudo mmctl --local import job show wpoxomezkt84xjhn9s1aekxyoc shows:

  ID: wpoxomezkt84xjhn9s1aekxyoc
  Type: import_process
  Status: error
  Created: 2025-03-07 15:49:03 +0000 UTC
  Started: 2025-03-07 15:49:12 +0000 UTC
  Data: map[error:Unable to update the job. extract_content:true import_file:/home/ubuntu/2025-03-07T15:43:36/000001-mattermost_import.jsonl local_mode:true]

Looking into the server logs, I see that that job fails with the log event:

{
  "caller": "jobs/jobs.go:230",
  "job": {
    "create_at": 1741362543914,
    "data": {
      "error": "Unable to update the job.",
      "extract_content": "true",
      "import_file": "/home/ubuntu/2025-03-07T15:43:36/000001-mattermost_import.jsonl",
      "local_mode": "true"
    },
    "id": "wpoxomezkt84xjhn9s1aekxyoc",
    "last_activity_at": 1741362552878,
    "priority": 0,
    "progress": 0,
    "start_at": 1741362552878,
    "status": "in_progress",
    "type": "import_process"
  },
  "job_create_at": "Mar  7 15:49:03.914",
  "job_id": "wpoxomezkt84xjhn9s1aekxyoc",
  "job_type": "import_process",
  "level": "error",
  "msg": "Unhandled panic in job",
  "panic": "runtime error: invalid memory address or nil pointer dereference",
  "stack": "goroutine 239 [running]:\nruntime/pprof.writeGoroutineStacks({0x5579b00, 0xc0020803a0})\n\truntime/pprof/pprof.go:743 +0x6a\nruntime/pprof.writeGoroutine({0x5579b00?, 0xc0020803a0?}, 0x0?)\n\truntime/pprof/pprof.go:732 +0x25\nruntime/pprof.(*Profile).WriteTo(0x48dd3e2?, {0x5579b00?, 0xc0020803a0?}, 0x0?)\n\truntime/pprof/pprof.go:369 +0x14b\ngithub.com/mattermost/mattermost/server/v8/channels/jobs.(*JobServer).HandleJobPanic(0xc001a4e360, {0x55ac8c8, 0xc001dd0c30}, 0xc0035025a0)\n\tgithub.com/mattermost/mattermost/server/v8/channels/jobs/jobs.go:229 +0xae\npanic({0x42d6100?, 0x7b154a0?})\n\truntime/panic.go:770 +0x132\ngithub.com/mattermost/mattermost/server/v8/channels/jobs/import_process.MakeWorker.func2({0x55ac8c8?, 0xc001dd0c30?}, 0xc0035025a0)\n\tgithub.com/mattermost/mattermost/server/v8/channels/jobs/import_process/worker.go:61 +0x5ed\ngithub.com/mattermost/mattermost/server/v8/channels/jobs.(*SimpleWorker).DoJob(0xc001e28190, 0xc0054dd080)\n\tgithub.com/mattermost/mattermost/server/v8/channels/jobs/base_workers.go:94 +0x3c6\ngithub.com/mattermost/mattermost/server/v8/channels/jobs.(*SimpleWorker).Run(0xc001e28190)\n\tgithub.com/mattermost/mattermost/server/v8/channels/jobs/base_workers.go:53 +0xc5\ncreated by github.com/mattermost/mattermost/server/v8/channels/jobs.(*Workers).Start in goroutine 1\n\tgithub.com/mattermost/mattermost/server/v8/channels/jobs/workers.go:52 +0x12b\n\ngoroutine 1 [chan receive, 12 minutes]:\ngithub.com/mattermost/mattermost/server/v8/cmd/mattermost/commands.runServer(0xc00124a230, 0xc00003fb00)\n\tgithub.com/mattermost/mattermost/server/v8/cmd/mattermost/commands/server.go:115 +0x225\ngithub.com/mattermost/mattermost/server/v8/cmd/mattermost/commands.serverCmdF(0x7bad940, {0x48ca4f2?, 0x4?, 0x48ca4c6?})\n\tgithub.com/mattermost/mattermost/server/v8/cmd/mattermost/commands/server.go:57 +0x10a\ngithub.com/spf13/cobra.(*Command).execute(0x7bad940, {0xc000102060, 0x0, 0x0})\n\tgithub.com/spf13/cobra@v1.8.1/command.go:985 +0xaca\ngithub.com/spf13/cobra.(*Command).ExecuteC(0x7bad940)\n\tgithub.com/spf13/cobra@v1.8.1/command...",
  "timestamp": "2025-03-07 15:49:12.920 Z",
  "worker_name": "ImportProcess"
}

Is there any additional troubleshooting that I should do, or any workaround?

1 Like
2

Hi @DeathCamel58! Thanks for sharing the detailed logs and steps you’ve taken so far. It looks like the issue might be related to the size or contents of the import file. I recommend verifying the formatting of the JSON data using the schema outlined in our Bulk Import and Export documentation. Additionally, checking for any invalid or unsupported entries in the import file could help. Let us know if you need further assistance! :blush: