Do I have to fix/migrate my database manually?

As I mentioned in an earlier message, I have a number of missing file uploads after an upgrade 3.4 → 3.9. They are on disk but are not rendered in the channels.

Was file handling changed in this interval? Should I expect my previous messages to be intact?

Should every file have an entry in the fileinfo table? It looks like many of the files that existed prior to the upgrade do not have such entries.

Will I have to script a manual reconstruction, if that’s even possible?

Hi @nedaura,

Thanks for your questions,

Here is the upgrade guide for v3.9.

Can you let us know whether you still have issues after following the steps in the guide?

I had this document on hand, and to the best of my knowledge followed it, when upgrading.

What I see is this:

  • new posts are handled correctly.

  • in my database backup prior to upgrading, posts with attached files do contain the filenames.

  • after the upgrade, some of these posts records do not have anything in either the filenames or fileids.

  • there is no fileinfo record for these posts either.

According to the code, fileinfo records are created lazily on access instead of part of the migration. I’m not sure what the issue is - some of my file references are updated, and some are not. I don’t know why the filenames field would be set to ‘[]’ for some posts when no fileinfo record had been (lazily?) created for it.

My issue aside, I question whether it’s wise to use this lazy migration technique instead of biting the bullet and decisively migrating them at the time of system upgrade.

Hi @nedaura,

Just so I understand you correctly, is your issue that files are missing from posts that have been migrated already, or are you just looking for a way to migrate them all at once?

Regarding why we do the migration lazily, it’s to avoid downtime while it takes place. At the time that we were making that change, I tested on a copy of our nightly build server, and it was 30 minutes to an hour to complete. We’ve had a customer’s system containing a lot more data take 20 minutes just to add a column to the Posts table, so this taking over half an hour on a smaller data set wasn’t an option. We have some plans in the future to do larger migrations just in the background instead of just on demand like that one was done, but it’s still pretty far out

A number of existing posts had their (old) filenames removed, but did not have new file info records created for the files.

So I’m wondering if the lazy-migration process experienced some kind of error creating file info records but went ahead and deleted the file names anyway. Maybe the api calls weren’t working yet? I don’t recall.

I resolved this by repopulating the filenames data from my backup - thereafter, the lazy migration did work.

Glad to hear that you were able to fix it with a backup. I don’t believe we had anything like this reported back in 3.5 when those changes were made, so it could be possible that it’s caused by the larger jump in version.

If it happens again, would you be able to check the system logs to see if any errors are printed? That would give us a better idea of what’s going wrong in case someone else runs into this problem