After server rebuild, older files in s3 bucket not available

Troubleshooting
1

Hi -

Due to losing a disk array we lost our Mattermost server. We did have a copy of the database backup and restored it to an ec2 instance and have recovered most of our message history/user accounts/etc - however files that were uploaded to the s3 bucket that we were using for filestorage don’t seem to be appearing in the mattermost client. Also the directory structure seems to show date folders from a long time ago and then folders for each day since teh server was rebuilt. This leads me to ask:

  1. What is the expected structure of data within the s3 bucket?
  2. Why am I seeing a ‘date’ folder for most days since I rebuilt the server but none before (not counting the one from 2017)
  3. Is there a way to merge the data from the older folders into whatever we are currently using or does the lack of earlier date folders mean that the s3 bucket was not configured correctly (or the inverse - do the date folders show that I did something wrong)?

I inherited this server so I don’t have as much details on the initial build as I would like so any info/advice is appreciated.

$ aws s3 ls s3://mattermost.company.com
                           PRE 20170920/
                           PRE 20190830/
                           PRE 20190903/
                           PRE 20190904/
                           PRE 20190906/
                           PRE 20190909/
                           PRE 20190910/
                           PRE 20190911/
                           PRE 20190912/
                           PRE 20190913/
                           PRE teams/
                           PRE users/
2

Hi @1tacocat1, can you check if this thread might help: URGENT: Old directly uploaded images missing after setting server to use S3?

3

I checked that one. I don’t think it helps because we already were supposed to be on S3 before the server crash. Based on my review it appeared that the solution there was to copy the data from the old server to the bucket. In this case we should have already had all the data on s3 before the crash so, in theory, it should already be there. That being said, I’m not 100% sure that the server was properly configured to store the data on s3 so what I guess I’m asking is how to verify that the data was being actually being stored on s3?

Thanks!

4

Nevermind. Just looked at my old config.json in depth. Despite the s3 keys being configured, the DriverName was set to local. That data is gone.

@amy.blais - thanks for offering the suggestion. I need to go shake my head disapprovingly at some people now… :wink: