The 504 gateway timeout means that nginx can’t reach the Mattermost server.
Can you check that the Mattermost server is running and responding to requests by using curl http://localhost:8065 on the box running Mattermost?
If that doesn’t help you solve the problem, please post your nginx config and Mattermost logs.