As far as I am aware, you must have some form of a server running in order to set headers in the method that you are talking about.
For example, with Apache, which I use with Mattermost, you can set specific headers from within the virtual host configuration file, and/or with a .htaccess file. However, without a web server, there is no engine to process the information in the .htaccess file, so it would virtually render it a nice text file otherwise.
I’m honestly a bit interested to hear a bit further about the issues you experienced with previous reverse proxy configurations and what may have been causing those issues, because that as well sounds like an issue with headers overriding those set by Mattermost, causing users to be logged out.