I found this on the Atlassian support site: https://confluence.atlassian.com/jirakb/how-to-collect-data-for-atlassian-support-for-troubleshooting-jira-webhooks-397083035.html
I went through the steps, and sending to the webhook.site works fine. Sending to Mattermost results in the same error.
2018-06-01 10:01:53,893 httpclient-callbacks:thread-21 WARN anonymous [c.a.webhooks.plugin.PublishTaskFactoryImpl$PublishTaskImpl] Server error - 502 when posting to web hook at 'https://ourdomain.com/plugins/jira/webhook?secret=bigsecret&team=test&channel=notifications'
Mattermost has been upgraded to the latest version.