[Solved] Outgoing webhook failing silently on internal site, even after allowing untrusted internal connections

Troubleshooting
1

Summary
I’m trying to connect a internally-hosted app with mattermost using an outgoing webhook. However the webhook fails silently even after ensuring connectivity between hosts, applying correct AllowUntrustedInternalConnections settings and ensuring certs are trusted.

Steps to reproduce
I’m running mattermost-team-edition:9.11 in docker behind Caddy at mm.domain.com.
I am running n8n in docker behind Caddy at n8n.domain.com.
I’m using dns challenge to get a valid letsencrypt cert for my subdomains.
Local dns is resolving n8n.domain.com and mm.domain.com.
n8n.domain.com is added as untrusted allowed connections (both in env variable and in developer system settings).

  • To be extra sure, I’ve also added 0.0.0.0/0 as an allowed connection and I get the same behavior.
    I have an outgoing webhook in mattermost pointed at n8n.domain.com
    I’ve confirmed my mm installation can hit n8n.domain.com
  • if I use an invalid or self-signed cert with n8n I get a error=Post "https://n8n.domain.com/webhook-test/<webhook>": tls: failed to verify certificate: x509: certificate signed by unknown authority error from mm logs; showing that mm was able to hit my n8n instance.
    I’ve confirmed my mm installation is correctly resolving n8n.domain.com
  • if I don’t add my host or local IP to AllowedUntrustedInternalConnections, I get an error=Post "https://n8n.domain.com/webhook-test/<webhook>": address forbidden, you may need to set AllowedUntrustedInternalConnections to allow an integration access to your internal network

Expected behavior
I would expect to see a message in mm logs showing a successful POST to the webhook; I’ve been able to get this when I expose my n8n application directly (e.g. not behind caddy with a hostname) and hit the webhook by using the IP address).

Observed behavior
I get no logs in MM related to the webhook.
I’ve confirmed the webhook is up and running by successfully triggering it by hitting it via curl from the server where mattermost is running.

Has anyone else experienced this? How can I get mattermost to successfully use an outgoing webhook against an internally-hosted domain?

2

PEBKAC issue; I wasn’t paying attention to the verb. n8n.domain.com was listening for a GET not a POST. Once I edited my webhook to listen for a POST everything worked smoothly (curl defaults to a GET, which is why the webhook worked just fine when I hit it with curl).
Also I misremembered and was wrong: mattermost doesn’t log a message on successful outgoing webhooks.

3

Woot! Thanks for following up with the solution, user745! Glad to hear you were able to get it working.