After upgrade to 5.37 seeing "http: panic serving" errors

For feature requests, please see: Contributing Feature Proposals - Mattermost.

For troubleshooting questions, please post in the following format:

Upgraded from 5.33.2 to 5.37. After the upgrade, I started seeing multiple error http/server.go:3093 http: panic serving runtime error: invalid memory address or nil pointer dereference errors in the mattermost.log

Steps to reproduce
Open the System Console

Expected behavior
no errors logged

Observed behavior
I upgraded from 5.33.2 to 5.37. When I upgraded, I went from 5.33.2 → 5.33.5 - > 5.34.5 → 5.35.4 → 5.36.1 → 5.37, each upgrade one at a time.

Currently running the server on ubuntu 16.04.7, mysql 5.7.33.

During the upgrades, I didn’t notice any failures or errors.

After the upgrade, every time I go into the system console, I see the http: panic crash errors like this:

2021-07-17T13:59:06.870-0500	error	http/server.go:3093	http: panic serving runtime error: invalid memory address or nil pointer dereference
goroutine 9663 [running]:
	net/http/server.go:1801 +0x147
panic(0x2521aa0, 0x3f2d9d0)
	runtime/panic.go:975 +0x47a*Handler).recoverWithSentry(0xc000cdc9e0, 0xc004d38b10, 0xc0054af300) +0x14f
panic(0x2521aa0, 0x3f2d9d0)
	runtime/panic.go:969 +0x1b9, 0x2cf16e0, 0xc004204140, 0xc0054af500) +0x63, 0xc001dda740, 0x29d65a8, 0x3ceacd8, 0x13, 0x100000001, 0x0, 0x0, 0x2cf4060, 0xc0040b7960, ...) +0x1db2, 0xc0040b7960, 0xc0054af500) +0x2f0
net/http.HandlerFunc.ServeHTTP(0xc0013f1d40, 0x2cf4060, 0xc0040b7960, 0xc0054af500)
	net/http/server.go:2042 +0x44*Router).ServeHTTP(0xc00024a300, 0x2cf4060, 0xc0040b7960, 0xc0054af300) +0xd3*Handler).handle.func1(0x2cf4060, 0xc0040b7960, 0xc0054af200) +0x402
net/http.HandlerFunc.ServeHTTP(0xc002d6e5c0, 0x2cf4060, 0xc0040b7960, 0xc0054af200)
	net/http/server.go:2042 +0x44*RateLimiter).RateLimitHandler.func1(0x2cf4060, 0xc0040b7960, 0xc0054af200) +0xc8
net/http.HandlerFunc.ServeHTTP(0xc002d6e600, 0x2cf4060, 0xc0040b7960, 0xc0054af200)
	net/http/server.go:2042 +0x44
net/http.serverHandler.ServeHTTP(0xc005a4a0e0, 0x2cf4060, 0xc0040b7960, 0xc0054af200)
	net/http/server.go:2843 +0xa3
net/http.(*conn).serve(0xc0050b3900, 0x2cfdf20, 0xc002b00840)
	net/http/server.go:1925 +0x8ad
created by net/http.(*Server).Serve
	net/http/server.go:2969 +0x36c	{"source": "httpserver"}

Is there a way to figure out what is causing these? Could it be an incompatibility with the server running ubuntu 16.04?

Best regards,

This is listed as a known in the changelog. The logs are polluted from the errors you posted, but there are are no functional effects and Mattermost should work properly. A dev is looking into a fix via [MM-37108] Fix possible nil dereference in team edition by streamer45 · Pull Request #17933 · mattermost/mattermost-server · GitHub & [MM-37108] Only make request if server is enterprise ready by streamer45 · Pull Request #8406 · mattermost/mattermost-webapp · GitHub PRs.

1 Like

Hi Amy,

Doh! Somehow I had completely missed that in the change log. Thanks!

Best regards,