Constant websocket errors

Hi @Podesta - were your log levels set to Info? We made a change in 6.6 (Use error log level for websocket timeouts (#19609) · mattermost/mattermost-server@9534efe · GitHub) which logs timeout errors in the error log. But they were always logged in the debug level. So nothing really changed except the log level.

This log line would appear whenever the server fails to read any message within a given timeout, and it disconnects the websocket connection. Ideally, there is a constant stream of ping/pong messages happening which prevents this from happening. And there is no reason for the ping/pong to stop. So this is indeed not expected to happen, and hence the error level.

I am wondering whether you are using AWS CLB/ALB in front? We have seen issues with these categories of load-balancers in that the idle timeout settings for AWS and the Mattermost server needs to be appropriately set, otherwise the load-balancer will keep disconnecting the connection: https://sigopt.com/blog/the-case-of-the-mysterious-aws-elb-504-errors/.

When this happens, the server timeout gets triggered because there’s nothing sending any data to it, and it disconnects the connection from its side as well, logging this error.

I’d suggest to check the load balancer settings and verify everything is in order.