WebSocket errors with Mattermost Kubernetes 5.3.1

Troubleshooting
1

We have a community member running Mattermost 5.3.1 in Kubernetes and receiving “Please check connection, Mattermost unreachable. If issue persists, ask administrator to check WebSocket port.” errors.

Chrome Developer Console shows websocket calls all have status „Finished“.

There‘s no proxy involved, but k8s ingress and service. config.json unchanged, regarding all WebSocket related entries (WebsocketURL „“, WebsocketSecurePort 443, WebsocketPort 80, ListenAdress „:8000“, SiteUrl set).

image

Does anyone have ideas on how to resolve the issue?

2

Hi @jasonblais, the community member in question would be me and I’d be happy to answer further questions here. For the record: I suspected a company proxy first, but another team member not using that proxy seems to have the same WebSocket problems accessing our Mattermost instance.

I’ll add some more configuration details:

config.json regarding WebSockets:
31

1 Like
3

kubernetes Service
34

4

kubernetes Ingress
06

5

Hello @Sillium
Thanks for using MM in K8s :slight_smile:

in the service side we are using ClusterIP instead of NodePort.
the ingress looks ok.
and we did not define in the config.json the WebsocketURL can you try to remove?

this is the only differences I can see

we have an instance running on k8s for testing you might want to take a look: https://ci-k8s-mysql.azure.k8s.mattermost.com

6

Hello @cpanato, thanks for the suggestions. WebsocketURL in the config.json is already empty. (I tried setting it explicitly to the same value as SiteURL earlier in my configuration attempts though.)

Could you detail your ClusterIP type service configuration please?

7

I tried commenting out type: NodePort in the service.yaml but to no avail.

8

This is the service we are using

apiVersion: v1
kind: Service
metadata:
  name: mattermost-app-service
  labels:
    app: <LABEL HERE IF YOU ARE USING THAT>
spec:
  selector:
    app: <LABEL HERE IF YOU ARE USING THAT>
  type: ClusterIP
  ports:
  - port: 8065
    targetPort: 8065
    protocol: TCP
    name: app
9

Hi @cpanato, thanks. Can you also tell me what ports you exposed in the container? First I did not explicitly specify container ports, but I also tried with:

spec:
  containers:
  - name: mattermost-app
    image: "mattermost/mattermost-prod-app:5.3.1"
    ports:
    - containerPort: 80
      protocol: TCP
    - containerPort: 8000
      protocol: TCP
    - containerPort: 443
      protocol: TCP

Would you mind showing the relevant parts of your config.json as well?

{
  "ServiceSettings": {
    "SiteURL": "http://<path given in ingress as spec.rules.host>",
    "WebsocketURL": "",
    …
    "ListenAddress": ":8000",
    …
    "WebsocketSecurePort": 443,
    "WebsocketPort": 80,
    …
  }
}

Service:

apiVersion: v1
kind: Service
metadata:
  name: mattermost
spec:
  type: ClusterIP
  ports:
  - port: 80
    targetPort: 8000
    protocol: TCP
    name: http
  selector:
    app: mattermost
    tier: app

Ingress:

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: mattermost
spec:
  rules:
  - host: mattermost.our.url
    http:
      paths:
      - backend:
          serviceName: mattermost
          servicePort: 80
10

humm, you are using the mattermost/mattermost-prod-app image. So for that I have my personal instance and this is the deployment and service I set up

Deployment

apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  name: mattermost-app
  labels:
    app: mattermost
    tier: app
  namespace: default
spec:
  replicas: 1
  template:
    metadata:
      name: mattermost-app
      labels:
        app: mattermost
        tier: app
    spec:
      containers:
      - name: mattermost-app
        image: "mattermost/mattermost-prod-app:5.3.1"
        env:
        - name: DB_HOST
          valueFrom:
            secretKeyRef:
              name: mattermost.env
              key: db-host
        - name: DB_PORT
          valueFrom:
            secretKeyRef:
              name: mattermost.env
              key: db-port
        - name: MM_USERNAME
          valueFrom:
            secretKeyRef:
              name: mattermost.env
              key: mm-username
        - name: MM_PASSWORD
          valueFrom:
            secretKeyRef:
              name: mattermost.env
              key: mm-password
        - name: MM_DBNAME
          valueFrom:
            secretKeyRef:
              name: mattermost.env
              key: mm-dbname
        volumeMounts:
        - name: etclocaltime
          mountPath: /etc/localtime
          readOnly: true
        - name: client-plugins
          mountPath: /mattermost/client/plugins
        - name: client-plugins
          mountPath: /mattermost/client/plugins
        - name: server-plugins
          mountPath: /mattermost/plugins
        - name: data
          mountPath: /mattermost/data
        - name: logs
          mountPath: /mattermost/logs
      volumes:
      - name: etclocaltime
        hostPath:
          path: /etc/localtime
      - name: client-plugins
        emptyDir: {}
      - name: server-plugins
        emptyDir: {}
      - name: data
        emptyDir: {}
      - name: logs
        emptyDir: {}

Service:

apiVersion: v1
kind: Service
metadata:
  name: mattermost
  namespace: default
spec:
  type: NodePort
  ports:
  - port: 80
    targetPort: 8000
    protocol: TCP
    name: http
  selector:
    app: mattermost
    tier: app