Mobile version showing blank groups

This is something I’ve noticed on my phone and it’s really annoying. Archived channels will sometimes appear on my phone when I’m searching for a channel name (but not on desktop).

Re “channels in question” do you mean ones that are giving me the connection problems (that’s all of them–even DM channels!) or archived ones?

Here’s an archived channel that shows up on my phone:

When searching for "bra"vo on desktop, I don’t see anything:

When I go to this channel on mobile, I see:

I tried sharing the link and opened on my desktop machine, but when I open in my browser, it just defaults to the “random” public channel.

BTW, we only have 25 people on MM, that with a pretty distributed team where half those people aren’t online at the same time. It’s worth nothing I’m on Android and those on my team on iPhone are not having these issues.

I verified on the web, the desktop and mobile that my search results always include archived channels (in a different sorting order, but still), so I would assume that it’s OK that they also show up in the search results on your phone. If the same search returns different results when invoked multiple times, that’s a whole different story then. Is this the case?

This is very interesting, that means that either the link is wrong or you should not have access to this channel at all, although there should be a different error message then. At the end of the link you should see an ID - can you post it here so we can check if this ID is part of your logs? Also when you search for this channel in the web version and open it, is the link different to the one you copied out from mobile?

Thanks for the context with regards to seemingly only Android being affected!

Yes, see above. Searching for “bra” (from Bravo), returns an archived result on Mobile, but nothing in the app. I also just confirmed it returns nothing in the browser:

So, to summarize, searching for “bra”, returns a result on mobile, but not on desktop or web.

Are you still a member of this channel or did you leave it before/after it was archived? There might a caching issue on the mobile app. I’m unsure what the expected result is here, is the mobile app right (as in you should be able to find it, because the archived channel is still available and you’re still a member of it) or are the other apps right, because none of this conditions apply? Can you check out the channel in your system console, does it still exist and have members and are you one of them?

This is what I see:

OK, so the channel exists and is definitely archived - you can only see the members in the system console if you unarchive it (you can archive it again later on if you like). I think I lost the overview here a bit, is the only issue left here that the search on mobile does not show this channel? If so, can you please confirm you already updated to version 2.2 of the mobile app which just got released in the last days?

Hi - lol no. That is just one of the smallest symptoms. Here’s some of the issues:

Valid channels show up as dupes–this happens sometimes and not others:

Constantly get errors I can’t connect to the server:


When I get a notification and go to a channel, the channel loads, I see all the recent conversations, and then suddenly it looks like it’s a blank channel:

When I try to share from an external app to a MM channel, I always get this error:

Every day, at least one time, I get this before I have to kill the app and restart:

On top of this, if I’m in MM and a notification comes in, I can’t press any buttons or type until the notification disapears.

Yes, I’m on 2.2:

Thanks for the detailed list, will forward to the developers now.

OK, here’s the first round of answers:

  1. duplicate channels in search view
    Are you able to reproduce that every time or is that just happening randomly and on different channels?

  2. Connection error
    Do you have access to the server logs and is this problem happening at different times of the day and in different networks (mobile and wifi)?

  3. Websocket error
    This message is being printed when the server connection is available, but when the websocket drops (you can think of a websocket as a fast-lane tunneled connection from your client to the server to exchange messages for instant notifications of events, etc.), so the client app then has to resort to polling these events which makes it slower. This is most likely related to 2 and it would also be interesting to see the server logs when that happens.

  4. Empty Channel after load
    This is a known issue which could not be reproduced by the developers so far - do you have stable reproduction steps for this issue or can you describe when it happens the most?

  5. Sharing does not work
    The error message seems to be irritating here, the devs said it is most likely related to anot being able to find channels and this could be related to the connection issues in point 2a) and 2b). Can you reproduce that or does it also happen just randomly since you said you’re always getting this error I assume you have a stable way to reproduce it, right? If so, would you please note down the time when you did try that and afterwards tap on your profile picture in the bottom right, then on Settings and then on “Report a problem” to send the logs of the app to your e-mail address. Please check them for relevant lines and post them here then (or forward them to me as a whole via DM so I can extract the relevant information out of them).

  6. Unexpected error
    Might be related to the app receiving an empty response for posts in a channel (see 3), the devs said: “I understand what is happening in the code, but not sure why that is happening. generateCombinedPosts expects a list of posts to combine. But that list is empty. And that is in theory possible…”

  7. Notification blocks input
    Do you mean the push notifications that get displayed at the top of the app when something happens in another channel or profile?

Random

I can get the server logs.

This is happeniing when I’m both on mobile and wifi data.

OK. What’s the best way to get these (instructions please). And, should I wait for specific instances of this to happen before I get the logs?

Not a consistent approach, but it happens 50% of the time. Click on an Android notification a message was received, get taken to the channel, see all the messages, then see them all disappear.

Will do

Would sending logs after I kill MM and reboot help?

That is correct.

ad 1)
OK, when you see it the next time, please write down the time you experienced it, it should contain entries in the mobile app’s logs which we might be able to refer to afterwards.

ad 2)
The server logs are usually available in /opt/mattermost/logs/mattermost.log, but the directory might be different depending on your installation type, for docker deployments the path is volumes/app/mattermost/logs/mattermost.log relative to your docker base directory, for the Omnibus setup it’s /var/log/mattermost/mattermost.log.

ad 3)
Yes, please note down the timestamp when the connection errors happen and once both have happened (the first one and the websocket one), grab the logs for this timeframe and upload them here (or send them to me so I can filter them out for the relevant parts). If only one of the errors happens, let’s tart with this one, don’t need to wait for both of them.

ad 4)
Same here, write down the time when this happens, the mobile app also keeps all the logs, so if we know what to look for in the logs, it might help to find the culprit then.

ad 5)
Also timestamp here please along with the logs.

ad 6)
It should be part of the logs we’re gathering anyways then ideally, but you can also send them to Mattermost directly for the developers to take a look at.

ad 7)
This is a known issue: [MM-51389] - Mattermost

Okay, update on my situation. Though it may not help here.

Looks like it had to do with our database! It used to be MySQL 5, now MySQL 8. There were intermittent errors in the log about illegal mix of collations. I finally stumbled across those errors because I realized Boards were not working, and the integration had that error in the system console.

After getting all the collation sorted out, things are working much, much better. Not only are the apps no longer having the issues I mentioned, but they are actually behaving differently (correctly, probably). Especially noticeable with the how the new draft features work.

I’m a little disappointed that something more obvious didn’t alert me to the fact that there was a problem :frowning:

Wow - that’s unexpected. Do you still have some of the error messages you’re seeing in the logs available so you could share them with us?

Another update on my issues - turns out, Apache was running out of workers to service requests. We’re using Apache as a reverse proxy in front of Mattermost. So this helps explain some of the intermittent problems users had with posting messages, pulling up channels, etc. After re-configuring Apache, things have improved even more.

Previously, this was part of the error message in mattermost.log with some requests and led me to the DB issue:

Error 1267 (HY000): Illegal mix of collations (utf8mb4_0900_ai_ci,IMPLICIT) and (utf8mb4_general_ci,IMPLICIT) for operation '='"}

Alright, thanks for this update - I should have asked about your environment more so we could also have checked on your Apache configuration earlier.

Did you fix the collation issues already or are they still prominent?

Collation issues are fixed - I have not seen that error since making the db/tables/fields all consistent.

However, there are still issues, just not as often.

First is that the newer version of the mobile client still behaves inconsistently sometimes. Won’t show a conversation when tapping into a channel. Shows user status as offline in the app, even though they are active. Nothing in the server logs that I can find to help explain.

Second, this for sure happens in the Windows desktop app, possibly also in the mobile app. A user clicks in the “Write to ” box, and starts typing a message. The “Write to” does not go away, and the user’s message overlaps on top of it. They can’t hit enter or click send to send the message. Usually adding a space at the end will then allow the message to be sent. Or sometimes they have to click into another channel and come back. Nothing in the MM logs, though I only have them at INFO level at the moment. Apache logs don’t show anything useful around that time. So at the moment I’m not sure what that is all about.

I am getting this message every 2 minutes, but I think it’s being fixed in the next release and from what I can tell it’s not supposed to be causing any issues.

{"timestamp":"2023-04-11 20:26:11.763 -04:00","level":"error","msg":"SimpleWorker: job execution error","caller":"jobs/base_workers.go:91","worker":"InstallNotifyAdmin","job_id":"14113pf8xbnqmgfyqz1kfbyj1y","error":"DoCheckForAdminNotifications: Unable to send notification post., No license found"}

Also getting some periodic 404’s

"GET /static/Metropolis-SemiBoldItalic.woff2 HTTP/1.1" 404

I just saw how to access the mobile client logs! Maybe some of this will help.

FYI, the reverse proxy is set up according to Configuring Apache2 as a proxy for Mattermost Server (Unofficial) — Mattermost documentation

I’m seeing a number of instances of these messages in the mobile client log:

2023/04/12 09:21:32.591 WARN  Handling Javascript error fetchSessions false
2023/04/12 09:21:32.615 ERROR fetchSessions {
  "jsEngine": "hermes",
  "intl": {
    "defaultMessage": "Received invalid response from the server.",
    "id": "mobile.request.invalid_response"
  },
  "url": "\/api\/v4\/users\/me\/sessions",
  "message": "Received invalid response from the server."
}
2023/04/12 09:27:36.096 INFO  websocket re-established connection to wss://mattermost.teamseis.com/api/v4/websocket?connection_id=e6414m1khifu3e1sngm8c8xe4c&sequence_number=1
2023/04/12 09:27:36.109 INFO  wss://mattermost.company.com/api/v4/websocket?connection_id=e6414m1khifu3e1sngm8c8xe4c&sequence_number=1 got connection id  1mf9yqeai3rhpbbih8pnw337xa
2023/04/12 09:27:36.111 INFO  wss://mattermost.company.com/api/v4/websocket?connection_id=e6414m1khifu3e1sngm8c8xe4c&sequence_number=1 long timeout, or server restart, or sequence number is not found, or first connect after failure.
2023/04/12 09:27:36.625 INFO  WEBSOCKET RECONNECT MODELS BATCHING TOOK 42ms
2023/04/12 09:27:52.114 INFO  websocket closed wss://mattermost.company.com/api/v4/websocket?connection_id=e6414m1khifu3e1sngm8c8xe4c&sequence_number=1

Via mobile app, here’s tapping into a channel and getting no discussion displayed

2023/04/12 10:45:23.005 INFO  channel switch to Channel Name Replaced 9ak3iics7br87j1ajumwp1g3yw 62.0 ms
2023/04/12 10:45:23.027 WARN  Handling Javascript error FetchPostsForChannel false
2023/04/12 10:45:23.030 ERROR FetchPostsForChannel {
  "jsEngine": "hermes",
  "intl": {
    "defaultMessage": "Received invalid response from the server.",
    "id": "mobile.request.invalid_response"
  },
  "url": "\/api\/v4\/channels\/9ak3iics7br87j1ajumwp1g3yw\/posts?page=0&per_page=60&collapsedThreads=false&collapsedThreadsExtended=false",
  "message": "Received invalid response from the server."
}
2023/04/12 10:45:23.035 INFO  Running "Channel
2023/04/12 10:45:23.066 DEBUG Error fetching apps {
  "jsEngine": "hermes",
  "intl": {
    "defaultMessage": "Received invalid response from the server.",
    "id": "mobile.request.invalid_response"
  },
  "url": "\/plugins\/com.mattermost.apps\/api\/v1\/bindings?user_id=gcy33ak33fn45jgtywjnw5ixer&channel_id=9ak3iics7br87j1ajumwp1g3yw&team_id=4jw1bqf6kfr97fkfx66fqrpeeh&user_agent=mobile",
  "message": "Received invalid response from the server."
}
2023/04/12 10:45:32.711 WARN  Handling Javascript error fetchSessions false
2023/04/12 10:45:32.716 ERROR fetchSessions {
  "jsEngine": "hermes",
  "intl": {
    "defaultMessage": "Received invalid response from the server.",
    "id": "mobile.request.invalid_response"
  },
  "url": "\/api\/v4\/users\/me\/sessions",
  "message": "Received invalid response from the server."
}
2023/04/12 10:45:41.999 INFO  Running "ChannelInfo
2023/04/12 10:45:54.028 INFO  Running "Settings
2023/04/12 10:46:06.977 WARN  Handling Javascript error fetchSessions false
2023/04/12 10:46:06.979 ERROR fetchSessions {
  "jsEngine": "hermes",
  "intl": {
    "defaultMessage": "Received invalid response from the server.",
    "id": "mobile.request.invalid_response"
  },
  "url": "\/api\/v4\/users\/me\/sessions",
  "message": "Received invalid response from the server."
}

There’s already a ticket for that, please confirm that this is what you’re seeing:
https://mattermost.atlassian.net/browse/MM-48670

This has been fixed already and will be shipped with 7.10:
https://mattermost.atlassian.net/browse/MM-51467

This has also been reported already:
https://mattermost.atlassian.net/browse/MM-51455

I’m wondering why your server responds with an invalid response here according to the client. Since this seems to be reproducible, could you maybe check the access log and probably also turn on debug logging for a short period of time in your Mattermost application server so we can see more detialed events around that time? Looks like either your session expires (which would be strange, because it shouldn’t do that after only a minute) or you’re receiving other invalid responses.

Care to share your Apache reverse proxy configuration?

I’ll see what I can come up with for the logs around those times of the invalid response message.

As for Apache, this is the full virtual host config:

<VirtualHost *:443>
ServerName mattermost.organization.com
SSLEngine on
SSLVerifyClient        none
SSLProtocol all -SSLv2 -SSLv3 -TLSv1 -TLSv1.1
SSLHonorCipherOrder on
ServerAdmin person@organization.com
ErrorLog /var/www/log/mattermost.organization.com_error_log
CustomLog /var/www/log/mattermost.organization.com_access_log combined
SSLCertificateFile /etc/letsencrypt/live/organization.com/fullchain.pem
SSLCertificateChainFile /etc/letsencrypt/live/organization.com/chain.pem
SSLCertificateKeyFile /etc/letsencrypt/live/organization.com/privkey.pem
ProxyPreserveHost On
RequestHeader set "X-Forwarded-Proto" expr=%{REQUEST_SCHEME}
RequestHeader set "X-Forwarded-SSL" expr=%{HTTPS}

  # setup the proxy
  <Proxy *>
         Order allow,deny
         Allow from all
  </Proxy>

  # Set web sockets
  RewriteEngine On
  RewriteCond %{REQUEST_URI} /api/v[0-9]+/(users/)?websocket [NC]
  #RewriteCond %{REQUEST_URI} /api/v[0-9]+/(users/)?websocket [NC,OR]
  RewriteCond %{HTTP:UPGRADE} ^WebSocket$ [NC,OR]
  RewriteCond %{HTTP:CONNECTION} ^Upgrade$ [NC]
  RewriteRule .* ws://127.0.0.1:8065%{REQUEST_URI} [P,QSA,L]

  <Location />
        Require all granted
        ProxyPass http://127.0.0.1:8065/
        ProxyPassReverse http://127.0.0.1:8065/
        ProxyPassReverseCookieDomain 127.0.0.1 mattermost.organization.com
  </Location>
</VirtualHost>