Using the mobile emoji keyboard renders "?" on other platforms

Using the mobile emoji keyboard renders “???” on other platforms.

When users use the internal Mattermost Emoji-System (typing : etc. or choosing from the emoji-List on the desktop client/desktop browser) everything looks good. But especially on mobile, users tend to just user their regular emoji keyboards.

There is potentially a combined Client-Backend-encoding problem. Clients send the Emoji Unicode to Server and somewhere on the way (Sending, Server-Side-Saving, or Delivering to others clients) there is a “translation” problem.

Original Twitter thread here: https://twitter.com/kraehenpost/status/1007245360769662976

Here are sample screenshots:

  1. MacOS-Mattermost-App (ver. 4.0.1)
    image

  2. MacOS-Firefox
    image

  3. iOS Classic App
    image

  4. iOS RN App: You can see that the emoji rendered here work fine. However, those posted from the Android RN App render as “???” as well
    image

1 Like

Additional info from reporter:

Mattermost-Version: 4.9.2
Database-Version: 4.9.0
Database: mysql
Server OS: Debian GNU/Linux 8
My Desktop OS: macOS (High Sierra)
My mobile OS: iOS 11.4

But client versions vary much in our team. Have lot's of Android users plus Desktop Browser-only users. Windows 7 perhaps, don't know.

If there are additional questions regarding my case (I’m user “matze” in the screenshots above) just let me know.

Thanks @kraehenpost!

One proposal you can help with is to get the raw text of the emojis posted from Firefox and the Desktop app to see if they’re different:

  1. Post the set of emojis from above screenshot in Firefox.
  2. Edit the post in Mattermost, so you have access to the raw text. Copy and paste this into a text editor.
  3. Post the set of emojis in MacOS Desktop App.
  4. Again, edit the post. Then copy and paste the raw text into a text editor.
  5. Share the contents of the text editor here.

That way we can compare if the emoji posted from the Mac emoji keyboard on Mattermost MacOS app are different from the ones posted in Firefox. Can help give a clue.

You can do the same from the iOS app if you have the time :slight_smile:

Hey, I#m note quite sure if I’m doing the right thing here. Again: If I’m using the built-in Mattermost Emoji-Button, the results are Inter-platform-compatible. Problem only starts, when I’m using the OS-specific Emoji-Keyboards. For the tests you suggested, I see the Emoji-keyboard built-in in macOS (for both, the Firefox-port und the Mattermost-Desktop-App-Post. Having said this, here are my results, directly copied from the text editor:

Result from step 1-2:
Here’s a set of emojis posted from Firefox: :smiling_imp::imp::japanese_ogre::japanese_goblin::clown_face::poop:

Result from step 3-4:
Here’s a set of emojis posted from Mattermos Desktop App: :smiling_imp::imp::japanese_ogre::japanese_goblin::clown_face::poop:

And here’s how the test post render on iOS, first the Mattermost iOS-Classic-App and second the Mattermost current iOS App (see the two lines on the bottom):IMG_7905IMG_7906

1 Like

Thanks @kraehenpost! This is great. Appreciate it! Minor ask, can you help paste the emojis above inside a code block by surrounding them by a backtick like:

` <emoji1> <emoji2> `

Our engineer was taking a look at the emojis, but seems like the forums are manipulating the emojis in some way.

1 Like

okay, please have another look into the post above. I used the backticks as suggested.

hey, just wanted to check if there’s anything new about this issue? (Please note, I’m using another account, my former account kraehenpost was twitter login-based and I deleted my twitter account (long story), so…

Just another observation regarding this issue: I observed, that Emoji rendering seems to be releated to the question, if the user, who only get’s the ??? instead of emojis is live in the chat or not. example:

When I have a multi-platform chat with another user, the emojis may render completely fine, when the other one is online in the same live chat. Then again, after the user closes his/her browser completely and re-opens it again, the same chat, where emojis seemed fine 1 min ago, is full of ???

So is this perhaps more of a caching issue or something like that? Or is it connected to the question, if real time messages is used or not? push notifications for a chat/channel message, for example, also display correctly, but when you open the chat/channel it’s often only ???

1 Like

@jasonblais
Hey Jason, as I’m limited to three replies as a new account, I’ll answer here:

Thanks for your tipp (below this comment). It’s totally weird. Couldn’t reproduce on your test system either.

BUT THE GOOD NEWS IS: I finally found the issue on our server. You can close this ticket now. I’m such a DAU User!!! Sorry for keeping you busy. But just for the record:

My installation is a Bitnami pre-configured AWS-Machine with Mattermost. And there was just a wrong line in the config.json regarding the mysql-connection. It was:

“DataSource”: “bn_username:password@tcp(127.0.0.1:3306)/bitnami_mattermost?charset=utf8”,

correct would be
“DataSource”: “bn_username:password@tcp(127.0.0.1:3306)/bitnami_mattermost?charset=utf8mb4,utf8”,

1 Like

Thanks for the follow-up. We haven’t so far been able to reproduce this on our end.

Would you be open to trying if it reproduces on our community instance at https://pre-release.mattermost.com/core?

thanks for your help. As I already mentioned in my updated message above, I found the issue and was able to solve it:

My installation is a Bitnami pre-configured AWS-Machine with Mattermost. And there was just a wrong line in the config.json regarding the mysql-connection. It was:

“DataSource”: “username:password@tcp(127.0.0.1:3306)/bitnami_mattermost?charset=utf8”,

correct would be
“DataSource”: “username:password@tcp(127.0.0.1:3306)/bitnami_mattermost?charset=utf8mb4,utf8”,

1 Like

I am seeing this as well on the Bitnami GCP deployment. I’ll try adding the charset=utf8mb4, to the connection string.