iOS app cannot connect to Mattermost server (iOS v1.8.0)

Troubleshooting
1

Summary

IOS app displaying “connecting…” all the time

Hello

I’m facing issues on our Mattermost setup with iOS devices not being able to connect to our Mattermost server.

Sertificates are from DigiCert and SSL is verified to pass tests.

Steps to reproduce

IOS app version: 1.8.0 (Build 101)
IOS: 11.3.1
Server version: 4.10.0
Database: mysql (Server version: 5.7.22-0ubuntu0.16.04.1 (Ubuntu))
License: Trial license
MM server: Ubuntu 16.04 LTS
Proxy server: Windows Server 2012R2 (6.3 Build 9600), ARR 3.0 (3.0.1988), URL Rewrite module 2 (7.2.1980)
Topology: public IPv4-address–> IIS ARR 3.0 Reverse proxy (DMZ)–> MM server(private ipv4 address, inside)

Config.json:

{
“ServiceSettings”: {
“SiteURL”: “https://mmgw.domain.tld”,
“WebsocketURL”: “”,
“LicenseFileLocation”: “”,
“ListenAddress”: “:443”,
“ConnectionSecurity”: “TLS”,
“TLSCertFile”: “./cert/star_domain_tld.crt”,
“TLSKeyFile”: “./cert/mattermost.key”,
“UseLetsEncrypt”: false,
“LetsEncryptCertificateCacheFile”: “./config/letsencrypt.cache”,
“Forward80To443”: true,
“ReadTimeout”: 300,
“WriteTimeout”: 300,
“MaximumLoginAttempts”: 10,
“GoroutineHealthThreshold”: -1,
“GoogleDeveloperKey”: “xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx”,
“EnableOAuthServiceProvider”: false,
“EnableIncomingWebhooks”: true,
“EnableOutgoingWebhooks”: true,
“EnableCommands”: true,
“EnableOnlyAdminIntegrations”: true,
“EnablePostUsernameOverride”: false,
“EnablePostIconOverride”: false,
“EnableAPIv3”: true,
“EnableLinkPreviews”: true,
“EnableTesting”: false,
“EnableDeveloper”: false,
“EnableSecurityFixAlert”: true,
“EnableInsecureOutgoingConnections”: true,
“AllowedUntrustedInternalConnections”: “”,
“EnableMultifactorAuthentication”: false,
“EnforceMultifactorAuthentication”: false,
“EnableUserAccessTokens”: false,
“AllowCorsFrom”: “*”,
“AllowCookiesForSubdomains”: false,
“SessionLengthWebInDays”: 30,
“SessionLengthMobileInDays”: 30,
“SessionLengthSSOInDays”: 30,
“SessionCacheInMinutes”: 10,
“SessionIdleTimeoutInMinutes”: 0,
“WebsocketSecurePort”: 443,
“WebsocketPort”: 80,
“WebserverMode”: “gzip”,
“EnableCustomEmoji”: true,
“EnableEmojiPicker”: true,
“RestrictCustomEmojiCreation”: “all”,
“RestrictPostDelete”: “all”,
“AllowEditPost”: “always”,
“PostEditTimeLimit”: -1,
“TimeBetweenUserTypingUpdatesMilliseconds”: 5000,
“EnablePostSearch”: true,
“EnableUserTypingMessages”: true,
“EnableChannelViewedMessages”: true,
“EnableUserStatuses”: true,
“ExperimentalEnableAuthenticationTransfer”: true,
“ClusterLogTimeoutMilliseconds”: 2000,
“CloseUnusedDirectMessages”: false,
“EnablePreviewFeatures”: true,
“EnableTutorial”: true,
“ExperimentalEnableDefaultChannelLeaveJoinMessages”: true,
“ExperimentalGroupUnreadChannels”: “disabled”,
“ImageProxyType”: “”,
“ImageProxyURL”: “”,
“ImageProxyOptions”: “”
},
“TeamSettings”: {
“SiteName”: “Organiztion Mattermost”,
“MaxUsersPerTeam”: 250,
“EnableTeamCreation”: false,
“EnableUserCreation”: true,
“EnableOpenServer”: false,
“RestrictCreationToDomains”: “domain.tld, domain2.tld, domain3.tld, domain4.tld”,
“EnableCustomBrand”: true,
“CustomBrandText”: “”,
“CustomDescriptionText”: “Lorem Ipsum”,
“RestrictDirectMessage”: “any”,
“RestrictTeamInvite”: “all”,
“RestrictPublicChannelManagement”: “channel_admin”,
“RestrictPrivateChannelManagement”: “channel_admin”,
“RestrictPublicChannelCreation”: “all”,
“RestrictPrivateChannelCreation”: “all”,
“RestrictPublicChannelDeletion”: “team_admin”,
“RestrictPrivateChannelDeletion”: “channel_admin”,
“RestrictPrivateChannelManageMembers”: “channel_admin”,
“EnableXToLeaveChannelsFromLHS”: false,
“UserStatusAwayTimeout”: 300,
“MaxChannelsPerTeam”: 2000,
“MaxNotificationsPerChannel”: 1000,
“EnableConfirmNotificationsToChannel”: true,
“TeammateNameDisplay”: “full_name”,
“ExperimentalEnableAutomaticReplies”: false,
“ExperimentalTownSquareIsReadOnly”: false,
“ExperimentalPrimaryTeam”: “”
},
“SqlSettings”: {
“DriverName”: “mysql”,
“DataSource”: "mmuser:PASSWORD@tcp(127.0.0.1:3306)/mattermost?charset=utf8mb4,utf8\u0026readTimeout=30$
“DataSourceReplicas”: [],
“DataSourceSearchReplicas”: [],
“MaxIdleConns”: 20,
“MaxOpenConns”: 300,
“Trace”: false,
“AtRestEncryptKey”: “aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa”,
“QueryTimeout”: 30
},
“LogSettings”: {
“EnableConsole”: true,
“ConsoleLevel”: “INFO”,
“ConsoleJson”: true,
“EnableFile”: true,
“FileLevel”: “DEBUG”,
“FileJson”: true,
“FileLocation”: “”,
“EnableWebhookDebugging”: true,
“EnableDiagnostics”: true
},
“PasswordSettings”: {
“MinimumLength”: 5,
“Lowercase”: false,
“Number”: false,
“Uppercase”: false,
“Symbol”: false
},
“FileSettings”: {
“EnableFileAttachments”: true,
“EnableMobileUpload”: true,
“EnableMobileDownload”: true,
“MaxFileSize”: 52428800,
“DriverName”: “local”,
“Directory”: “./data/”,
“EnablePublicLink”: false,
“PublicLinkSalt”: “wrfs98yu9n7gj6xrpr67kob1mf9e8ase”,
“InitialFont”: “luximbi.ttf”,
“AmazonS3AccessKeyId”: “”,
“AmazonS3SecretAccessKey”: “”,
“AmazonS3Bucket”: “”,
“AmazonS3Region”: “”,
“AmazonS3Endpoint”: “s3.amazonaws.com”,
“AmazonS3SSL”: true,
“AmazonS3SignV2”: false,
“AmazonS3SSE”: false,
“AmazonS3Trace”: false
},
“EmailSettings”: {
“EnableSignUpWithEmail”: true,
“EnableSignInWithEmail”: true,
“EnableSignInWithUsername”: true,
“SendEmailNotifications”: true,
“UseChannelInEmailNotifications”: false,
“RequireEmailVerification”: false,
“FeedbackName”: “No-Reply-Mattermost”,
“FeedbackEmail”: “mattermost@domain.tld”,
“FeedbackOrganization”: “Organization”,
“EnableSMTPAuth”: false,
“SMTPUsername”: “user@domain.tld”,
“SMTPPassword”: “password”,
“SMTPServer”: “smtp.domain.tld”,
“SMTPPort”: “25”,
“ConnectionSecurity”: “”,
“InviteSalt”: “aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa”,
“SendPushNotifications”: true,
“PushNotificationServer”: “https://push.mattermost.com”,
“PushNotificationContents”: “generic”,
“EnableEmailBatching”: true,
“EmailBatchingBufferSize”: 256,
“EmailBatchingInterval”: 30,
“SkipServerCertificateVerification”: false,
“EmailNotificationContentsType”: “full”,
“LoginButtonColor”: “”,
“LoginButtonBorderColor”: “”,
},
“RateLimitSettings”: {
“Enable”: false,
“PerSec”: 10,
“MaxBurst”: 100,
“MemoryStoreSize”: 10000,
“VaryByRemoteAddr”: true,
“VaryByUser”: false,
“VaryByHeader”: “”
},
“PrivacySettings”: {
“ShowEmailAddress”: true,
“ShowFullName”: true
“SupportSettings”: {
“TermsOfServiceLink”: “https://about.mattermost.com/default-terms/”,
“PrivacyPolicyLink”: “https://about.mattermost.com/default-privacy-policy/”,
“AboutLink”: “https://about.mattermost.com/default-about/”,
“HelpLink”: “https://about.mattermost.com/default-help/”,
“ReportAProblemLink”: “https://about.mattermost.com/default-report-a-problem/”,
“SupportEmail”: “mail@domain.tld”
},
“AnnouncementSettings”: {
“EnableBanner”: true,
“BannerText”: “”,
“BannerColor”: “#f2a93b”,
“BannerTextColor”: “#333333”,
“AllowBannerDismissal”: false
},
“ThemeSettings”: {
“EnableThemeSelection”: true,
“DefaultTheme”: “default”,
“AllowCustomThemes”: true,
“AllowedThemes”: []
},
“LdapSettings”: {
“Enable”: false,
“EnableSync”: false,
“LdapServer”: “”,
“LdapPort”: 389,
“ConnectionSecurity”: “”,
“BaseDN”: “”,
“BindUsername”: “”,
“BindPassword”: “”,
“UserFilter”: “”,
“FirstNameAttribute”: “”,
“LastNameAttribute”: “”,
“EmailAttribute”: “”,
“UsernameAttribute”: “”,
“NicknameAttribute”: “”,
“IdAttribute”: “”,
“PositionAttribute”: “”,
“SyncIntervalMinutes”: 60,
“SkipCertificateVerification”: false,
“QueryTimeout”: 60,
“MaxPageSize”: 0,
“LoginFieldName”: “”,
“LoginButtonColor”: “”,
“LoginButtonBorderColor”: “”,
“LoginButtonTextColor”: “”
},
“ComplianceSettings”: {
“Enable”: false,
“Directory”: “./data/”,
“EnableDaily”: false
},
“LocalizationSettings”: {
“DefaultServerLocale”: “en”,
“DefaultClientLocale”: “en”,
“AvailableLocales”: “en”
},
“SamlSettings”: {
“Enable”: false,
“Verify”: true,
“Encrypt”: true,
“IdpUrl”: “”,
“IdpDescriptorUrl”: “”,
“AssertionConsumerServiceURL”: “”,
“ScopingIDPProviderId”: “”,
“ScopingIDPName”: “”,
“IdpCertificateFile”: “”,
“PublicCertificateFile”: “”,
“PrivateKeyFile”: “”,
“FirstNameAttribute”: “”,
“LastNameAttribute”: “”,
“EmailAttribute”: “”,
“UsernameAttribute”: “”,
“NicknameAttribute”: “”,
“LocaleAttribute”: “”,
“PositionAttribute”: “”,
“LoginButtonText”: “With SAML”,
“LoginButtonColor”: “”,
“LoginButtonBorderColor”: “”,
},
“NativeAppSettings”: {
“AppDownloadLink”: “softwarecenter:SoftwareID=ScopeId–xxx”
“AndroidAppDownloadLink”: “https://about.mattermost.com/mattermost-android-app/”,
“IosAppDownloadLink”: “https://about.mattermost.com/mattermost-ios-app/”
},
“ClusterSettings”: {
“Enable”: false,
“ClusterName”: “”,
“OverrideHostname”: “”,
“UseIpAddress”: true,
“UseExperimentalGossip”: false,
“ReadOnlyConfig”: true,
“GossipPort”: 8074,
“StreamingPort”: 8075
},
“MetricsSettings”: {
“Enable”: false,
“BlockProfileRate”: 0,
“ListenAddress”: “:8067”
},
“AnalyticsSettings”: {
“MaxUsersForStatistics”: 2500
},
“WebrtcSettings”: {
“Enable”: false,
“GatewayWebsocketUrl”: “”,
“GatewayAdminUrl”: “”,
“GatewayAdminSecret”: “”,
“StunURI”: “”,
“TurnURI”: “”,
“TurnUsername”: “”,
“TurnSharedKey”: “”
},
“ElasticsearchSettings”: {
“ConnectionUrl”: “http://dockerhost:9200”,
“Username”: “elastic”,
“ConnectionUrl”: “http://dockerhost:9200”,
“Username”: “elastic”,
“Password”: “changeme”,
“EnableIndexing”: false,
“EnableSearching”: false,
“Sniff”: true,
“PostIndexReplicas”: 1,
“PostIndexShards”: 1,
“AggregatePostsAfterDays”: 365,
“PostsAggregatorJobStartTime”: “03:00”,
“IndexPrefix”: “”,
“LiveIndexingBatchSize”: 1,
“BulkIndexingTimeWindowSeconds”: 3600,
“RequestTimeoutSeconds”: 30
},
“DataRetentionSettings”: {
“EnableMessageDeletion”: false,
“EnableFileDeletion”: false,
“MessageRetentionDays”: 365,
“FileRetentionDays”: 365,
“DeletionJobStartTime”: “02:00”
},
“MessageExportSettings”: {
“EnableExport”: false,
“ExportFormat”: “actiance”,
“DailyRunTime”: “01:00”,
“ExportFromTimestamp”: 0,
“BatchSize”: 10000,
“GlobalRelaySettings”: {
“CustomerType”: “A9”,
“SmtpUsername”: “”,
“SmtpPassword”: “”,
“EmailAddress”: “”
}
},
“JobSettings”: {
“RunJobs”: true,
“RunScheduler”: true
},
“PluginSettings”: {
“Enable”: true,
“EnableUploads”: false,
“Directory”: “./plugins”,
“ClientDirectory”: “./client/plugins”,
“Plugins”: {},
“PluginStates”: {
“jira”: {
“Enable”: true
}
}
},
“DisplaySettings”: {
“ExperimentalTimezone”: false
},
“TimezoneSettings”: {
“SupportedTimezonesPath”: “timezones.json”
}
}

Expected behavior

iOS app able to connect without “Connecting…” bar showing up constantly.

Observed behavior

With our current setup and config these are working:

  • Windows: client (inside)
  • Windows: browser (inside)
  • Windows: client (outside)
  • Windows: browser (outside)
  • IOS: using via Browser [safari] (outside).
  • IOS: app against MM demo site 4.10.0 RC2 (different license) .
  • Android: using via Browser [chrome] (outside)
  • Android: App (outside)

IOS devices has been tested with MDM (AirWatch) and without MDM client.
mmgw

CORS setting * allows android apps to connect. CORS setting to wss://mmgw.domain.tld:443 didn’t solve our issue as other users have reported to be working for them. I have tried to eliminate possible reasons one by one, but I’m running out of ideas what to do and test next.

Any help would be greatly appreciated.

Best Regards

2

@jazuliuz - If the SSL cert is verified and not missing any intermediate certificates, the next thing to check might be the configuration for your proxy.

3

Hi

Our web proxy (IIS 8.5) configs are:

<?xml version="1.0" encoding="UTF-8"?>
<configuration>
    <system.webServer>
        <rewrite>
            <outboundRules>
                <clear />
                <rule name="ReverseProxyOutboundMM" preCondition="ResponseIsHtml1" enabled="true">
                    <match filterByTags="A, Form, Img" pattern="^http(s)?://inside.domain.com/(.*)" />
                    <conditions logicalGrouping="MatchAll" trackAllCaptures="true" />
                    <action type="Rewrite" value="http{R:1}://outside.domain.com/{R:2}" />
                </rule>
                <preConditions>
                    <preCondition name="ResponseIsHtml1">
                        <add input="{RESPONSE_CONTENT_TYPE}" pattern="^text/html" />
                    </preCondition>
                </preConditions>
            </outboundRules>
            <rules>
                <clear />
                <rule name="ReverseProxy_MM" enabled="true" stopProcessing="true">
                    <match url="(.*)" />
                    <conditions logicalGrouping="MatchAll" trackAllCaptures="true">
                    </conditions>
                    <action type="Rewrite" url="https://inside.domain.com/{R:0}" />
                </rule>
            </rules>
            <rewriteMaps>
                <rewriteMap name="HTTP_SEC_WEBSOCKET_EXTENSIONS">
                    <add key="" value="" />
                </rewriteMap>
            </rewriteMaps>
        </rewrite>
        <tracing>
            <traceFailedRequests>
               <remove path="*" />
                <add path="*">
                    <traceAreas>
                        <add provider="WWW Server" areas="Rewrite" verbosity="Verbose" />
                    </traceAreas>
                    <failureDefinitions timeTaken="00:00:00" statusCodes="300" verbosity="Error" />
                </add>
            </traceFailedRequests>
        </tracing>
        <security>
            <requestFiltering allowDoubleEscaping="true" />
        </security>
    </system.webServer>
</configuration>

I have tested with allowDoubleEscaping=“true” and without HTTP_SEC_WEBSOCKET_EXTENSIONS

With the current settings, inside (private LAN) and outside (DMZ) have been tested:
Windows: client (inside)
Windows: browser (inside)
Windows: client (outside)
Windows: browser (outside)
IOS: using via Browser [safari] (outside).
IOS: app against MM demo site 4.10.0 RC2 (different license) .
Android: using via Browser [chrome] (outside)
Android: App (outside)

For reference, from Mattermost server to DMZ Proxy ports 80, 443 and 8065 are open. From DMZ Proxy to WAN ports 80 and 443 are open.

According to these test everything except iOS app is working as expected. iOS app has been tested also from inside (LAN) and app still shows with Connecting bar.

I’ve used Official Ubuntu 16.04 LTS installation guide for server, and unofficial Windows IIS guide for proxy.

4

I don’t know much about IIS but you need to make sure the websocket connection can be upgraded when making a request to api/v4/websocket, the connecting bar indicates that the websocket couldn’t establish the connection

5

Desktop browser upgrades connection from http/https to websocket. But using debug tools on iPhone clearly app doesn’t establish websocket connection or the app doesn’t seem to even try to establish websocket.

What puzzles me is, that demo site with iPhone app is working and no Connecting bar issues persist, but debug tools don’t show websocket connections to demo site either. Demo site had 4.10.0-rc2 at the time of inital tests and our on premise server 4.9.2.

6

Could you share privately (With the mattermost developers team) the real url and a user account to an empty team to try to reproduce it? You can find us in https://pre-release.mattermost.com or you can send me an email to jesus@mattermost.com.

7

Hi jespino

I sent you via secure email login credentials and site url.

8

I received that. Looks like some kind of problem with the SSL/TLS, but is something that we need to investigate more. The problem is that the application isn’t able to open the websocket connection, but we don’t know exactly why yet.

9

@jespino Thank you for helping with this issue, let me know at any point if I should create a ticket for this or if you need any other help with troubleshooting this!

10

I’ll be on vacation from 1st June to 17th June, and won’t be following this thread. I’ll ask my collegue to follow up on this thread and provide necessary credentials and system info to support from our side.

1 Like
11

Got the same problem. Thanks for your help

12

Did this issue ever get solved? I have a very similar setup with IIS8.5 Reverse Proxy on the DMZ.

All internal traffic is connecting to websockets just fine. With the relevant AllowCorsFrom config Andorid, desktop and all web clients are able to also connect successfully. The only issue is the iOS app.

We are on MM Server version 5.1.0
iOS app version 1.12

Thanks

13

Hi @tomstieger, thank you for reaching out.

Can you help take a look at this troubleshooting doc for tips: https://docs.mattermost.com/mobile/mobile-troubleshoot.html#i-see-a-connecting-bar-that-does-not-go-away?

14

Hi

We are still having this issus. I changed our proxy from IIS to nginx but it didn’t solve this. We have support ticket on business side open and my collegue is taking care of that. I’m afraid it this issue can’t be solved shortly, my managers will tell me to shut down our Mattermost server and abandon this proof of concept project for good.

15

Thanks Amy. I had seen that link. This week I will look again at the rewrite rules on IIS8.5 (ARR3.0) to see if that can fix the problem.

Again, this is only happening on the iOS app when connecting outside the firewall.

We have an nginx reverse proxy on the mattermost server that is working for internal connections.

16

I found the following message regarding Websockets and IIS 8.5 (ARR3.0)

…add HTTP_SEC_WEBSOCKET_EXTENSIONS to the allowed server variable list, then add it to your rule:

<serverVariables><set name="HTTP_SEC_WEBSOCKET_EXTENSIONS" value="" /></serverVariables>

This seemed to work for our setup and our iOS mattermost app clients outside the firewall are able to connect with real time support.

Let me know if you would like any would detailed configs.

UPDATE: Never mind. I am finding the same problem as previously reported, that this settings still does not help. (My test device had switched to the internal network, which caused websockets to connect.)

Current setup:

WAN (https)-> DMZ with IIS8.5 (https)-> mmserver nginx -> mmserver:8065
                                 LAN --/

All connections work with websockets except for iOS app.

18

Do you have any other ideas to trouble shoot this problem?

I have copies of the request headers that are being sent from iOS app and failing along with successful headers from Windows/Chrome and Android, but I am not able to see any meaningful difference.

19

@jazuliuz

I think I found a solution for looking at the fail request tracing in IIS. The iOS app is the only method that was sending the Sec-WebSocket-Protocol variable in the HTTP request header. In my URL rewrite rule, I set a Server Variable HTTP_Sec_WebSocket_Protocol to a static value of ‘mattermost’ and everything started working.

Hope this helps your setup.

20

I have also tried connecting the mattermost server but I have been asked to connect to the server. As I was trying this from my iPhone, so I needed my Apple Id. But I don’t know I was having Error connecting to Apple id server.

How can I connect the mattermost now?

21

Hi @lilamartin,

Please take a look at this section of our documentation as a first step: https://docs.mattermost.com/mobile/mobile-troubleshoot.html#i-keep-getting-a-message-cannot-connect-to-the-server-please-check-your-server-url-and-internet-connection.