Hi Guys, I was able to set up the connection between MM vs B2 via S3 interface.
But I got the problem after testing connection successfully.
Any idea?
{“timestamp”:“2023-03-13 16:40:35.008 -07:00”,“level”:“warn”,“msg”:“Error decoding the config”,“caller”:“api4/system.go:473”,“path”:“/api/v4/file/s3_test”,“request_id”:“c74rhj11n7ntmmocfhykcfogkr”,“ip_addr”:“x.x.x.x”,“user_id”:“npgpt8cmn84ue9b53qfwc”,“method”:“POST”,“error”:“EOF”}
Hi @semighoti and welcome to the Mattermost forums!
I’m sorry you’d have to help me with that - what is B2 and what instructions did you follow for the interconnect?
Hi @agriesser sorry for the confusion, I should have been more clear about my question.
B2 is BackBlaze storage. I just filled out all the credentials onto MM from B2. When I tested the connection, it worked, but not uploading any thing to B2 after I sent a user a pdf file.
Hope clear my question above.
Thanks, never heard of them. It might well be that they are not 100% compatible to the client inside the Mattermost server application. As far as I know the only officially supported S3 backend is AWS S3, although they claim that other backens should be supported as well. Can you share the settings you’ve made in system console for this storage provider? Not sure how thoroughly the connection tester works here, but it maybe only tries to connect and not tries to write and read a file and the connection could work for you, but maybe writing files doesn’t.
Any specialities that you did enable on the S3 bucket with regards to client side encryption requirements or the like?
this is my setup
And when the connection was successfully, I went to check server logs and got this error
{“timestamp”:“2023-03-13 21:25:17.723 -07:00”,“level”:“warn”,“msg”:“Error decoding the config”,“caller”:“api4/system.go:473”,“pathdn”:“/api/v4/file/s3_test”,“request_id”:“aqw7y3rwtjfopkw81941ytedfh”,“ip_addr”:“x.x.x.x”,“user_id”:“z34dhnpgpt8cmn84ue9b53qfwc”,“method”:“POST”,“error”:“EOF”}
Use this instruction to install MM on docker compose without Nginx (I installed Nginx after MM installation)
https://docs.mattermost.com/install/install-docker.html
Can you please enable the debugging option in system console on your screenshot and try again to upload a file and check the servers logs then? Maybe they contain additional helpful information then.
I got some of them, not sure which one is useful
{“timestamp”:“2023-03-13 16:44:44.024 -07:00”,“level”:“info”,“msg”:“api endpoint requires a license”,“caller”:“web/context.go:115”,“path”:“/api/v4/users/drb55non9tfmjcd9jtoprn7oua/groups”,“request_id”:“3xky4y47ijfo88gda6tfdimwsc”,“ip_addr”:“172.26.0.1”,“user_id”:“drb55non9tfmjcd9jtoprn7oua”,“method”:“GET”,“err_where”:“”,“http_code”:501,“error”:“: api endpoint requires a license”}
{“timestamp”:“2023-03-13 16:47:16.538 -07:00”,“level”:“info”,“msg”:“api endpoint requires a license”,“caller”:“web/context.go:115”,“path”:“/api/v4/users/drb55non9tfmjcd9jtoprn7oua/groups”,“request_id”:“us93mge9ipfqjf5y8xzhtmnd1c”,“ip_addr”:“172.26.0.1”,“user_id”:“drb55non9tfmjcd9jtoprn7oua”,“method”:“GET”,“err_where”:“”,“http_code”:501,“error”:“: api endpoint requires a license”}
{“timestamp”:“2023-03-13 16:47:45.083 -07:00”,“level”:“info”,“msg”:“api endpoint requires a license”,“caller”:“web/context.go:115”,“path”:“/api/v4/users/drb55non9tfmjcd9jtoprn7oua/groups”,“request_id”:“553upjjwityqtq414si1wjhpfa”,“ip_addr”:“172.26.0.1”,“user_id”:“drb55non9tfmjcd9jtoprn7oua”,“method”:“GET”,“err_where”:“”,“http_code”:501,“error”:“: api endpoint requires a license”}
{“timestamp”:“2023-03-13 17:35:07.917 -07:00”,“level”:“info”,“msg”:“SimpleWorker: Job is complete”,“caller”:“jobs/base_workers.go:96”,“worker”:“ProductNotices”,“job_id”:“zikyyuahnbdipxtiy8j3i6aq6y”}
{“timestamp”:“2023-03-13 18:35:13.273 -07:00”,“level”:“info”,“msg”:“SimpleWorker: Job is complete”,“caller”:“jobs/base_workers.go:96”,“worker”:“ProductNotices”,“job_id”:“jbccwtxugt8tbki85k5h9wgith”}
{“timestamp”:“2023-03-13 19:35:08.926 -07:00”,“level”:“info”,“msg”:“SimpleWorker: Job is complete”,“caller”:“jobs/base_workers.go:96”,“worker”:“ProductNotices”,“job_id”:“1oj7jdtuif8appjiu77ephis7a”}
{“timestamp”:“2023-03-13 20:35:07.028 -07:00”,“level”:“info”,“msg”:“SimpleWorker: Job is complete”,“caller”:“jobs/base_workers.go:96”,“worker”:“ProductNotices”,“job_id”:“513hnzpnbbypiem558ueqjikch”}
{“timestamp”:“2023-03-13 21:18:35.285 -07:00”,“level”:“warn”,“msg”:“Error decoding the config”,“caller”:“api4/system.go:473”,“path”:“/api/v4/file/s3_test”,“request_id”:“yopx44ahmjdhxkea4uf9ddpkqa”,“ip_addr”:“172.26.0.1”,“user_id”:“z34dhnpgpt8cmn84ue9b53qfwc”,“method”:“POST”,“error”:“EOF”}
{“timestamp”:“2023-03-13 21:25:17.723 -07:00”,“level”:“warn”,“msg”:“Error decoding the config”,“caller”:“api4/system.go:473”,“path”:“/api/v4/file/s3_test”,“request_id”:“aqw7y3rwtjfopkw81941m18ezo”,“ip_addr”:“172.26.0.1”,“user_id”:“z34dhnpgpt8cmn84ue9b53qfwc”,“method”:“POST”,“error”:“EOF”}
{“timestamp”:“2023-03-13 21:35:06.985 -07:00”,“level”:“info”,“msg”:“SimpleWorker: Job is complete”,“caller”:“jobs/base_workers.go:96”,“worker”:“ProductNotices”,“job_id”:“pwoasp38ntgkirdoq9r9yjj45y”}
{“timestamp”:“2023-03-13 21:42:13.692 -07:00”,“level”:“info”,“msg”:“api endpoint requires a license”,“caller”:“web/context.go:115”,“path”:“/api/v4/users/drb55non9tfmjcd9jtoprn7oua/groups”,“request_id”:“1z5c4p75mjfkicz7qoy5dssq4h”,“ip_addr”:“172.26.0.1”,“user_id”:“drb55non9tfmjcd9jtoprn7oua”,“method”:“GET”,“err_where”:“”,“http_code”:501,“error”:“: api endpoint requires a license”}
{“timestamp”:“2023-03-13 21:56:00.277 -07:00”,“level”:“warn”,“msg”:“Unrecognized config permissions tag value.”,“caller”:“api4/config.go:426”,“tag_value”:“sysconsole_write_*_read”}
{“timestamp”:“2023-03-13 21:56:04.953 -07:00”,“level”:“warn”,“msg”:“Error decoding the config”,“caller”:“api4/system.go:473”,“path”:“/api/v4/file/s3_test”,“request_id”:“k7hxdhuewfb8dyr1wcfdu3nqmy”,“ip_addr”:“172.26.0.1”,“user_id”:“z34dhnpgpt8cmn84ue9b53qfwc”,“method”:“POST”,“error”:“EOF”}
Unfortunately, there’s no additional information posted, it might well be that you need to switch the server log to debug too for the logs to appear.
In your System Console, go to Environment → Logging and set the file log level to DEBUG and try again please.
Hi @agriesser I followed your instruction and got these warning. Could you check them for me please? Thanks. I wanna set up a webhook as well, and I run this on my local network
{"timestamp":"2023-03-16 03:46:01.229 Z","level":"debug","msg":"Received HTTP request","caller":"web/handlers.go:171","method":"GET","url":"/api/v4/system/ping","request_id":"5q4nkxtgo7bh8xodngkyjf5pbr","status_code":"200"}
{"timestamp":"2023-03-16 03:46:02.687 Z","level":"info","msg":"Draft.Message has size restrictions","caller":"sqlstore/draft_store.go:257","max_characters":16383,"max_bytes":65535}
{"timestamp":"2023-03-16 03:46:02.700 Z","level":"debug","msg":"Received HTTP request","caller":"web/handlers.go:171","method":"POST","url":"/api/v4/drafts","request_id":"mjyhg9z86bdr8fiew3fg897s9a","status_code":"201"}
{"timestamp":"2023-03-16 03:46:04.239 Z","level":"debug","msg":"Received HTTP request","caller":"web/handlers.go:171","method":"PUT","url":"/api/v4/users/y8mxmju4wbbtbmep15p1arjg7c/preferences","request_id":"fi3965g7ojbqurqp3mbeauo8ke","status_code":"200"}
{"timestamp":"2023-03-16 03:46:08.443 Z","level":"debug","msg":"Received HTTP request","caller":"web/handlers.go:171","method":"POST","url":"/api/v4/channels/members/me/view","request_id":"gcr9bcaj43f3xy8a5e9ea146dw","status_code":"200"}
{"timestamp":"2023-03-16 03:46:09.028 Z","level":"debug","msg":"Received HTTP request","caller":"web/handlers.go:171","method":"POST","url":"/api/v4/channels/members/me/view","request_id":"athgng7ie3nnjmrm1wa6nni7qo","status_code":"200"}
{"timestamp":"2023-03-16 03:46:09.353 Z","level":"debug","msg":"Received HTTP request","caller":"app/plugin_api.go:970","plugin_id":"playbooks","request_id":"euephgzeyin9bm4y6kdds5cp6r","user_agent":"Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:109.0) Gecko/20100101 Firefox/110.0","method":"GET","url":"/api/v0/actions/channels/s73e9ocmzbfa7p4uuarzohtnwo?trigger_type=new_member_joins","user_id":"y8mxmju4wbbtbmep15p1arjg7c","plugin_caller":"github.com/mattermost/mattermost-plugin-playbooks/server/api/logger.go:44"}
{"timestamp":"2023-03-16 03:46:09.364 Z","level":"debug","msg":"Handled HTTP request","caller":"app/plugin_api.go:970","plugin_id":"playbooks","time":"11","method":"GET","url":"/api/v0/actions/channels/s73e9ocmzbfa7p4uuarzohtnwo?trigger_type=new_member_joins","user_id":"y8mxmju4wbbtbmep15p1arjg7c","request_id":"euephgzeyin9bm4y6kdds5cp6r","user_agent":"Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:109.0) Gecko/20100101 Firefox/110.0","status":"200","plugin_caller":"github.com/mattermost/mattermost-plugin-playbooks/server/api/logger.go:57"}
{"timestamp":"2023-03-16 03:46:09.367 Z","level":"debug","msg":"Received HTTP request","caller":"web/handlers.go:171","method":"POST","url":"/api/v4/files","request_id":"m3ai5y9ux7bf7ykqfe3qcz39pe","status_code":"201"}
{"timestamp":"2023-03-16 03:46:09.383 Z","level":"error","msg":"Failed to extract file content","caller":"app/file.go:704","error":"failed to save the extracted file content: failed to update FileInfo content with id=6oj3p6k5kbfrzqkx97t3oekd3h: pq: invalid byte sequence for encoding \"UTF8\": 0xa0","fileInfoId":"6oj3p6k5kbfrzqkx97t3oekd3h"}
{"timestamp":"2023-03-16 03:46:10.210 Z","level":"debug","msg":"Received HTTP request","caller":"web/handlers.go:171","method":"DELETE","url":"/api/v4/users/me/channels/s73e9ocmzbfa7p4uuarzohtnwo/drafts","request_id":"un9fhkp3nbrz786qarzjuk9m8r","status_code":"200"}
{"timestamp":"2023-03-16 03:46:10.234 Z","level":"debug","msg":"Received HTTP request","caller":"web/handlers.go:171","method":"POST","url":"/api/v4/posts","request_id":"p48dqd7o9jdh9fm3tp5omkkusy","status_code":"201"}
{"timestamp":"2023-03-16 03:46:10.250 Z","level":"debug","msg":"Worker received a new candidate job.","caller":"resend_invitation_email/worker.go:67","worker":"resend_invitation_email"}
{"timestamp":"2023-03-16 03:46:10.324 Z","level":"debug","msg":"Received HTTP request","caller":"app/plugin_api.go:970","plugin_id":"playbooks","url":"/api/v0/actions/channels/s73e9ocmzbfa7p4uuarzohtnwo?trigger_type=new_member_joins","user_id":"y8mxmju4wbbtbmep15p1arjg7c","request_id":"j6u89fn3nty5ic1q4x86hxt7ya","user_agent":"Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:109.0) Gecko/20100101 Firefox/110.0","method":"GET","plugin_caller":"github.com/mattermost/mattermost-plugin-playbooks/server/api/logger.go:44"}
{"timestamp":"2023-03-16 03:46:10.336 Z","level":"debug","msg":"Handled HTTP request","caller":"app/plugin_api.go:970","plugin_id":"playbooks","time":"13","status":"200","request_id":"j6u89fn3nty5ic1q4x86hxt7ya","user_agent":"Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:109.0) Gecko/20100101 Firefox/110.0","method":"GET","url":"/api/v0/actions/channels/s73e9ocmzbfa7p4uuarzohtnwo?trigger_type=new_member_joins","user_id":"y8mxmju4wbbtbmep15p1arjg7c","plugin_caller":"github.com/mattermost/mattermost-plugin-playbooks/server/api/logger.go:57"}
{"timestamp":"2023-03-16 03:46:12.311 Z","level":"debug","msg":"Feature requires an upgrade to Enterprise Edition.","caller":"web/context.go:113","path":"/api/v4/trial-license/prev","request_id":"7tfinohi9ffitj1ntmkjo5i8pc","ip_addr":"172.19.0.1","user_id":"y8mxmju4wbbtbmep15p1arjg7c","method":"GET","err_where":"getPrevTrialLicense","http_code":403,"error":"getPrevTrialLicense: Feature requires an upgrade to Enterprise Edition."}
{"timestamp":"2023-03-16 03:46:12.311 Z","level":"debug","msg":"Feature requires an upgrade to Enterprise Edition.","caller":"web/context.go:113","path":"/api/v4/trial-license/prev","request_id":"smo4h1xu7t8pjrb5q3kckyoc7h","ip_addr":"172.19.0.1","user_id":"y8mxmju4wbbtbmep15p1arjg7c","method":"GET","err_where":"getPrevTrialLicense","http_code":403,"error":"getPrevTrialLicense: Feature requires an upgrade to Enterprise Edition."}
{"timestamp":"2023-03-16 03:46:12.311 Z","level":"debug","msg":"Received HTTP request","caller":"web/handlers.go:171","method":"GET","url":"/api/v4/trial-license/prev","request_id":"7tfinohi9ffitj1ntmkjo5i8pc","status_code":"403"}
{"timestamp":"2023-03-16 03:46:12.311 Z","level":"debug","msg":"Received HTTP request","caller":"web/handlers.go:171","method":"GET","url":"/api/v4/trial-license/prev","request_id":"smo4h1xu7t8pjrb5q3kckyoc7h","status_code":"403"}
{"timestamp":"2023-03-16 03:46:13.574 Z","level":"debug","msg":"websocket.NextReader: client side closed socket","caller":"platform/web_conn.go:826","user_id":"y8mxmju4wbbtbmep15p1arjg7c"}
{"timestamp":"2023-03-16 03:46:13.574 Z","level":"debug","msg":"Received HTTP request","caller":"web/handlers.go:171","method":"GET","url":"/api/v4/websocket","request_id":"ay7c7a5debrmmxpmbzmb4fbrue"}
{"timestamp":"2023-03-16 03:46:13.584 Z","level":"debug","msg":"Feature requires an upgrade to Enterprise Edition.","caller":"web/context.go:113","path":"/api/v4/trial-license/prev","request_id":"6smihos6fbgk5ro74dw6r65q9e","ip_addr":"172.19.0.1","user_id":"y8mxmju4wbbtbmep15p1arjg7c","method":"GET","err_where":"getPrevTrialLicense","http_code":403,"error":"getPrevTrialLicense: Feature requires an upgrade to Enterprise Edition."}
{"timestamp":"2023-03-16 03:46:13.584 Z","level":"debug","msg":"Received HTTP request","caller":"web/handlers.go:171","method":"GET","url":"/api/v4/trial-license/prev","request_id":"6smihos6fbgk5ro74dw6r65q9e","status_code":"403"}
{"timestamp":"2023-03-16 03:46:13.585 Z","level":"debug","msg":"Received HTTP request","caller":"web/handlers.go:171","method":"GET","url":"/api/v4/plugins","request_id":"nuxiqccon3ytfkzyxsm57hxc9a","status_code":"200"}
{"timestamp":"2023-03-16 03:46:13.586 Z","level":"debug","msg":"Received HTTP request","caller":"web/handlers.go:171","method":"GET","url":"/api/v4/users/stats/filtered","request_id":"xn74u1ytbt8fzq6775fu6ey7te","status_code":"200"}
{"timestamp":"2023-03-16 03:46:13.590 Z","level":"debug","msg":"Received HTTP request","caller":"web/handlers.go:171","method":"GET","url":"/api/v4/analytics/old","request_id":"4873sgjq8bnkpdcmwssir6hdzw","status_code":"200"}
{"timestamp":"2023-03-16 03:46:13.598 Z","level":"debug","msg":"Received HTTP request","caller":"web/handlers.go:171","method":"GET","url":"/api/v4/config/environment","request_id":"b3159w9tybnaxgbqjxjyyege3e","status_code":"200"}
{"timestamp":"2023-03-16 03:46:13.598 Z","level":"debug","msg":"Received HTTP request","caller":"web/handlers.go:171","method":"GET","url":"/api/v4/config","request_id":"74y53zsj8tbsfy86xz6xd948sc","status_code":"200"}
{"timestamp":"2023-03-16 03:46:13.598 Z","level":"debug","msg":"Received HTTP request","caller":"web/handlers.go:171","method":"GET","url":"/api/v4/license/client","request_id":"h19gtz9e5fnmpfsgbkw9h9xzjw","status_code":"200"}
{"timestamp":"2023-03-16 03:46:13.598 Z","level":"debug","msg":"Received HTTP request","caller":"web/handlers.go:171","method":"GET","url":"/api/v4/upgrade_to_enterprise/status","request_id":"h6n7tooa5b8cjbuwkip8dgbiiw","status_code":"200"}
{"timestamp":"2023-03-16 03:46:13.599 Z","level":"debug","msg":"Received HTTP request","caller":"web/handlers.go:171","method":"GET","url":"/api/v4/users/stats/filtered","request_id":"bfqmbuh6rpgw3eww7etzhwz9dh","status_code":"200"}
{"timestamp":"2023-03-16 03:46:13.715 Z","level":"debug","msg":"inactive connection found for webconn, reusing","caller":"app/plugin_api.go:970","plugin_id":"focalboard","webConnID":"dys3ejarkbf73cib9drswh98zc","userID":"y8mxmju4wbbtbmep15p1arjg7c"}
This log does not contain any errors with regards to the S3 connection.
Are you sure that you enabled debug logging for S3 as well as the system and then tried to upload a file?
The only error I can see here is this one:
{"timestamp":"2023-03-16 03:46:09.383 Z","level":"error","msg":"Failed to extract file content","caller":"app/file.go:704","error":"failed to save the extracted file content: failed to update FileInfo content with id=6oj3p6k5kbfrzqkx97t3oekd3h: pq: invalid byte sequence for encoding \"UTF8\": 0xa0","fileInfoId":"6oj3p6k5kbfrzqkx97t3oekd3h"}
It might be related to your problem, but in that case, you would also be unable to store files locally and not on S3 but you said that that’s working, right? So you do only have problems with uploading files to S3, the local file storage works for you?
Also with regards to the webhook, please create a new topic here so we can work on those things in parallel if you like, let’s try to stick with one issue per topic.
Would it be possible for you to create a new, empty bucket in your Backblaze setup for me and send me a token for that via DM here so I can try to set it up on my end and see what needs to be done to make it work? Or is there a way to create a free bucket there for me?
I have not checked the storage locally. Will check it when I have time.
I sent you a DM. Thanks for the help @agriesser
Thanks, got it - will check that latest on Sunday.
I’m also trying to connect Mattermost to Backblaze B2 via S3 compatibility. I can connect as shown by @semighoti above. When uploading a file, the browser version hangs at “processing” and the mobile app says “stream closed.” A debug-level log shows an error:
error [2023-04-01 23:51:35.231 Z] Unable to write the file. caller="web/context.go:117" path=/api/v4/files request_id=*REDACTED* ip_addr=*REDACTED* user_id=*REDACTED* method=POST err_where=WriteFile http_code=500 error="WriteFile: Unable to write the file., unable write the data in the file path/to/file/Screenshot 2023-03-31 at 9.34.26 AM.png: Backblaze does not support the 'x-amz-checksum-algorithm' header for this API call."
Backblaze states: (S3-Compatible API)
Unsupported Features
The Backblaze S3 Compatible API does not currently support the following features:
- ACLs (see above)
- IAM roles
- Object Tagging
- Bucket Logging
- Website configuration
- Lifecycle Rules (supported by the Backblaze B2 Native API)
- Requests which include the
x-amz-sdk-checksum-algorithmHTTP header (these will be rejected with HTTP status 400)
I wonder if that is the cause of the issue above?
And - is there any way around this?
Looking around some more…I wonder if it is related to this, where minio adopted the AWS S3 checksum features from 2022:
Hi @YSHcPSbPM
Did you firgure out the error?
Hi, I have the exact same problem here, is there anyone already solved this?
Thanks