As part of our ongoing efforts to enhance and streamline the Mattermost platform, we’re announcing an upcoming deprecation in Mattermost v11.
What’s changing?
Mattermost v11 will retire experimental Bleve Search functionality as it no longer aligns with our vision for streamlined search solutions.
Why are we making this change?
Alignment with Platform Goals: We strongly recommend enterprise customers enable Elasticsearch support, while database-only search provides efficient performance tailored to non-commercial customer use cases.
Resource Focus: Deprecating Bleve search allows us to reallocate resources to priorities that provide more stability and better support for the majority of users.
Addressing Stability Issues: Bleve search has presented recurring stability challenges for users, and upstream maintainers have faced difficulty addressing these concerns.
What do you need to do?
If you currently use Bleve Search, we recommend transitioning to database-only search by ensuring the configuration setting Disable database search is disabled.
For teams with advanced search requirements, our Enterprise plan includes Elasticsearch for robust performance and scalability.
We understand changes like this can require adjustments and are here to support you as you make the shift. If you have questions or need assistance, please don’t hesitate to reach out.
What happens after upgrade to servers with Bleve enabled and database search disabled?
Search will not work until DisableDatabaseSearch is set to false.
We are a self-hosted Team Edition user, and we would like to express a significant concern regarding the planned retirement of the Experimental Bleve Search functionality in Mattermost v11.
We have greatly benefited from Bleve Search, as it has been essential for accurate keyword searching in multibyte languages like Japanese (and presumably Korean and Chinese).
Our concern is as follows:
Search Functionality Loss for Multibyte Users: If Bleve is disabled and the system reverts to the default database search (Core Search), we anticipate that the search functionality will become practically unusable for multibyte language content, as the standard relational database search (PostgreSQL/MySQL) does not perform the necessary morphological analysis (tokenization) for these languages. This will severely impact the user experience for a large segment of the global Mattermost user base.
Impact on Team Edition Users: We understand that the Enterprise Search options (Elasticsearch/OpenSearch) are available for Professional and Enterprise plans. However, the current Team Edition users, who utilize Mattermost for essential collaboration, will be left without a viable and officially supported solution for high-quality multibyte language search.
Suggestion for a Core Solution: To support all users, regardless of their plan, we are wondering if Mattermost v11 introduces a mechanism to allow for custom search preprocessing. Specifically:
Is there a new structural change that allows an admin to integrate a custom preprocessor (like a morphological analyzer for the database) before data is indexed or searched in the core database?
Alternatively, is Mattermost v11 implementing any built-in, out-of-the-box solution to ensure accurate search for multibyte languages without relying on Enterprise Search?
We truly appreciate Mattermost’s commitment to self-hosted users. Since Bleve has been functioning well for our non-English content, we sincerely hope that Mattermost provides a clear, sustainable path forward for multibyte language users in the free/Team Edition tier after the v11 change.
Thank you for your time and any insights you can provide on this critical topic.
Hi @DaisukeMatsuoh. Thank you for taking the time to share your concerns, and for adopting Mattermost!
I regret that we have no plans to reintroduce Bleve, but are indeed focusing our investment into Elasticsearch and OpenSearch to meet our enterprise environment needs.
Two possible paths forward you might consider:
Does your use case allow you to use Mattermost Entry? See Editions and Offerings - Mattermost documentation for more details, but in short this is an avenue for small, self-hosted teams to gain the full enterprise advanced suite of features for free – albeit with usage limitations.
Would you consider maintaining a fork that restores these changes for the open source community? Here’s the PR that removed Bleve if this is something you’d consider: https://github.com/mattermost/mattermost/pull/33430
You mentioned another option:
Is there a new structural change that allows an admin to integrate a custom preprocessor (like a morphological analyzer for the database) before data is indexed or searched in the core database?
I’m also interested in hearing more about this. Would you be willing to put together a proposal for how this might work in practice with PostgreSQL? I can’t make any guarantees we’d incorporate such a change, but I’d love to at least give it some consideration.
Without enabling Bleve search, I couldn’t find almost any post in Japanese.
It was an experimental feature for your team, but it is an essential feature for all the Japanese users.
Without the Bleve support, we have to make a fork (I don’t know how difficult it is at this time)
or switch to other solutions.
(Telegram also has this search problem, so I have a plan to forward all the messages to Mattermost, just for search. )
Bleve search is quite good for Japanese.
We love the Bleve search. Just works as expected.
In short, I used a very old Postgres extension called textsearch_ja that supports tokenizing Japanese sentences for PostgreSQL’s full-text search framework. It uses a morphological analyzer called MeCab.
Although the full-text search for Japanese works right now for me, this isn’t an ideal solution for several reasons:
textsearch_ja is an extremely old project (last updated in 2009(!)). While we can still download its source code from PgFoundry’s FTP archive, its official website is long gone. It’s great that an extension targeted at Postgres 9 still works for Postgres 17 with minimal modification, but obviously, we cannot rely on this in the long run.
You need to install many dependencies and manually build the extension using a Makefile (though I’ve provided a working Dockerfile in the link above).
The post-installation process is also nontrivial and unsafe. You need to manually issue CREATE/ALTER/DROP INDEX using psql following this guideline.
There are more recent and sophisticated full-text search solutions that support Japanese, such as PGroonga and ParadeDB’s pg_search with Lindera. However, these use their own SQL operators such as @@@ or &@. Unfortunately, Mattermost’s full-text query syntax appears to be hard-coded to @@here, preventing us from using these modern search solutions.
In my opinion, if Bleve isn’t going to be re-enabled, adding support for ParadeDB seems like a good approach. Is there an issue tracking this?
This is in the end a breaking change that they have no intent to go back on, which is unfortunate. Doubly so that they don’t seem to have considered that Japan has many large enterprises which they may be able to reach with a solution that - previously - was better on the market than most other options due to bleve being much more suited to these character sets.
We enjoy using Bleve a lot too (it was a huge gain to have earlier on mysql and made search actually usable). I must admit – I haven’t tried search without it on Postgres.
ElasticSearch and/or OpenSearch seem to be available only on the Enterprise tier which has no publicly available pricing, while the only tier (Professional) with pricing displayed ($10/month/user) has only the default-SQL/core-search? or is Bleve available on the Professional plan?