Bleve trouble in .26: Shell commands for `mattermost` broken in 5.26 after enabling Bleve?

Troubleshooting
1

After upgrading to 5.26 I enabled Bleve and set it to index. After this it seems as though shell commands via mattermost no longer function.

For example, if I execute a simple:

$ /opt/mattermost/bin/mattermost channel list TEAM_NAME

with Bleve enabled, I just get:

{"level":"info","msg":"Server is initializing..."}
2020-08-16T20:30:41.846Z	info	app/web_hub.go:83	Starting websocket hubs	{"number_of_hubs": 2}
2020-08-16T20:30:41.847Z	info	bleveengine/bleve.go:157	EXPERIMENTAL: Starting Bleve
^C

After I disable Bleve again within the System Console shell commands work again just fine.

I also believe Bleve is not actually indexing when turned on because it stays at 0% indexed on a relatively small Mattermost installation after even two hours of it just sitting there.

2

Could this doc help https://docs.mattermost.com/administration/command-line-tools.html#executing-a-command-hangs-and-doesn-t-complete?

3

That’s really interesting @amy.blais - thanks for pointing that out. Hmm. Well, in addition to this, I’m getting the following with the Bleve hang:
image

4

Well good news, but such an insanely long time:
image

5

Still going:
image

I suppose it’s just a long waiting game. Clearly this somehow this needs to give better feedback.

6

Still going…
image

7

Known issue regarding starting such a command:

  • Enabling Bleve search engine makes the Command Line Interface (CLI) mutually exclusive with the running server. This issue does not apply when using mmctl Command Line Tool.
8

You can use the mmctl local mode to run commands locally with Bleve enabled.

As for the indexing progress, is there any information in the server logs related to Bleve or any error log?

9

The problem when using mmctl is that it appears to be feature incomplete. For example, moving channels between teams seems impossible at present.

It finally finished after 2191 minutes:
image

10

Maybe not, it appears to still be going:
image

11

Regarding mmctl, @justinegeffen has updated the docs so this doc update is inbound it appears:
https://mattermost.github.io/docs/justinegeffen/administration/mmctl-cli-tool.html#mmctl-channel-move

12

Finally finished:
image

@amy.blais we clearly need some better status updates due to these potentially insanely long run times…