Hi @gudbes,
I believe the only way to configure that is via Advanced Permissions (Enterprise feature): https://docs.mattermost.com/deployment/advanced-permissions.html.
Other than this, I believe all channel members are able to remove other channel members from both public and private channels. However, System Admins have the ability to re-join all public and private channels.