I did some further debugging now.
Debian 11, freshly installed, apt install docker docker-compose will get you docker-compose version 1.25.0 and this version is too old.
The first working version is 1.26.0, from this version upwards, it will immediately work, so if you want to give it a try, grab the latest docker-compose version from there and try again:
wget -O /usr/local/bin/docker-compose https://github.com/docker/compose/releases/download/v2.9.0/docker-compose-linux-x86_64
chmod 755 /usr/local/bin/docker-compose
/usr/local/bin/docker-compose -f docker-compose.yml -f docker-compose.without-nginx.yml up -d
This should get you going, right?