Offering custom fonts shouldn’t greatly increase load time - only the chosen font need be loaded. And in general the negative UX outcomes of locking down the font would, imo, massively outweigh the benefits of the freedom to choose fonts.
Speaking to point 1, I think that if viewed as a technical restriction (users must be able to choose their own fonts), it won’t be hard, certainly not impossible, to solve the load time problem without restricting to a single font.
Speaking to point 2, It’s not hard to find fonts that support many languages. I would support a UX change that encouraged people to choose those fonts, rather than forcing them.