Are contrastive losses just cross entropy all along?

Yes. The contrastive losses are just the cross entropy over embedding similarity, and once you see that, it's obvious why these models overfit.