DINOv2 way worse than SigLIP in k-NN. Is this expected? [R]

Doing a bachelor thesis on fine-grained car classification (telling apart VW Golf generations from listing photos). Simple setup: frozen encoder → embeddings → weighted k-NN. On my small dataset (175 train / 132 test): SigLIP2 SO400M: ~92% CLIP ViT-L: ~59% DINOv2 Giant: ~41% I thought maybe it was a cosine vs euclidean thing, but my embeddings are L2-normalized so both give the same ranking. Tried both, DINOv2 stays at 41%. I get that SigLIP was trained contrastively so its space is basically built for cosi