Towards Physics of Multimodal Pretraining