Any downside of using MergerFS?

I have a Ugreen dxp4800 plus running debian 13 with a 4TB 990 Pro and 4x20TB WD Pro drives. My four hdds are XFS individual filesystems no redundancy. I went this route primarily because my data is 99% movies and shows that I have cross seeded on multiple trackers so if a drive fails I can always just redownload. Right now I have four different folders for movies and shows across the drives. Should I just merge the four filesystems with mergerfs? And is there any downside with that approach like more cpu usage or slower transfers?