Unity vs. Floating Point

A tweet by @VehiclePhysics sparked my interest. It basically says:
For most math functions (Sqrt, Sin, Cos, Log, Pow…), prefer System.MathF over UnityEngine.Mathf. Unity’s Mathf casts to double, calls the double version, then converts back to float. System.MathF calls the float-native implementations directly. Less work, same result.
This advice is basically correct! But turns out, things are slightly more complicated.
Hidden double precision in Unity The advice above applies to all UnityEngine.