Tencent releases Hy3 open-source model that allegedly matches models up to five times its active size

Tencent has released Hy3, an open-source language model with 295 billion parameters built on a mixture-of-experts architecture. Only 21 billion parameters are active at any given time. Tencent says Hy3 matches models two to five times its size while cutting its hallucination rate in half to 5.4 percent.

The article Tencent releases Hy3 open-source model that allegedly matches models up to five times its active size appeared first on The Decoder.