I developed a 270 million parameter language model entirely from scratch as an independent research project

The model is built on a custom Transformer architecture featuring Rotary Positional Embeddings, RMSNorm, SwiGLU feed forward layers, grouped query attention, and an efficient autoregressive decoder optimized for local inference. submitted by /u/ConfectionAfter2366 [link] [comments]