Running GPT-2 in WebGL with Classic GPU Programming

A few weeks back, I implemented GPT-2 using WebGL and shaders (Github Repo) which made the front page of Hacker News. Here is a short write-up over what I learned about old-school general-purpose GPU programming over the course of this project! Above is a gif of the final demo, which you can run locally via the github repo above.

This article appeared on Hacker News. Link to the discussion here.

The Origins of General-Purpose GPU Programming#

In the early 2000s, NVIDIA introduced programmable shaders with the GeForce 3 (2001) and GeForce FX (2003). Instead of being limited to predetermined transformations and effects of earlier GPUs, developers were now given unprecedented control over the rendering pipeline, enabling much more sophisticated visual effects. These programmable shaders laid the foundation for modern GPU computing.