A month of one rented GPU costs what 21 million API chats do

A month of a rented B200 costs $4,365. At DeepInfra's price for gpt-oss-120b the same money buys about 21 million chats, and in SemiAnalysis's vLLM benchmark the GPU only turns out that many with about 96 requests running at once, day and night. Where the lines cross for small, mid-size and large open models, and the reasons to self-host anyway.

添加评论
点赞收藏
点踩分享查看原文
评论
?
参与讨论