Did they nerf Opus 5.5? LiveNerf baseline established: Day 10

Last week I made LiveNerf, a project to independently track Opus 5.5’s performance day to day and see whether there’s actually evidence of models getting “nerfed” after release. If interested in tracking the performance of Opus 5.5, here’s the repo: github.com/ninjahawk/livenerf So far we’ve not been able to know conclusively one way or another, today that begins to change. The 10-day baseline is now complete. Tomorrow we begin collecting data up to date 30 after which we should be able to compare to the baseline to measure sustained model performance. I’m very grateful to everyone, I started measuring this to answer a question for myself about a week ago, I really didn’t expect it to get as much support as it has. I expected maybe some people tune in. Hopefully we’re able to get some answers soon enough. I’m a strong believer in transparency and if AI companies refuse it, we should build tools to measure it.

添加评论
点赞收藏
点踩分享查看原文
评论
?
参与讨论