Comparing Fable and 10 other LLMs on refactoring a LangGraph god node

!!! info "Other languages" Эта статья также доступна на русском: Гибель богов.

!!! abstract "Materials & raw data" All 11 model proposals, the cross-reviews, the theses runs, and the ranking script are published here: Materials & reproduce this experiment.

This is a detailed write-up of one experiment. I took a god node from a real LangGraph agent and asked 5 American and 6 Chinese models first to propose how to untangle it, then to evaluate each other's proposals. After that, I tried three different ways to figure out which of them to trust on the matter.