Do you guys ever downgrade a workflow after you’ve got it working?
We have a few Claude workflows where the model choice was basically made while we were still building everything out but like at that point I wasn't really trying to optimize much I just wanted the outputs to be reliable and went with the model that was giving us the fewest troubles. Now some of those workflows have been running for a while and they're way more predictable. The prompts barely change, we know what the usual edge cases look like and some of the tasks are honestly pretty boring compared to what we originally built around. The thing is we almost never go back and question the model choice after something is working. It usually only comes up when a new model gets released or there's some other reason to touch the workflow. Part of me thinks we should periodically test the mature stuff on smaller models because we're using more than we need in a few places. The other part of me doesn't really want to mess with something that's been reliable for months to save a bit if it means introducing weird regressions we then have to chase down. Do you guys that run Claude in production revisit model choice on older workflows and if you do, what are you comparing before you're comfortable moving one down to a smaller model?