When Your AI Gateway Silently Swaps the Model, You Can't Trust the Output

If you run AI agents that do real work, ask this: when your configured model gets rejected mid-request, does your stack tell you, or does it answer with something weaker and let you believe nothing changed? Mine didn't tell me.

I run OpenClaw, an open source gateway that routes agents across model providers. I noticed responses that felt off. Not broken, just weaker than they should have been. When I dug in, the model I had configured, one of the gpt-5.6 tiers on the ChatGPT OAuth route, was being rejected by the provider at request time. The gateway caught the rejection and answered with a fallback model. There was no warning, and the status command still reported that the primary model was resolving fine.

Why a silent downgrade is worse than an outage

An outage is honest. Things stop, you notice, you fix it. A silent downgrade keeps every output arriving in the same format, produced by a model you didn't pick. For agents that draft research, summarize filings or watch systems, that means the quality drops while the evidence of the drop disappears. You keep trusting numbers and summaries you would have checked harder if you'd known which model wrote them.

There was a second layer. The model catalog itself was misleading. The gateway listed the full gpt-5.6 lineup on the OAuth route whether or not the account was entitled to it. Models passed validation when configured, then the provider rejected them at request time with a 400. So a model could pass setup and never answer a single request.

How I reported it

I wrote it up as two GitHub issues, #105445 and #106786, each with a reproduction and the exact failure path. A bug report that includes the steps and the observed output gives a maintainer something to act on; a complaint in a group chat doesn't.

Fixes tied to those reports are visible in the project's public pull requests. PR #108254 concerns classifying usage-limit rejections structurally and rotating to a sibling auth profile on the same model before falling back to a different model. PR #108683 concerns the discovery handshake reporting the real client version instead of a placeholder, which was part of why models validated and then bounced. Catalog changes moved toward account-backed discovery so the list reflects what an account can use. Loud surfacing when a fallback fires was still open when I wrote this. Read the linked threads for the current state rather than relying on my summary.

Checks any agent stack should pass

You can test your own gateway in about an hour:

1. Force a rejection. Configure a model your account can't use, or revoke a key in a test profile, and send one request.
2. Look for the answer's source. Does the response metadata or log name the model that actually answered? If it names the configured one, you can't tell fallbacks apart.
3. Check the status command against a live request. A status that says "resolving" without making a request only proves the config parses.
4. Compare the catalog to your entitlements. If the tool lists models your plan doesn't include, treat validation as untrusted until a real request succeeds.
5. Make fallback loud. A fallback that fires should log a warning, ideally with an alert, and optionally refuse to run for tasks that need a specific model.

For anything where output quality matters, I'd rather the call fail than succeed with a different model. A failed request costs you minutes. A quiet downgrade can sit in your work for weeks.

The takeaway

Trust in an AI stack depends less on how capable the model is and more on whether the plumbing tells the truth when something goes wrong. Audit what your gateway does when a provider says no, and when you find a gap in software you depend on, file it with a reproduction.