Model routing for LLM agents and applications.

The official diagram separates the host or gateway from model selection and upstream calls. Official figure source
Architecture and data flow
- InputPrepare an app request
observation → summary requestThe app turns a video observation into a text task.
- ProcessSelect a model
request + model pool + routing strategySwitchyard helps the host select the model for the request.
- OutputCall the endpoint
model selection → responseThe host calls the selected provider and the app uses its response.
Switchyard separates model choice from the application’s task logic. A host gives the routing library a request, a pool of available models and a strategy. Task routing can ask a model to classify the request, and stage routing can take recent tool activity into account. The host handles requested upstream calls and consumes the routing outcome. Gateway integrations can translate OpenAI and Anthropic request, response and streaming formats.
Where does a request go after video analysis?
Suppose a video service detects a maintenance event at 00:12. The app turns that observation into a text summary request. The host uses Switchyard with the request, available models, and routing strategy to choose a model. The app uses the selected endpoint's text to finish the report. Record the clip ID, selected model, response time, and report completion together to see which stage needs attention. Official Switchyard repository
Physical AI example
For an inspection assistant, keep image and video analysis in a vision-capable service such as VSS. Route its downstream text summarization and report drafting across a model pool. Compare a fixed single-model baseline with the routed setup on the same clips and prompts.
What to verify
Measure cost and latency per successfully completed report. Verify tool calls, streaming, provider error handling and accepted modalities end to end. The standalone server is documented for demos and local evaluation. Check the gateway integration and its version before deployment.
License and availability
Apache-2.0 repository code. Model weights, external services and dependencies can have separate terms.
