Studio: add OrcaRouter as an external chat provider#6818
Studio: add OrcaRouter as an external chat provider#6818jinhaosong-source wants to merge 1 commit into
Conversation
OrcaRouter is an OpenAI-compatible meta-router (one key for models across major providers), wired the same way as OpenRouter: a curated registry entry that flows through the generic /chat/completions passthrough, with HTTP-Referer / X-Title attribution headers. Capability table exposes only the OpenAI-compat base sampling knobs because OrcaRouter forwards params verbatim to heterogeneous upstreams. Includes unit tests pinning the registry contract and the outbound wire shape.
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request integrates OrcaRouter, an OpenAI-compatible meta-router, as a new external inference provider. The changes include adding OrcaRouter to the backend provider registry, implementing comprehensive unit tests, and updating the frontend to support its logo, vision capabilities, namespaced model IDs, and specific parameter constraints. The review feedback suggests a minor improvement in the test suite to use asyncio.run instead of manually creating an event loop, which prevents potential resource leaks.
Important
The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.
| def _drive(coro): | ||
| return asyncio.new_event_loop().run_until_complete(coro) |
There was a problem hiding this comment.
Using asyncio.new_event_loop().run_until_complete(coro) leaves the newly created event loop unclosed, which can leak resources and cause warnings in the test suite. It is cleaner and safer to use asyncio.run(coro), which automatically manages the event loop lifecycle.
| def _drive(coro): | |
| return asyncio.new_event_loop().run_until_complete(coro) | |
| def _drive(coro): | |
| return asyncio.run(coro) |
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: de7bf9df84
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
| "openai/gpt-5.5", | ||
| "google/gemini-3.5-flash", | ||
| "anthropic/claude-opus-4.8", | ||
| "grok/grok-4.3", | ||
| "deepseek/deepseek-v4-pro", |
There was a problem hiding this comment.
When a user selects one of the advertised reasoning models here (openai/gpt-5.5, anthropic/claude-opus-4.8, or deepseek/deepseek-v4-pro), the OrcaRouter request takes the generic OpenAI-compatible path, whose body always includes temperature, top_p, and presence_penalty; this registry entry does not set body_omit or add per-model stripping. OrcaRouter forwards unsupported sampling fields to upstreams, so these reasoning models can 400 even with untouched UI defaults. Please strip the unsupported fields for OrcaRouter reasoning/model IDs before adding them to the curated picker.
Useful? React with 👍 / 👎.
1 participant
What
Adds OrcaRouter to Unsloth Studio's external provider registry. OrcaRouter is an OpenAI-compatible meta-router: one API key routes chat requests to models across major providers (OpenAI, Google, Anthropic, xAI, DeepSeek, MiniMax, Qwen), with gateway-level agent security controls and a rule-based routing DSL, plus an
orcarouter/autoid that picks an upstream per request.Disclosure: I'm an engineer on the OrcaRouter team.
How
Wired the same way as the existing OpenRouter entry, with no changes to the streaming path:
PROVIDER_REGISTRYentry (model_list_mode: "curated", Bearer auth,HTTP-Referer/X-Titleattribution headers). Requests flow through the existing generic OpenAI-compatible/chat/completionspassthrough.autorouter plus one flagship per upstream), logo, and namespaced-id handling ingetExternalMaxOutputTokens(shared with OpenRouter).OPENAI_COMPAT_BASEinstead ofALL_SUPPORTED, because OrcaRouter forwards sampling params verbatim to heterogeneous upstreams (some reject params they do not support), so Studio only surfaces the OpenAI-compat base knobs.Tests
studio/backend/tests/test_orcarouter_provider.py: pins the registry contract and asserts the outbound wire shape (URL, Bearer auth, attribution headers, verbatim namespaced model id,stream: true) viahttpx.MockTransport. Runs offline.test_providers_api.py(registry count, vision param set, provider config for the live suite).Verification
test_external_provider_usage_chunk.py,test_providers_api.py,test_external_provider_proxy_env.py); frontendnpx tsc -bclean./v1/modelsreturns all curated ids, invalid keys get 401 (connection test works), streamed chat throughorcarouter/autoand each curated flagship, vision (image input) throughautoandgoogle/gemini-3.5-flash, and OpenAI-shape function calling all succeed.