Skip to content

feat(nebius): sync Token Factory models with automated catalog#2971

Open
amrrs wants to merge 2 commits into
anomalyco:devfrom
amrrs:nebius-token-factory-sync
Open

feat(nebius): sync Token Factory models with automated catalog#2971
amrrs wants to merge 2 commits into
anomalyco:devfrom
amrrs:nebius-token-factory-sync

Conversation

@amrrs

@amrrs amrrs commented Jul 1, 2026

Copy link
Copy Markdown

Adds a sync provider for Nebius Token Factory (verbose /v1/models endpoint) so model list, pricing, context length, and capability flags stay in sync automatically, resolving base_model references against canonical models/ metadata where available. Also switches the model sync automation from an hourly to a daily schedule.

Adds a sync provider for Nebius Token Factory (verbose /v1/models
endpoint) so model list, pricing, context length, and capability flags
stay in sync automatically, resolving base_model references against
canonical models/ metadata where available. Also switches the model
sync automation from an hourly to a daily schedule.

Co-authored-by: Cursor <cursoragent@cursor.com>
The previous commit inadvertently changed the global sync cadence from
hourly to daily. That change affects every provider, not just Nebius,
and is out of scope for the Nebius Token Factory sync PR. Reverting to
the original hourly schedule; a cadence change should be evaluated on
its own merits in a separate PR.

Co-authored-by: Cursor <cursoragent@cursor.com>
@rekram1-node

Copy link
Copy Markdown
Collaborator

Thanks for adding the Nebius Token Factory sync. A few things need addressing before this can go in:

1. Please resolve the merge conflicts

This PR is currently in a conflicting state against dev and can't be merged. Please rebase/merge dev and push.

2. context_length regression — the sync is producing invalid limits

The provider trusts model.context_length as authoritative (const context = model.context_length ?? existing?.limit?.context ?? 0), but the verbose=true endpoint appears to be returning a throttled/default 8000 for a lot of models. That clobbers correct values and yields entries where context < input, which is logically invalid:

model context (was → now) input
MiniMax-M2.5 204_800 (canonical) → 8_000 190_000
Qwen3.5-397B-A17B 262_144 → 8_000 250_000
Qwen3-Next-80B-A3B-Thinking 131_072 → 8_000 120_000
Qwen3-32B 131_072 → 40_960 120_000
Llama-3_1-Nemotron-Ultra-253B 128_000 → 8_000 120_000
Nemotron-3-Nano-Omni 65_536 → 8_000 60_000

All of your newly-created models (Kimi-K2.6, Nemotron-3-Ultra, Cosmos3, MiniCPM, etc.) also came through as context = 8_000, which reinforces that the API is handing back a limited value rather than the real context window. This looks like it may be tied to the API key/account tier used to generate the sync — worth double-checking. At minimum the sync should guard against overwriting a larger existing/canonical context with a smaller API value, and never emit context < input.

3. The "hourly → daily" schedule change was documented but not actually made

The PR description and sync.md both say the automation was switched from hourly to daily, but .github/workflows/sync-models.yml still has cron: "17 * * * *" (hourly). So the docs now contradict the workflow. Either make the cron change or revert the sync.md edit. Separately, this is a global change unrelated to adding Nebius — it'd be cleaner as its own PR rather than bundled here.

4. Inconsistent with the other sync providers

Every other provider that resolves canonical base models (baseten, huggingface, vercel, llmgateway) imports the shared resolveCanonicalBaseModel from openrouter.ts. This PR reimplements its own resolveCanonicalBaseModel, modelMetadataExists, and the modelMetadataFilesByProvider cache in nebius.ts. Your NEBIUS_ORG_TO_MODEL_PROVIDER map is essentially a re-keyed subset of openrouter's CANONICAL_PROVIDER_PREFIXES. Consider parameterizing the shared helper with a prefix→namespace map instead of duplicating the file-scan/cache logic, so this stays consistent with the rest of the sync providers.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants