Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
96 commits
Select commit Hold shift + click to select a range
68f55d0
Auto Xet to HTTP download fallback in from_pretrained; share Studio's…
danielhanchen Jun 24, 2026
18d277c
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Jun 24, 2026
f0c1d47
Address review: guard Studio shim import, inject prep on snapshot, pr…
danielhanchen Jun 24, 2026
c59d19d
Merge branch 'main' of https://github.com/unslothai/unsloth into hf-x…
danielhanchen Jun 24, 2026
ca77f43
Merge commit '18d277c90' into hf-xet-fallback-wire
danielhanchen Jun 24, 2026
4d4e45a
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Jun 24, 2026
0c10ffc
Prewarm PEFT adapter repos and respect checkpoint subfolders
danielhanchen Jun 24, 2026
e30d868
Merge remote-tracking branch 'origin/main' into hf-xet-fallback-wire
danielhanchen Jun 24, 2026
81c953b
Merge branch 'hf-xet-fallback-wire' of https://github.com/unslothai/u…
danielhanchen Jun 24, 2026
ec9c6e9
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Jun 24, 2026
7f6e137
Guard Studio HTTP prep and detect home-relative local paths (Gemini r…
danielhanchen Jun 24, 2026
d29a46b
Merge remote-tracking branch 'origin/main' into hf-xet-fallback-wire
danielhanchen Jun 25, 2026
eb6954b
Forward force_download through the prefetch (review 3)
danielhanchen Jun 25, 2026
2bc08b5
Degrade Studio gracefully when the shared zoo helper is absent (revie…
danielhanchen Jun 25, 2026
ed16544
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Jun 25, 2026
1fda803
Fix forced double-download, guard diffusion path, honor cancel in deg…
danielhanchen Jun 25, 2026
d321f98
Merge branch 'hf-xet-fallback-wire' of https://github.com/unslothai/u…
danielhanchen Jun 25, 2026
0387885
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Jun 25, 2026
e229d38
Diffusion prefetch ordering/cache_dir and degraded heartbeats (review…
danielhanchen Jun 25, 2026
5a95196
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Jun 25, 2026
4d02717
Load tokenizers/processors from the cache the prefetch warmed (review)
danielhanchen Jun 25, 2026
1caea30
Scope the prefetch .bin skip and forward cache_dir to fallback loads
danielhanchen Jun 25, 2026
9f6f604
Merge remote-tracking branch 'origin/main' into hf-xet-fallback-wire
danielhanchen Jun 25, 2026
a816c77
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Jun 25, 2026
fc88b76
Fix Studio shim CI: degrade when unsloth_zoo is absent, clear lint bl…
danielhanchen Jun 26, 2026
d896b73
Merge remote-tracking branch 'origin/main' into hf-xet-fallback-wire
danielhanchen Jun 26, 2026
872ca5e
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Jun 26, 2026
4e82427
Degrade the Studio shim on ImportError too, not just ModuleNotFoundError
danielhanchen Jun 26, 2026
e54ff08
Merge remote-tracking branch 'origin/main' into hf-xet-fallback-wire
danielhanchen Jun 26, 2026
27b32b5
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Jun 26, 2026
163bc81
Degrade/recover the Studio shim when unsloth_zoo GPU init fails
danielhanchen Jun 26, 2026
3953afb
Merge remote-tracking branch 'origin/main' into hf-xet-fallback-wire
danielhanchen Jun 26, 2026
9253c7b
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Jun 26, 2026
2313f55
Fix the GPU-init degradation test to intercept the parent import
danielhanchen Jun 26, 2026
a358828
Merge remote-tracking branch 'origin/main' into hf-xet-fallback-wire
danielhanchen Jun 26, 2026
9c601c6
Treat use_safetensors as a format allowlist in the prefetch (Codex P2)
danielhanchen Jun 26, 2026
ba2aa9f
Fix three prefetch/load consistency issues from review (Codex P2)
danielhanchen Jun 26, 2026
6a25b5d
Tighten prefetch heuristics from review (Codex P2)
danielhanchen Jun 26, 2026
480bfae
Protect adapter / TF / Flax / custom-cache_dir loads in the prefetch …
danielhanchen Jun 26, 2026
d0e4f3f
Warm adapter and root tokenizer files correctly in the prefetch (Code…
danielhanchen Jun 26, 2026
4c11447
Prefetch a distinct tokenizer repo; require the unsloth_zoo with the …
danielhanchen Jun 27, 2026
2f998a9
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Jun 27, 2026
8138bf7
Cover vLLM tokenizers, TF/Flax warmups, and remote-code assets in the…
danielhanchen Jun 27, 2026
ac44e2f
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Jun 27, 2026
4b45d62
Keep unsloth_zoo floor at 2026.6.7 until 2026.6.8 is published
danielhanchen Jun 27, 2026
6d01b7e
Validate before prefetching; warm weights when vLLM is unavailable; s…
danielhanchen Jun 27, 2026
9cee677
Merge remote-tracking branch 'origin/main' into hf-xet-fallback-wire
danielhanchen Jun 27, 2026
96aeae9
Merge remote-tracking branch 'origin/main' into hf-xet-fallback-wire
danielhanchen Jun 27, 2026
92800b9
Narrow the cache prewarm to what each load actually reads (adapter, r…
danielhanchen Jun 27, 2026
e3e139e
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Jun 27, 2026
aee9006
Merge remote-tracking branch 'origin/main' into hf-xet-fallback-wire
danielhanchen Jun 27, 2026
844c92b
Forward cache_dir to the VLM processor fallback; make the adapter and…
danielhanchen Jun 28, 2026
cafa5ab
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Jun 28, 2026
6a0bb93
Tighten prefetch variant detection, tokenizer warm coverage, and cach…
danielhanchen Jun 28, 2026
b8010f7
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Jun 28, 2026
c3d674b
Tighten prefetch format selection, tokenizer vocab coverage, and the …
danielhanchen Jun 28, 2026
be49ab1
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Jun 28, 2026
4710764
Warm any trust_remote_code module, not just the modeling_*.py convention
danielhanchen Jun 28, 2026
1c99378
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Jun 28, 2026
0b60a20
Scope the adapter format probe to root, and warm the requested GGUF file
danielhanchen Jun 28, 2026
58b872f
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Jun 28, 2026
290944f
Warm a subfolder GGUF at its real path and ignore nested TF/Flax weig…
danielhanchen Jun 28, 2026
affcc0a
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Jun 28, 2026
8e315aa
Trim prefetch / Studio shim comments to a succinct core
danielhanchen Jun 29, 2026
44eded4
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Jun 29, 2026
6702228
Pre-download the snapshot for FastSentenceTransformer loads too
danielhanchen Jun 29, 2026
33255b1
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Jun 29, 2026
880f688
Strengthen the FastSentenceTransformer prefetch wiring test
danielhanchen Jun 30, 2026
20737ca
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Jun 30, 2026
83a7a93
Route a custom cache_dir through the ST module and tokenizer-repair f…
danielhanchen Jun 30, 2026
a8696ab
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Jun 30, 2026
7cec30b
Merge remote-tracking branch 'origin/main' into hf-xet-fallback-wire
danielhanchen Jun 30, 2026
010bb50
Forward variant to the killable prewarm so the cache-skip gate can defer
danielhanchen Jul 1, 2026
c05734d
Merge origin/main into hf-xet-fallback-wire
danielhanchen Jul 1, 2026
cb8cf91
Adapt the Studio force_download test to the deduplicated shim
danielhanchen Jul 1, 2026
3203753
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Jul 1, 2026
7423f33
Close four SentenceTransformer prefetch gaps
danielhanchen Jul 1, 2026
83b60f2
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Jul 1, 2026
f1fe53a
Forward cache_folder to the fast-encoder SentenceTransformer load
danielhanchen Jul 1, 2026
43ffea7
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Jul 1, 2026
a5e67ba
Close three SentenceTransformer / prefetch scoping gaps
danielhanchen Jul 1, 2026
8998e9a
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Jul 1, 2026
2866838
Resolve SENTENCE_TRANSFORMERS_HOME for fallback module loads
danielhanchen Jul 1, 2026
636ff82
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Jul 1, 2026
76e0ced
Load ST fallback modules from the same revision as the weights
danielhanchen Jul 1, 2026
a5ca1bc
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Jul 1, 2026
ca1fd06
Tighten comments (comment-only, no logic change)
danielhanchen Jul 1, 2026
3f750f7
Tighten frontend comments (comment-only, no logic change)
danielhanchen Jul 1, 2026
a89ec0d
Warm classification loads and gate the vision prefetch on load-mode v…
danielhanchen Jul 1, 2026
4f5f8da
Point the fallback FastModel weight load at the warmed ST cache
danielhanchen Jul 1, 2026
e883f71
Close four prefetch cache/variant/tokenizer gaps that left an in-proc…
danielhanchen Jul 1, 2026
08ba451
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Jul 1, 2026
6d4ce26
Map explicit HF cache_dir onto cache_folder for the native SentenceTr…
danielhanchen Jul 2, 2026
7a6fc8d
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Jul 2, 2026
af2f079
Trim verbose comments in the HF prefetch wiring and Studio shim
danielhanchen Jul 3, 2026
9889390
Warm SentenceTransformer force_download in the killable child
danielhanchen Jul 3, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
560 changes: 271 additions & 289 deletions studio/backend/tests/test_hf_xet_fallback.py

Large diffs are not rendered by default.

44 changes: 13 additions & 31 deletions studio/backend/tests/test_model_update_robustness.py
Original file line number Diff line number Diff line change
Expand Up @@ -5,8 +5,8 @@
Covers:
* GGUF variant listing computes update_available from the already-fetched
sibling metadata instead of a second Hub call.
* hf_hub_download_with_xet_fallback(force_download=True) bypasses the
try_to_load_from_cache cache-first early-return.
* hf_hub_download_with_xet_fallback forwards force_download through the shim to the
shared unsloth_zoo helper (which owns the cache-first early-return and its bypass).

The cache "Update" action now runs through the download manager as a normal
managed download (so it shows in the Downloads panel with progress + cancel),
Expand Down Expand Up @@ -341,44 +341,26 @@ def test_cached_model_scan_keeps_local_safetensors_repo(monkeypatch, tmp_path):
# ── hf_hub_download_with_xet_fallback force_download bypass (X2/F2) ───


def test_force_download_bypasses_cache_first_early_return(monkeypatch):
"""force_download=True skips the try_to_load_from_cache early-return and
proceeds to the real download path; force_download=False returns the cached
path without ever attempting a download (X2/F2)."""
import huggingface_hub as hf
def test_force_download_is_forwarded_through_the_shim(monkeypatch):
"""The shim's contract is to forward force_download unchanged to the shared helper (which owns the
cache-first early-return and bypass). Verify both False and True reach it (X2/F2)."""
import utils.hf_xet_fallback as X

cached_path = "/cache/blob/cached.gguf"
seen = []

# Pretend the blob IS cached on disk (try_to_load_from_cache is imported
# inside the function from huggingface_hub, and os.path.exists must agree).
monkeypatch.setattr(hf, "try_to_load_from_cache", lambda *a, **k: cached_path, raising = False)
monkeypatch.setattr(X.os.path, "exists", lambda p: True, raising = False)
def fake_shared(repo_id, filename, token, **kwargs):
seen.append(kwargs.get("force_download"))
return "/downloaded/path"

attempts = []
monkeypatch.setattr(X, "_shared_hf_hub_download_with_xet_fallback", fake_shared, raising = True)

def fake_attempt(repo_id, filename, token, **kwargs):
attempts.append(
{"repo_id": repo_id, "filename": filename, "force": kwargs.get("force_download")}
)
return ("ok", "/freshly/downloaded/path")

monkeypatch.setattr(X, "_run_download_attempt", fake_attempt, raising = True)

# force_download=False: cache-first early-return, no download attempt.
out = X.hf_hub_download_with_xet_fallback(
X.hf_hub_download_with_xet_fallback(
"unsloth/repo", "model.gguf", token = None, force_download = False
)
assert out == cached_path
assert attempts == [] # never reached the real download

# force_download=True: bypass the early-return, run the real download.
out2 = X.hf_hub_download_with_xet_fallback(
X.hf_hub_download_with_xet_fallback(
"unsloth/repo", "model.gguf", token = None, force_download = True
)
assert out2 == "/freshly/downloaded/path"
assert len(attempts) == 1
assert attempts[0]["force"] is True
assert seen == [False, True] # the shim forwards force_download to the shared helper unchanged


# ── multi-revision GGUF blob comparison and update reclaim ──
Expand Down
Loading
Loading