Skip to content

Fix: accept scalar audio formats in Voxtral transcription requests#47045

Merged
eustlb merged 1 commit into
huggingface:mainfrom
Sunt-ing:25
Jul 7, 2026
Merged

Fix: accept scalar audio formats in Voxtral transcription requests#47045
eustlb merged 1 commit into
huggingface:mainfrom
Sunt-ing:25

Conversation

@Sunt-ing

@Sunt-ing Sunt-ing commented Jul 3, 2026

Copy link
Copy Markdown
Contributor

CI

What does this PR do?

VoxtralProcessor.apply_transcription_request() documents format as either a string or a list. The array-audio path did not normalize a scalar string, so format="wav" was treated as three per-sample formats and a single local audio array failed before the transcription request was encoded.

This broadcasts a scalar format to the audio batch length, keeps the existing length check for per-sample format lists, and raises a clear optional-dependency error before encoding array audio without soundfile.

Repro and output
import numpy as np
from transformers import AutoProcessor


model_id = "mistralai/Voxtral-Mini-3B-2507"
processor = AutoProcessor.from_pretrained(model_id)
audio = np.zeros(16000, dtype=np.float32)

for fmt in ["wav", ["wav"]]:
    print("CASE", fmt)
    try:
        out = processor.apply_transcription_request(
            audio=audio,
            model_id=model_id,
            format=fmt,
            sampling_rate=16000,
        )
        print("OK", sorted(out.keys()), out["input_features"].shape, out["input_ids"].shape)
    except Exception as exc:
        print("ERR", type(exc).__name__, str(exc))

Before this PR:

CASE wav
ERR ValueError When passed as a list of audio, the length (1) must match the number of format (3)
CASE ['wav']
OK ['attention_mask', 'input_features', 'input_ids'] torch.Size([1, 128, 3000]) torch.Size([1, 380])

After this PR:

CASE wav
OK ['attention_mask', 'input_features', 'input_ids'] torch.Size([1, 128, 3000]) torch.Size([1, 380])
CASE ['wav']
OK ['attention_mask', 'input_features', 'input_ids'] torch.Size([1, 128, 3000]) torch.Size([1, 380])
Tests
RUN_SLOW=1 python -m pytest -q -s tests/models/voxtral/test_modeling_voxtral.py::VoxtralForConditionalGenerationIntegrationTest::test_transcription_request_accepts_single_format_for_array_audio

tests/models/voxtral/test_modeling_voxtral.py::VoxtralForConditionalGenerationIntegrationTest::test_transcription_request_accepts_single_format_for_array_audio PASSED

======================= 1 passed, 11 warnings in 26.85s ========================

python -m ruff check src/transformers/models/voxtral/processing_voxtral.py tests/models/voxtral/test_modeling_voxtral.py
All checks passed!

git diff --check

Code Agent Policy

  • I confirm that this is not a pure code agent PR.

Before submitting

  • This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
  • Did you read the contributor guideline and the
    Pull Request checks?
  • Was this discussed/approved via a Github issue or the forum? Please add a link
    to it if that's the case.
  • Did you make sure to update the documentation with your changes according to the guidelines?
  • Did you write any new necessary tests?

Who can review?

cc @zucchini-nlp @Rocketknight1

@github-actions

github-actions Bot commented Jul 3, 2026

Copy link
Copy Markdown
Contributor

[For maintainers] Suggested jobs to run (before merge)

run-slow: voxtral

@github-actions

github-actions Bot commented Jul 3, 2026

Copy link
Copy Markdown
Contributor

CI recap

Dashboard: View test results in Grafana
Latest run: 28673130375
Result: success | Grafana metrics are not available yet.

@Rocketknight1

Copy link
Copy Markdown
Member

LGTM but cc @eustlb @ebezzam for audio to confirm!

@eustlb eustlb left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thanks a lot !! 🤗

)

if not is_soundfile_available():
raise ImportError("Please install `soundfile` to encode audio arrays with VoxtralProcessor.")

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To be replace with requires_backend when #47113 is merged (other fixed there is this get merged before)

@eustlb eustlb enabled auto-merge July 7, 2026 09:14
@eustlb eustlb added this pull request to the merge queue Jul 7, 2026
Merged via the queue into huggingface:main with commit b406ced Jul 7, 2026
37 checks passed
@HuggingFaceDocBuilderDev

Copy link
Copy Markdown

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants