Skip to content

Add support for text to speech conversion - #31

Open
dkotter wants to merge 3 commits into
trunkfrom
feature/text-to-speech
Open

Add support for text to speech conversion#31
dkotter wants to merge 3 commits into
trunkfrom
feature/text-to-speech

Conversation

@dkotter

@dkotter dkotter commented Jul 21, 2026

Copy link
Copy Markdown
Collaborator

What?

Adds support for text to speech conversion

Why?

We currently don't support text to speech conversion in this provider, even though that is supported upstream in the PHP AI Client. By adding this support, it allows others to build out text to speech systems using this provider plugin.

How?

  • Introduce a new GoogleTextToSpeechConversionModel that handles all requests to convert text to speech
  • Ensure this model is loaded when a TTS generation request is made
  • Ensure we properly map model options and capabilities when determining what models support TTS

Use of AI Tools

AI assistance: Yes
Tool(s): Claude Code
Model(s): Opus 4.8
Used for: Putting together a plan and executing on that plan. Plan reviewed and modified by me and all code was reviewed and tested by me

Testing Instructions

Hard to test on it's own as this plugin provides functionality but doesn't actually do anything with that. Easiest approach is the following:

  1. Checkout the WordPress AI plugin from this PR
  2. Download this PR, activate and configure the Google Provider
  3. In the AI settings page, turn on Text to Speech
  4. Go to a post and find the Text to Speech panel in the sidebar
  5. Click on the Generate Audio button and ensure it works as expected

Changelog Entry

Added - Support for text to speech conversion

@dkotter dkotter added this to the 1.2.0 milestone Jul 21, 2026
@dkotter dkotter self-assigned this Jul 21, 2026
@dkotter
dkotter requested a review from a team July 21, 2026 21:05
@github-actions

github-actions Bot commented Jul 21, 2026

Copy link
Copy Markdown

The following accounts have interacted with this PR and/or linked issues. I will continue to update these lists as activity occurs. You can also manually ask me to refresh this list by adding the props-bot label.

If you're merging code through a pull request on GitHub, copy and paste the following into the bottom of the merge commit message.

Co-authored-by: dkotter <dkotter@git.wordpress.org>
Co-authored-by: superdav42 <superdav42@git.wordpress.org>

To understand the WordPress project's expectations around crediting contributors, please review the Contributor Attribution page in the Core Handbook.

@superdav42 superdav42 added the status:in-review PR open, awaiting review/merge label Jul 22, 2026
@superdav42 superdav42 self-assigned this Jul 22, 2026

@superdav42 superdav42 left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Requesting changes for one end-to-end compatibility blocker.

The provider advertises WAV-only output here and returns every TTS result as audio/wav (src/Models/GoogleTextToSpeechConversionModel.php:249-252). However, the WordPress/ai PR #888 named in this PR's testing instructions splits content longer than 4,000 characters and explicitly rejects every multi-chunk job unless the first chunk is exactly audio/mpeg (includes/Experiments/Text_To_Speech/Job_Manager.php:278-280).

As a result, a short post can pass the manual test while a longer post fails immediately after generating its first chunk with “Combining audio chunks requires MP3 output, which the provider did not return.” This means the stated integration does not work for the normal long-content path.

Please coordinate one of these fixes before merge:

  • add WAV-aware combining in WordPress/ai #888 by extracting each WAV's PCM data and rebuilding a single RIFF/WAVE container; or
  • make this provider return genuine audio/mpeg output.

Simply concatenating complete WAV files is not valid. Please also cover at least a two-chunk Google TTS job in the integration test/manual verification.

Verification performed at head 42877994d3416634412b036623f437acf16d7abb:

  • PHPCS passes on all three changed files.
  • PHPStan passes on all three changed files.
  • The pinned wordpress/php-ai-client 0.4.2 symbols used by the implementation are present.
  • WAV header generation passes a targeted runtime smoke test.
  • The repository-wide lint command remains red only for pre-existing violations in unchanged GoogleTextAndImageGenerationModel.php.

@dkotter

dkotter commented Jul 22, 2026

Copy link
Copy Markdown
Collaborator Author

Requesting changes for one end-to-end compatibility blocker.

The provider advertises WAV-only output here and returns every TTS result as audio/wav (src/Models/GoogleTextToSpeechConversionModel.php:249-252). However, the WordPress/ai PR #888 named in this PR's testing instructions splits content longer than 4,000 characters and explicitly rejects every multi-chunk job unless the first chunk is exactly audio/mpeg (includes/Experiments/Text_To_Speech/Job_Manager.php:278-280).

As a result, a short post can pass the manual test while a longer post fails immediately after generating its first chunk with “Combining audio chunks requires MP3 output, which the provider did not return.” This means the stated integration does not work for the normal long-content path.

Please coordinate one of these fixes before merge:

  • add WAV-aware combining in WordPress/ai #888 by extracting each WAV's PCM data and rebuilding a single RIFF/WAVE container; or
  • make this provider return genuine audio/mpeg output.

Simply concatenating complete WAV files is not valid. Please also cover at least a two-chunk Google TTS job in the integration test/manual verification.

While I appreciate the feedback here, unless I'm missing something it seems like this feedback only applies to the PR on the WordPress AI plugin side. If so, ideally that feedback and discussion should happen there and not here.

At the moment, I've purposely only supported MP3 formats for chunking on that PR, though could try and extend that for WAV as well, but again, that's not something that would require changes here.

  • make this provider return genuine audio/mpeg output.

This is the bit that would apply here and is what I wanted to originally support but my understanding is there's not a great way to convert from WAV to MPEG without having access to a valid audio encoder, which may not be available in all PHP environments. So for now, I've left this was WAV only.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

status:in-review PR open, awaiting review/merge

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants