1
1
Fork
You've already forked zop
0

feat: speed up TTS with chunking, markdown sanitization, and GPU support #32

Merged
peterwwillis merged 2 commits from feature/speed-up-tts into main 2026年03月21日 23:37:56 +01:00
peterwwillis commented 2026年03月21日 22:22:46 +01:00 (Migrated from github.com)
Copy link

This PR implements improvements to the TTS system as described in issue #28.

Key changes:

  • Text Chunking: Implemented a sentence-aware text chunker that splits long texts into smaller segments (~100 tokens/400 chars) for pipelined audio generation. This prevents crashes on large texts and improves perceived performance.
  • Markdown Sanitization: Added a markdown sanitizer to strip formatting characters (bold, italics, headers, links, code blocks) before text is sent to the TTS engine.
  • Hardware Acceleration: Added a 'provider' configuration option to enable GPU acceleration in sherpa-onnx (cpu, cuda, coreml, directml, tensorrt).
  • Configurable Options: Added provider, chunk_size, and sanitize_markdown to the [tts] configuration section, with examples in configs/default.toml.

Verification:

  • Added internal/tts/text_utils_test.go to verify sanitization and chunking logic.
  • Verified build and existing tests pass with make build and make test.
This PR implements improvements to the TTS system as described in issue #28. Key changes: - **Text Chunking:** Implemented a sentence-aware text chunker that splits long texts into smaller segments (~100 tokens/400 chars) for pipelined audio generation. This prevents crashes on large texts and improves perceived performance. - **Markdown Sanitization:** Added a markdown sanitizer to strip formatting characters (bold, italics, headers, links, code blocks) before text is sent to the TTS engine. - **Hardware Acceleration:** Added a 'provider' configuration option to enable GPU acceleration in sherpa-onnx (cpu, cuda, coreml, directml, tensorrt). - **Configurable Options:** Added `provider`, `chunk_size`, and `sanitize_markdown` to the `[tts]` configuration section, with examples in `configs/default.toml`. Verification: - Added `internal/tts/text_utils_test.go` to verify sanitization and chunking logic. - Verified build and existing tests pass with `make build` and `make test`.
gemini-code-assist[bot] commented 2026年03月21日 22:23:05 +01:00 (Migrated from github.com)
Copy link

Summary of Changes

Hello, I'm Gemini Code Assist1 ! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request significantly enhances the Text-to-Speech (TTS) system by improving its performance, robustness, and configurability. It introduces intelligent text processing to handle long inputs more gracefully, ensures that the TTS engine receives clean, unformatted text, and provides options for leveraging hardware acceleration to speed up audio generation.

Highlights

  • Text Chunking: Implemented a sentence-aware text chunker that splits long texts into smaller segments (~100 tokens/400 chars) for pipelined audio generation. This prevents crashes on large texts and improves perceived performance.
  • Markdown Sanitization: Added a markdown sanitizer to strip formatting characters (bold, italics, headers, links, code blocks) before text is sent to the TTS engine.
  • Hardware Acceleration: Added a 'provider' configuration option to enable GPU acceleration in sherpa-onnx (cpu, cuda, coreml, directml, tensorrt).
  • Configurable Options: Added provider, chunk_size, and sanitize_markdown to the TTS configuration section, with examples in configs/default.toml.
Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature Command Description
Code Review /gemini review Performs a code review for the current pull request in its current state.
Pull Request Summary /gemini summary Provides a summary of the current pull request in its current state.
Comment @gemini-code-assist Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help /gemini help Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for GitHub and other Google products, sign up here.

  1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩︎

## Summary of Changes Hello, I'm Gemini Code Assist[^1]! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed! This pull request significantly enhances the Text-to-Speech (TTS) system by improving its performance, robustness, and configurability. It introduces intelligent text processing to handle long inputs more gracefully, ensures that the TTS engine receives clean, unformatted text, and provides options for leveraging hardware acceleration to speed up audio generation. ### Highlights * **Text Chunking**: Implemented a sentence-aware text chunker that splits long texts into smaller segments (~100 tokens/400 chars) for pipelined audio generation. This prevents crashes on large texts and improves perceived performance. * **Markdown Sanitization**: Added a markdown sanitizer to strip formatting characters (bold, italics, headers, links, code blocks) before text is sent to the TTS engine. * **Hardware Acceleration**: Added a 'provider' configuration option to enable GPU acceleration in sherpa-onnx (cpu, cuda, coreml, directml, tensorrt). * **Configurable Options**: Added `provider`, `chunk_size`, and `sanitize_markdown` to the `TTS` configuration section, with examples in `configs/default.toml`. <details> <summary><b>Using Gemini Code Assist</b></summary> <br> The full guide for Gemini Code Assist can be found on our [documentation page](https://developers.google.com/gemini-code-assist/docs/review-github-code), here are some quick tips. <b>Invoking Gemini</b> You can request assistance from Gemini at any point by creating a comment using either `/gemini <command>` or `@gemini-code-assist <command>`. Below is a summary of the supported commands on the current page. Feature | Command | Description --- | --- | --- Code Review | `/gemini review` | Performs a code review for the current pull request in its current state. Pull Request Summary | `/gemini summary` | Provides a summary of the current pull request in its current state. Comment | @gemini-code-assist | Responds in comments when explicitly tagged, both in pull request comments and review comments. Help | `/gemini help` | Displays a list of available commands. <b>Customization</b> To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a `.gemini/` folder in the base of the repository. Detailed instructions can be found [here](https://developers.google.com/gemini-code-assist/docs/customize-gemini-behavior-github). <b>Limitations & Feedback</b> Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with :thumbsup: and :thumbsdown: on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for GitHub and other Google products, sign up [here](https://google.qualtrics.com/jfe/form/SV_2cyuGuTWsEw84yG). </details> [^1]: Review the [Privacy Notices](https://policies.google.com/privacy), [Generative AI Prohibited Use Policy](https://policies.google.com/terms/generative-ai/use-policy), [Terms of Service](https://policies.google.com/terms), and learn how to configure Gemini Code Assist in GitHub [here](https://developers.google.com/gemini-code-assist/docs/customize-gemini-behavior-github). Gemini can make mistakes, so double check it and [use code with caution](https://support.google.com/legal/answer/13505487).
gemini-code-assist[bot] (Migrated from github.com) reviewed 2026年03月21日 22:24:29 +01:00
gemini-code-assist[bot] (Migrated from github.com) left a comment
Copy link

Code Review

This pull request introduces several significant improvements to the Text-to-Speech (TTS) system. The implementation of text chunking and markdown sanitization will improve performance and robustness, especially for long and complex model outputs. Adding configuration for GPU providers and other TTS options is also a great enhancement for users with capable hardware.

My review includes two main suggestions:

  • In internal/tts/text_utils.go, I've recommended pre-compiling regular expressions at the package level to improve performance.
  • In internal/tts/tts_cgo.go, I've suggested restoring a helpful debug log message that was removed, to maintain good debuggability for the TTS generation process.

Overall, these are excellent additions to the project.

## Code Review This pull request introduces several significant improvements to the Text-to-Speech (TTS) system. The implementation of text chunking and markdown sanitization will improve performance and robustness, especially for long and complex model outputs. Adding configuration for GPU providers and other TTS options is also a great enhancement for users with capable hardware. My review includes two main suggestions: - In `internal/tts/text_utils.go`, I've recommended pre-compiling regular expressions at the package level to improve performance. - In `internal/tts/tts_cgo.go`, I've suggested restoring a helpful debug log message that was removed, to maintain good debuggability for the TTS generation process. Overall, these are excellent additions to the project.
@ -0,0 +66,4 @@
text=reNewlines.ReplaceAllString(text,"\n\n")
returnstrings.TrimSpace(text)
}
gemini-code-assist[bot] (Migrated from github.com) commented 2026年03月21日 22:24:29 +01:00
Copy link

medium

This function repeatedly compiles regular expressions on each call, which is inefficient. For better performance, you should compile them once at the package level using a var block with regexp.MustCompile and reuse the compiled objects.

For example:

var(codeBlockRegex=regexp.MustCompile("(?s)```.*?```")// ... other regexes)funcsanitizeMarkdown(textstring){text=codeBlockRegex.ReplaceAllString(text,"")// ...}

This advice also applies to the splitIntoChunks function. Additionally, the nested regex compilation on line 18 is particularly inefficient and should be avoided.

![medium](https://www.gstatic.com/codereviewagent/medium-priority.svg) This function repeatedly compiles regular expressions on each call, which is inefficient. For better performance, you should compile them once at the package level using a `var` block with `regexp.MustCompile` and reuse the compiled objects. For example: ```go var ( codeBlockRegex = regexp.MustCompile("(?s)```.*?```") // ... other regexes ) func sanitizeMarkdown(text string) { text = codeBlockRegex.ReplaceAllString(text, "") // ... } ``` This advice also applies to the `splitIntoChunks` function. Additionally, the nested regex compilation on line 18 is particularly inefficient and should be avoided.
gemini-code-assist[bot] (Migrated from github.com) commented 2026年03月21日 22:24:29 +01:00
Copy link

medium

This debug log message, which reports the number of generated samples and the sample rate, is valuable for debugging. Consider restoring it inside the speakChunk function, guarded by the ZOP_DEBUG_TTS environment variable check, to maintain debuggability.

![medium](https://www.gstatic.com/codereviewagent/medium-priority.svg) This debug log message, which reports the number of generated samples and the sample rate, is valuable for debugging. Consider restoring it inside the `speakChunk` function, guarded by the `ZOP_DEBUG_TTS` environment variable check, to maintain debuggability.
github-actions[bot] commented 2026年03月21日 22:31:02 +01:00 (Migrated from github.com)
Copy link

Mobile UI Screenshot

zop mobile UI

### Mobile UI Screenshot ![zop mobile UI](https://raw.githubusercontent.com/peterwwillis/zop/ui-screenshots/pr-32/zop-mobile-ui.png) <!-- Sticky Pull Request Commentzop-mobile-ui -->
Sign in to join this conversation.
No reviewers
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
mutablecc/zop!32
Reference in a new issue
mutablecc/zop
No description provided.
Delete branch "feature/speed-up-tts"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?