fix: preserve sentence punctuation in chunk_text, align clean_text defaults, fix version#131
Open
voidborne-d wants to merge 1 commit intoKittenML:mainfrom
Open
Conversation
…faults, fix version Three interrelated fixes: 1. chunk_text destroys sentence-ending punctuation (affects KittenML#67, KittenML#72) - re.split(r'[.!?]+', text) strips all periods, question marks, and exclamation marks from the text before feeding it to the TTS model - The model needs this punctuation for correct prosody and intonation - Fix: use lookbehind split r'(?<=[.!?])\s+' to keep punctuation attached to the preceding sentence 2. clean_text default mismatch between public API and internal model - KittenTTS wrapper: clean_text=False (skips text preprocessing) - KittenTTS_1_Onnx: clean_text=True (runs text preprocessing) - Users calling the public API never get text preprocessing (number expansion, abbreviations, etc.) unless they explicitly pass clean_text=True - generate_to_file didn't expose clean_text at all - Fix: align wrapper defaults to True, add clean_text param to generate_to_file, remove debug print() from generate() 3. Version mismatch (KittenML#80) - __init__.py: 0.1.0 - pyproject.toml: 0.8.1 - setup.py: 0.8.1 - Fix: update __init__.py to 0.8.1 Tests: 26 new regression tests covering punctuation preservation, edge cases, API defaults, version consistency, and source audit.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Three interrelated fixes for text processing and API consistency.
1.
chunk_textdestroys sentence-ending punctuation (relates to #67, #72)Root cause:
re.split(r'[.!?]+', text)splits on sentence-ending punctuation and removes it entirely. The resulting chunks lose all periods, question marks, and exclamation marks.ensure_punctuation()then adds a comma to each chunk, so the TTS model sees commas where periods/question marks should be — degrading prosody, intonation, and causing the model to treat declarative, interrogative, and exclamatory sentences identically.This contributes to:
Fix: Replace the destructive
re.split(r'[.!?]+', text)withre.split(r'(?<=[.!?])\s+', text)— a lookbehind pattern that splits on whitespace after sentence-ending punctuation, keeping the punctuation attached to the preceding sentence.2.
clean_textdefault mismatch between public API and internal modelUsers calling the public API (
KittenTTS) never get text preprocessing (number expansion, abbreviation handling, etc.) unless they explicitly passclean_text=True, becauseKittenTTS.generate()defaults toclean_text=Falsewhile the internalKittenTTS_1_Onnx.generate()defaults toTrue. Additionally,generate_to_file()didn't exposeclean_textat all.Fix: Align all
KittenTTSwrapper defaults toTrue, addclean_textparameter togenerate_to_file(), remove stale debugprint()fromgenerate().3. Version mismatch (#80)
__init__.__version__says0.1.0whilepyproject.tomlandsetup.pysay0.8.1, causing wheel installation failures.Fix: Update
__init__.__version__to0.8.1.Tests
26 new regression tests (
tests/test_chunk_text_and_api.py), all lightweight (no model/ONNX/GPU/espeak needed):