fix(textkit): skip hyphen insertion for CJK characters in line breaking - #3315
fix(textkit): skip hyphen insertion for CJK characters in line breaking#3315matangot wants to merge 3 commits into
Conversation
CJK (Chinese/Japanese) text gets incorrectly hyphenated with `-` dashes at line breaks. breakLines unconditionally inserts a hyphen glyph when breaking at a penalty node, which is correct for Latin scripts but wrong for CJK text where characters wrap naturally without hyphens. Check whether the character at the break point is in the CJK Unicode range and skip hyphen insertion if so.
🦋 Changeset detectedLatest commit: 2322daa The changes in this PR will be included in the next version bump. This PR includes changesets to release 9 packages
Not sure what this means? Click here to learn what changesets are. Click here if you're a maintainer who wants to add another changeset to this PR |
Adds a changeset and a paired test: a CJK string that wraps keeps no hyphen at the break, while a latin string under the same width and helper still gets one, so the guard is shown to be selective rather than blanket. Without the fix the CJK case reports ['日本語-', '組版-', '処理'].
|
Rebased-by-merge onto current The test is a pair, so the guard is shown to be selective rather than blanket — same helper, same width, only the script differs:
Without the fix the CJK case reports On overlap with the other line-breaking work in flight (#3288 CSS-compatible |
Summary
CJK (Chinese/Japanese) text gets incorrectly hyphenated with
-dashes at line breaks. This happens becausebreakLinesunconditionally inserts a hyphen glyph when breaking at a penalty node, which is correct for Latin scripts but wrong for CJK text where characters wrap naturally without hyphens.Before
After
The Problem
CJK text has no spaces between characters, so the entire text block becomes a single "word" in the layout engine. When a user's
hyphenationCallbacksplits this into individual characters (the correct approach for CJK),breakLinesinserts a-hyphen at every line break point — because it assumes all penalty-node breaks are Latin-style hyphenation.The Fix
In
breakLines, check whether the character at the break point is in the CJK Unicode range. If so, skip the hyphen glyph insertion.Where
CJK_RANGE = /[\u3000-\u9fff\uf900-\ufaff\uff00-\uffef]/This covers:
Korean (Hangul) is excluded as it uses spaces between words and doesn't hit this code path in practice.
Notes
A complete CJK line breaking solution would also include kinsoku (禁則処理) rules in the layout engine — preventing punctuation like
。,、from starting a new line. That's currently left to userland viahyphenationCallbackbut could be a follow-up enhancement.