|
| 1 | +# Internationalization & localization — depth (domain R) |
| 2 | + |
| 3 | +**Read this when** the target renders user-facing text in more than one language |
| 4 | +or locale, accepts user-supplied Unicode text, or the review question is "is this |
| 5 | +correct across locales and scripts?" It is the depth behind domain R's |
| 6 | +at-a-glance checklist in `domain-checklists.md` (§R) — the standards-heavy |
| 7 | +correctness a checklist can't hold. Cross-link the checklist; don't restate it. |
| 8 | + |
| 9 | +## Encoding & Unicode normalization |
| 10 | +- **UTF-8 everywhere, declared in-document.** W3C i18n: "Choose UTF-8 for all |
| 11 | + content," and "always use an in-document encoding declaration, even if you are |
| 12 | + also using the HTTP header." When a document's bytes and its declared encoding |
| 13 | + disagree, non-ASCII text mojibakes silently. |
| 14 | +- **Normalize before you compare, dedup, or key on user text — and normalize |
| 15 | + *after* concatenation, not before.** Two visually identical strings can be |
| 16 | + different byte sequences (`é` as one code point vs `e` + a combining accent), so |
| 17 | + an equality check, a `GROUP BY`, a uniqueness constraint, or a cache key over |
| 18 | + un-normalized user text treats equal strings as different. Normalize to one form |
| 19 | + (NFC is the web default — W3C recommends it). The subtle trap: **"None of the |
| 20 | + Normalization Forms are closed under string concatenation … even if two strings |
| 21 | + X and Y are normalized, their string concatenation X+Y is _not_ guaranteed to be |
| 22 | + normalized"** (Unicode UAX #15). A pipeline that normalizes fragments and then |
| 23 | + concatenates can still emit un-normalized output — normalize at the boundary |
| 24 | + where the value is compared or stored, *after* assembly. (The checklist's |
| 25 | + 🚩 "concatenated translated fragments" is this defect's surface form.) |
| 26 | + |
| 27 | +## Bidirectional text (RTL) and the bidi-override attack |
| 28 | +- **Declare direction; don't rely on the renderer guessing.** W3C i18n: add |
| 29 | + `dir="rtl"` when the overall document direction is right-to-left, and "tightly |
| 30 | + wrap every opposite-direction phrase in markup that sets its base direction." An |
| 31 | + RTL string (Arabic, Hebrew) interleaved with LTR runs (numbers, Latin |
| 32 | + identifiers) reorders incorrectly without an explicit base direction — a |
| 33 | + displayed IBAN, price, or file path can render in the wrong visual order. |
| 34 | +- **Reject or flag Unicode bidi control characters in source and in untrusted |
| 35 | + input.** UAX #9 (the Unicode Bidirectional Algorithm) defines explicit |
| 36 | + directional-formatting controls in distinct classes: **overrides** (RLO/LRO), |
| 37 | + embeddings, **isolates** (RLI/LRI/FSI/PDI — the *safer* mechanism the |
| 38 | + direction-wrapping bullet above relies on), and the terminators PDF/PDI. UAX #9 |
| 39 | + says the **overrides** in particular "are to be avoided wherever possible, because |
| 40 | + of security concerns" (see Unicode UTR #36). This is the Trojan-Source class: a |
| 41 | + bidi control in a source comment or string literal can make code *read* in review |
| 42 | + as the opposite of what compiles (the attack abused isolates as well as |
| 43 | + overrides), and in user content can spoof a displayed value — so flag bidi |
| 44 | + controls broadly in source and untrusted input, not only the overrides. |
| 45 | + (`domain-checklists.md` |
| 46 | + flags zero-width / bidi control characters for instruction-smuggling in code — |
| 47 | + the invisible-character check; this is the rendering-and-review-legibility side |
| 48 | + of the same characters.) |
| 49 | + |
| 50 | +## Plurals, formatting, and collation |
| 51 | +- **Plural selection uses the locale's CLDR categories, never `n == 1`.** Unicode |
| 52 | + CLDR defines up to six plural categories — **zero, one, two, few, many, other** — |
| 53 | + and "languages vary in how they handle plurals"; a hardcoded |
| 54 | + `count === 1 ? singular : plural` is wrong for languages with dual/paucal forms. |
| 55 | + **`other` is the required default** every language has. Route plurals through an |
| 56 | + ICU / CLDR-backed API, never string concatenation of a count with a fixed suffix. |
| 57 | +- **Format numbers, dates, currency, and times through a locale-aware formatter.** |
| 58 | + Grouping separators, decimal marks, calendar system, first-day-of-week, and |
| 59 | + 12/24-hour clock all vary by locale; a hand-rolled format is correct only for the |
| 60 | + author's locale. And **sort / collate with a locale collator, not a byte or |
| 61 | + `.sort()` compare** — `.sort()` on localized text orders by code point, which |
| 62 | + mis-orders accented and non-Latin scripts (the checklist's 🚩 ".sort() on |
| 63 | + localized text"). A collation test asserts the *locale's* expected order, not the |
| 64 | + ASCII order. |
| 65 | +- **Leave room for text expansion.** Translated strings run longer than English |
| 66 | + (German and Finnish are the usual worst cases); a layout that fits English |
| 67 | + exactly clips or overflows once localized. Verify the running UI in a |
| 68 | + long-string locale, not only in English (cross-ref `product-ux-quality.md` layout |
| 69 | + invariants across the sweep matrix). |
| 70 | + |
| 71 | +## Standards (verified URLs + dates in `docs/standards-index.md`) |
| 72 | +W3C Internationalization (authoring techniques); Unicode UAX #15 (Normalization |
| 73 | +Forms, Unicode 18.0.0); Unicode UAX #9 (Bidirectional Algorithm, Unicode 18.0.0) |
| 74 | +with UTR #36 (security considerations); Unicode CLDR plural rules. |
0 commit comments