Multilingual OCR: Supported Languages and Capabilities

Advanced Guide: Multiscript Recognition, RTL Handling, and Locale Rules

This guide expands on practical techniques for multilingual OCR: detecting the right script, preserving directionality, handling complex shaping and combining marks, tokenizing text in script-appropriate ways, and exporting results that remain searchable and unambiguous. The focus is on changes you can apply without altering the current UI or code paths.

1) Script Detection and Routing

Treat script detection as a routing problem. Use lightweight heuristics (Unicode ranges by codepoint, dominant glyph shapes, presence of combining marks) to guess the active script per region. When multiple scripts appear, segment visually into blocks—headers, tables, side notes—then run recognition per block with a minimal language set. This reduces the hypothesis space and lowers look-alike confusions such as O/0, l/1, S/5, and Latin–Cyrillic overlaps.

• Default to a single dominant script per block; add a secondary pass only if confidence drops below a threshold.
• Keep a small “script manifest” with page→region→script mapping so results are reproducible and auditable.
• Where automatic routing is uncertain, allow users to pin a language for a selected region and retry.

2) Directionality (Bidi) Basics

Right-to-left scripts (Arabic, Hebrew) follow the Unicode Bidirectional Algorithm. Mixed lines that include numbers, Latin acronyms, or punctuation require explicit control marks to render predictably. When reconstructing text, preserve or insert the minimal marks needed for clarity:

• LRM/RLM: Left-to-Right/Right-to-Left Mark for subtle direction hints between tokens.
• LRE/RLE & PDF: Explicit embedding; use sparingly when whole phrases flip direction.
• LRI/RLI/FSI & PDI: Isolates for inline spans in modern text; helpful in UI strings.

In exports, preserve logical order (reading order) and avoid stripping directional marks during normalization. For display fields, choose fonts with full RTL shaping support and test caret movement/selection behavior.

3) Shaping, Ligatures, and Join Controls

Arabic letters change form based on position and joining; Indic scripts create conjuncts with virama and matras; Latin uses optional ligatures (fi/ffl). Recognition operates on raster shapes, but postprocessing must keep text valid at the codepoint level:

• Honor ZWJ (zero-width joiner) and ZWNJ (non-joiner) where they affect legibility.
• Normalize canonically (NFC) for stable storage and searching; reserve NFKC for cases where compatibility forms are acceptable.
• Avoid converting presentation-forms in Arabic back to base letters if that removes intended join behavior.

4) Grapheme Clusters and Combining Marks

A “character” to users may be multiple codepoints: base + combining accents, or family clusters in Indic scripts. For cursoring, selection, and substring operations, operate on grapheme clusters rather than codepoints. In OCR exports, ensure that combining marks follow their base and remain in canonical order so copy/paste behaves as expected.

5) Tokenization Across Scripts

Words are not always separated by spaces. CJK scripts often omit spaces; Thai lacks explicit word boundaries; Arabic clitics attach to words; German compounds are long. Use script-aware tokenizers:

• CJK: Prefer dictionary or statistical segmentation; keep punctuation as separate tokens.
• Thai/Khmer/Lao: Apply word-break algorithms using dictionaries and heuristics for numerals/dates.
• Arabic/Hebrew: Handle affixes and diacritics carefully; normalize tatweel (kashida) where decorative.
• Latin/Cyrillic: Preserve hyphenation rules and avoid collapsing soft hyphens if they carry meaning.

6) Numerals, Dates, and Locale Conventions

Different locales vary in digit shapes, decimal separators, thousands marks, and calendar systems. Normalize with explicit locale context:

• Map Eastern Arabic numerals (٠١٢٣٤٥٦٧٨٩) to ASCII digits for search if the product demands it—store both when round-trip fidelity matters.
• Respect locale for currency (e.g., non-breaking spaces in French thousands), and disallow unsafe character folding in legal/financial exports.
• When detecting dates, prefer ISO-8601 in structured fields while preserving the original string alongside.

7) Fonts, Fallbacks, and Variant Forms

OCR accuracy depends on stroke fidelity. Subpixel rendering, synthetic bolding, or color fringing in screenshots can confuse recognition. Before OCR, desaturate subpixel-rendered UI text to grayscale. For CJK, be aware of Han unification and compatibility ideographs; do not over-normalize when identity distinctions matter for names or legal entries.

8) Region-Wise Processing for Mixed Pages

Mixed manuals and passports frequently blend Latin, Arabic, and numeric fields. Segment into labeled regions (photo caption, MRZ, address block, stamps) and apply per-region language packs. Merge outputs in reading order, keeping a simple provenance structure (page, region id, script, confidence) for later auditing.

9) Dictionaries, Lexicons, and Validators

Lightweight language packs—common names, place lists, product catalogs—dramatically improve effective accuracy in multilingual settings. Couple them with validators for phone numbers, postal codes, and IBAN/credit-card checksums where appropriate. When a correction is made, log which rule or lexicon entry triggered it for transparency.

10) Confidence, Calibration, and Human-in-the-Loop

Confidence scores are more informative when tracked by script. If a single page reports high variance (e.g., RTL regions low, Latin high), route low-confidence spans to a secondary pass or a quick review tool. A small review on the lowest-confidence 5–10% of tokens often provides the biggest lift for mixed-language archives.

11) Unicode-Safe Export

Make UTF-8 the default. Preserve direction marks and zero-width controls that carry semantics. Use NFC normalization for storage and search, and include a “raw” field when you need byte-exact reproduction for legal purposes. For layout-aware exports (hOCR/ALTO), keep coordinates and reading order so downstream systems can reconstruct structure.

12) Evaluation by Script and Domain

Evaluate separately per script and document class. CER/WER alone can hide systematic failures—keep confusion matrices per script (Arabic joining mishaps, Latin diacritics, Cyrillic–Latin swaps). Include stress tests: small diacritics, mixed numerals, vertical text (Japanese), and heavily stylized headings. Freeze a tiny “golden set” for quick pre-release checks.

13) Case Study: Bilingual Receipts

A retailer processed bilingual receipts (Arabic/English) with amounts and item lines. Baseline errors concentrated in Arabic joining forms and currency lines. The team split pages into two regions, forced Arabic + digits in the left block and English + digits in the right, normalized Eastern Arabic numerals, and applied a currency/total validator. With confidence-routed review on the bottom 8% of tokens, effective errors fell by more than half without UI changes.

14) Accessibility and Search

Keep output compatible with screen readers and search tools. Use language tags where your format allows (lang attributes or metadata) so TTS engines select the correct voice. For search, index both normalized and original forms when users commonly paste text from external sources that differ in normalization.

15) Practical Checklists

• Limit languages per region; expand only when confidence suggests a mismatch.
• Preserve bidi marks and test mixed LTR/RTL lines in a simple viewer.
• Normalize to NFC for storage; retain originals when legal traceability matters.
• Keep small, script-specific lexicons and pattern validators (names, addresses, dates).
• Record region→script decisions and per-script confidence in exports.

Mini-Glossary

Bidi: Bidirectional text handling for LTR/RTL mixing. Grapheme Cluster: A user-perceived character; may consist of multiple codepoints. ZWJ/ZWNJ: Zero-width join/non-join controls that influence shaping. NFC/NFKC: Unicode normalization forms; NFC preserves canonical equivalence, NFKC applies compatibility folds. Han Unification: Shared ideograph set across CJK languages with regional variants.

Summary

Multilingual OCR works best when scripts are detected early, routed to the smallest viable language set, and reconstructed with directionality, shaping, and locale rules intact. Pairing script-aware tokenization with careful exports yields text that is both readable and reliable—ready for search, translation, and archival use.