OCR Speed and Performance

Advanced Guide: Performance Engineering for Browser-Based OCR

This section dives into practical techniques to shorten time-to-text on real devices—from low-end phones to high-core desktops—while keeping the UI responsive. The guidance assumes a fully client-side pipeline and focuses on compute, memory, decoding, scheduling, and measurement discipline you can apply without changing the current design.

1) End-to-End Pipeline: Keep Stages Short and Idempotent

Model the flow as decode → normalize → segment → recognize → postprocess → export. Keep each stage: (a) bounded in time (so progress can advance frequently), (b) idempotent (safe to retry), and (c) stream-friendly (deliver partial results early). Short stages prevent the main thread from stalling and make perceived speed match actual speed.

2) WebAssembly Hot Path: SIMD, Threads, and Isolation

• SIMD: Prefer WASM builds with SIMD for pixel math (grayscale, threshold, morphology). It reduces per-pixel cost dramatically.
• Multithreading: Use Worker threads for recognition shards. Value scales until memory bandwidth saturates; typical sweet spot is min(cores, 4–6) on laptops and 2–3 on mobiles.
• Cross-origin isolation: When you need SharedArrayBuffer for peak performance, enable the appropriate headers at publish time (no UI changes required).

3) Task Graph & Scheduling

Represent work as a small DAG. CPU-heavy steps (binarization, recognition) run in Workers; lightweight UI updates stay on the main thread. Use back-pressure—do not enqueue unbounded tiles. When the device is under thermal throttle, reduce concurrency automatically and prioritize the last visible page/region first.

4) Image Decode & Upload: Avoid Extra Copies

• Decode path: Favor native decoders (createImageBitmap / HTMLImageElement.decode) and render to an offscreen canvas only once.
• Zero-copy transfers: Pass ArrayBuffers via postMessage with transfer lists; avoid base64 data URLs for big images.
• One resize only: If you need to scale, do it once up front to reach a target x-height (≈20–30 px) and reuse that buffer downstream.

5) Tiling Strategy for Large Pages

For A4/Letter scans over ~3000 px on the long edge, tile into 1024–1536 px squares with a tiny overlap (8–12 px) so characters at tile borders remain intact. Recognize tiles in parallel and merge results by reading order. Tiling bounds memory, keeps Workers busy, and avoids single giant allocations that trigger GC pauses.

6) Memory Discipline

• Pool reusable Uint8Array/Uint8ClampedArray buffers; long-lived pools reduce churn.
• Prefer ImageData over ad-hoc objects; it is predictable and transferable.
• Release Blob URLs and revoke object URLs as soon as exports complete.
• Avoid keeping both color and grayscale copies unless needed; compute on demand.

7) Heuristics by Device Class

Tune defaults using simple signals:

• Concurrency: start with max(1, Math.min(4, hardwareConcurrency-1)), then adapt if the progress per second drops.
• Resolution guardrails: cap the working width on low-memory devices (e.g., 2200–2600 px) and prefer higher contrast over higher DPI when constrained.
• Battery/thermal hints: when available, lower concurrency under heavy thermals to stabilize speed and avoid OS throttling.

8) Preprocessing That Pays for Itself

Only add steps that reduce total cost. A quick deskew and gentle contrast stretch can reduce recognition passes. Aggressive denoise or heavy unsharp masks often cost more than they save and may harm small strokes—avoid unless metrics prove a gain.

9) Postprocessing with Awareness of Cost

• Do light whitespace normalization and digit/decimal fixes inline.
• Defer expensive dictionary checks until the user pauses (idle callback) or chooses “finalize.”
• Make validators incremental—run them on changed spans, not entire pages.

10) Caching: Fast Where Safe, Never for User Content

Cache the engine, models, and language data; never cache user images or recognized text by default. Keep a tiny “warm start” that includes compiled WASM and the most common language pack to reach a responsive first run.

11) PDF & Multi-Page Considerations

• Render each page independently at a target x-height rather than one global DPI.
• Prioritize the current page; queue the rest with low priority so interaction stays smooth.
• Skip embedded thumbnails—seek the highest-resolution XObject to avoid re-renders.

12) Throughput vs. Latency Modes

Interactive mode optimizes for the first visible result; batch mode optimizes total time. Use the same code path with different queue policies: interactive = short tiles, frequent UI updates; batch = larger tiles, fewer paints, bigger worker pools.

13) Measuring What Matters

• T_decode, T_segment, T_recognize, T_post: per-stage timings with rolling averages.
• P99 per stage: track tail latency to spot a single slow tile/page.
• Progress cadence: update the bar at least every 150–300 ms to keep users informed.
• Memory watermark: highest concurrent allocation; lower it if GC spikes appear.

14) UI Responsiveness Hygiene

• Never block the main thread for >50 ms; chunk work and yield frequently.
• Use requestAnimationFrame for progress paints; group DOM writes.
• Debounce expensive layout operations; avoid forced reflows while jobs run.

15) Energy & Thermals

Sustained full-core use on mobiles triggers throttling. Prefer steady utilization (70–85%) over spikes. Reduce tile size or worker count when frame time or progress cadence degrades; the UI will feel faster even if raw compute is lower.

16) Practical Checklists

• Decode once, resize once, reuse buffers.
• Keep tiles ~1024–1536 px with 8–12 px overlap.
• Start with 2–4 workers; adapt if progress stalls.
• Stream partial lines/blocks to the UI; don’t wait for the entire page.
• Measure P99 per stage; optimize the slowest stage first.

17) Case Study: Fast Receipts on Mid-Tier Phones

A team processed long thermal receipts on mid-range devices. Initial runs lagged due to giant single-pass images. They switched to 1280-px tiles, pooled a single grayscale buffer, and limited concurrency to 2 workers under heat. Per-receipt latency dropped noticeably, and progress feedback became smooth—even though peak CPU usage fell.

18) Troubleshooting Speed Regressions

• Symptom: progress freezes near the end. Likely a large postprocess step—make it incremental.
• Symptom: memory spikes. Double buffering or stray canvases—recycle and revoke URLs.
• Symptom: uneven speeds across pages. Mixed DPI—normalize target x-height per page.

19) Summary

Real-world speed comes from disciplined staging, minimal copies, right-sized tiles, adaptive concurrency, and honest measurement. Keep the hot path tight, stream results early, and optimize the tail—not just the average. The payoff is simple: faster first text, steadier progress, and a smoother experience on every device.