torchocr#

Pipeline#

class torchocr.OCRPipeline(detector, recognizer, post_processor, decoder, *, crop_size=None, detector_transforms=None, recognizer_transforms=None, recognizer_batch_size=64, min_text_score=None, orientation=None)[source]#

Bases: object

End-to-end OCR pipeline: image -> populated DocumentTensor.

Composes a detector, post-processor, recognizer, and decoder:

  1. orientation (if given) sets a page scanned sideways or upside down upright; every later step sees the upright page.

  2. detector_transforms (if given) resizes/normalizes the image for the detector, e.g. detector_weights.transforms().

  3. The post-processor yields text regions. Rotated quads are used when it offers quadrilaterals() (as DBPostProcessor does); otherwise its (K, 5) [batch_idx, x1, y1, x2, y2] boxes are used. Regions are mapped back to input-image coordinates.

  4. recognizer_transforms.normalize (if given) normalizes the original-resolution image, and every region is rectified out of it with torchocr.ops.crop_quads() – aspect-preserving, padded, one batched call on the model’s device.

  5. The recognizer and decoder turn crops into strings.

Use from_pretrained() to assemble all four from published weights.

Parameters:
  • detector (Module) – Module returning a DBNetOutput.

  • recognizer (Module) – Module mapping (K, C, H, W) crops to (T, K, num_classes) logits.

  • post_processor (Callable[[DBNetOutput], Tensor]) – Callable mapping a DBNetOutput to boxes (see above).

  • decoder (Callable[[Tensor], list[str]]) – Callable mapping (T, K, num_classes) logits to a list of K strings.

  • crop_size (tuple[int, int] | None) – (height, max_width) of recognizer crops. Default: from recognizer_transforms when given, else (32, 128).

  • detector_transforms (Callable[[Tensor], Tensor] | None) – Optional preset applied to the input image before the detector (e.g. DetectionPreset).

  • recognizer_transforms (RecognitionPreset | None) – Optional RecognitionPreset whose normalize is applied to the page before cropping.

  • recognizer_batch_size (int) – Words cropped and recognized per forward pass, bounding memory on dense documents. Default 64.

  • min_text_score (float | None) – Drop regions whose recognition confidence (DocumentTensor.scores) is below this value. None (default) keeps every region. 0.8 maximizes ICDAR-2015 training-split end-to-end hmean for the published scene-text models.

  • orientation (PageOrientation | None) – Optional PageOrientation run before detection. Default None (pages are read as given).

Without presets the input is fed to both models as-is, so it must already be normalized and sized for the detector (the v0.1 behavior).

classmethod from_pretrained(detector_weights=None, recognizer_weights=None, device=None, decoder='auto', min_text_score=None, correct_orientation=False)[source]#

Assemble a pipeline from published weights, in eval mode.

Each stage gets its own preprocessing (weights.transforms()); the post-processor uses detector_weights.meta["postprocess"] and the decoder the charset named in recognizer_weights.meta.

Parameters:
  • detector_weights (WeightsEnum | str | None) – DBNet weights enum member or "Cls.NAME". Default DBNet_MobileNetV3_Large_05_Weights.DEFAULT.

  • recognizer_weights (WeightsEnum | str | None) – CRNN weights enum member or "Cls.NAME". Default CRNN_ResNet34_VD_Weights.DEFAULT.

  • device (device | str | None) – Where to place both models. Default: CPU.

  • decoder (str) – "auto" uses the decoder declared in recognizer_weights.meta["decoder"] (e.g. beam search with a language model), else greedy; "greedy" or "beam" force one.

  • min_text_score (float | None) – See OCRPipeline. Default None (keep all).

  • correct_orientation (bool) – Set pages upright first with PageOrientation.from_pretrained(). Default False.

Return type:

OCRPipeline

class torchocr.DocumentTensor(pixels, text=<factory>, bounding_boxes=None, polygons=None, scores=None, rotation=0)[source]#

Bases: object

Container for OCR-ready document data.

Parameters:
pixels#

The input image, unmodified – or set upright when the pipeline corrects page orientation (see rotation).

Type:

torch.Tensor

text#

K recognized strings, one per region.

Type:

list[str]

bounding_boxes#

(K, 4) axis-aligned (x1, y1, x2, y2) boxes.

Type:

torch.Tensor | None

polygons#

(K, 4, 2) rotated text quads in reading order (top-left, top-right, bottom-right, bottom-left), when the post-processor provides them.

Type:

torch.Tensor | None

scores#

(K,) recognition confidence in [0, 1]: the mean probability of the characters read (0 if none).

Type:

torch.Tensor | None

rotation#

Clockwise angle (0, 90, 180 or 270) the input page was turned by; pixels and every region are in the upright frame.

Type:

int

to(*args, **kwargs)[source]#

Return a copy moved to the requested device/dtype.

Parameters:
Return type:

DocumentTensor

Page orientation#

class torchocr.PageOrientation(classifier, transforms, min_confidence=0.0)[source]#

Bases: Module

Predict how far each page is turned, and turn it back.

Wraps a 4-way classifier (0, 90, 180, 270 degrees) and its preprocessing. A page is only turned when the classifier is confident enough (min_confidence): most real pages arrive upright, and turning an upright page costs more than leaving a turned one as it is.

Parameters:
  • classifier (Module) – Module mapping (B, 3, h, w) preprocessed pages to (B, 4) logits, class c meaning “turned c * 90 degrees clockwise”.

  • transforms (Module) – Preprocessing for one (3, H, W) uint8 RGB page, e.g. weights.transforms().

  • min_confidence (float) – Smallest softmax probability at which a page is turned; below it the page is kept as it is (angle 0).

Forward:

(3, H, W) page, (B, 3, H, W) batch or list of pages (sizes may differ), uint8 RGB -> (B,) int64 clockwise angles in {0, 90, 180, 270}.

Example:

orientation = PageOrientation.from_pretrained()
page, angle = orientation.upright(load_image("scan.jpg"))
classmethod from_pretrained(weights=None, device=None, min_confidence=None)[source]#

Classifier, preset and threshold from published weights, in eval mode.

Parameters:
  • weights (WeightsEnum | str | None) – PPLCNet_x1_0_Weights member or name. Default PPLCNet_x1_0_Weights.DEFAULT.

  • device (device | str | None) – Where to place the classifier. Default: CPU.

  • min_confidence (float | None) – Default weights.meta["min_confidence"].

Return type:

PageOrientation

probabilities(pages)[source]#

(B, 4) softmax probabilities of 0, 90, 180 and 270 degrees clockwise.

Parameters:

pages (Tensor | list[Tensor])

Return type:

Tensor

forward(pages)[source]#

Define the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

Parameters:

pages (Tensor | list[Tensor])

Return type:

Tensor

upright(page)[source]#

(page set upright, clockwise angle it was turned) for one (3, H, W) page.

Parameters:

page (Tensor)

Return type:

tuple[Tensor, int]

extra_repr()[source]#

Return the extra representation of the module.

To print customized extra information, you should re-implement this method in your own modules. Both single-line and multi-line strings are acceptable.

Return type:

str

torchocr.rotate_page(page, angle)[source]#

Undo a clockwise turn of angle degrees (a multiple of 90): rotate (..., H, W) back.

Parameters:
Return type:

Tensor

Post-processing#

class torchocr.DBPostProcessor(threshold=0.3, box_thresh=0.7, max_candidates=1000, unclip_ratio=1.5, min_size=3)[source]#

Bases: object

Extract per-region bounding boxes from DBNet probability maps.

Implements PaddleOCR’s contour-based DB post-processing flow: threshold the probability map, run cv2.findContours(), fit a rotated minimum-area rectangle to each contour, score it by averaging the probability map under the rectangle mask, expand the polygon outward by unclip_ratio via pyclipper, and refit the rectangle. Boxes whose score is below box_thresh or whose minor side is below min_size are discarded.

Calling the processor projects the rotated rectangles to axis-aligned (x1, y1, x2, y2) boxes (the v0.1 contract); quadrilaterals() returns the rotated quads themselves.

The output is shaped (K, 5) with columns [batch_idx, x1, y1, x2, y2] so it composes directly with torchvision.ops.roi_align(). K is the total number of accepted boxes across the batch (not capped at one per image). Coordinates are floats in input-pixel units, with x2/y2 being inclusive (x2 = x1 is a one-pixel-wide column) to match the convention of torchvision.ops.masks_to_boxes().

Note

cv2 and pyclipper run on CPU. GPU probability maps are moved to CPU for processing, then the resulting box tensor is placed back on the original device.

Parameters:
  • threshold (float) – Probability threshold for binarization. Default 0.3, matching PaddleOCR.

  • box_thresh (float) – Minimum mean probability under a candidate box for it to be kept. Default 0.7.

  • max_candidates (int) – Maximum number of contours examined per image. Default 1000.

  • unclip_ratio (float) – Polygon expansion ratio used by pyclipper.PyclipperOffset.Execute(). Default 1.5 (matches PaddleOCR v2 weights). Larger values produce looser boxes.

  • min_size (int) – Reject candidates whose minor rotated-rectangle side is below this many pixels. Default 3.

quadrilaterals(output)[source]#

Return one (N_b, 4, 2) tensor of rotated text quads per image.

Regions are sorted in reading order (torchocr.ops.reading_order(): lines top to bottom, words left to right). Vertices are (x, y) input-pixel coordinates clipped to the page, ordered top-left, top-right, bottom-right, bottom-left so they feed torchocr.ops.crop_quads(). Unlike __call__(), the rotation of each region is preserved, which is what polygon-based benchmarks such as ICDAR-2015 score.

Parameters:

output (DBNetOutput)

Return type:

list[Tensor]

Decoders#

class torchocr.CTCGreedyDecoder(charset, blank_index=0)[source]#

Bases: object

Greedy CTC decoder for CRNN-style logits.

Implements the canonical CTC decoding rule: take the per-step argmax, collapse consecutive duplicate indices (treating blanks as ordinary symbols during the collapse), then remove the remaining blank indices. Padding the recognizer’s input with blanks therefore encodes “same character twice” correctly.

Parameters:
  • charset (list[str]) – Character strings indexed by class id. The string at charset[blank_index] is unused; only the index matters.

  • blank_index (int) – Index of the CTC blank class. Default 0, matching the convention used by torch.nn.CTCLoss.

class torchocr.CTCBeamDecoder(charset, beam_width=10, lm=None, lm_weight=0.3, blank_index=0, top_k=8, blank_skip=0.999)[source]#

Bases: object

CTC prefix beam search with optional character-LM shallow fusion.

Implements the prefix beam search of Hannun et al. (2014): each prefix tracks the probability of ending in blank and in its last character, so “ll” (separated by a blank) and “l” (one repeated character) stay distinct. When a character is appended its LM log-probability, scaled by lm_weight, is added; at the end every beam also pays the LM’s end-of-word probability. Frames whose blank probability exceeds blank_skip are treated as pure blank and only the top_k most likely characters are expanded per frame, which keeps a pure-Python search fast on short words.

Parameters:
  • charset (list[str]) – Character strings indexed by class id.

  • beam_width (int) – Prefixes kept per frame. Default 10.

  • lm (CharNGramLM | None) – Optional CharNGramLM.

  • lm_weight (float) – LM scale in log space. Default 0.3.

  • blank_index (int) – CTC blank class. Default 0.

  • top_k (int) – Characters expanded per frame. Default 8.

  • blank_skip (float) – Blank probability above which a frame is skipped. Default 0.999.

class torchocr.CharNGramLM(counts, order)[source]#

Bases: object

Case-insensitive character n-gram language model (interpolated Witten-Bell).

P(c | h) = (C(h, c) + T(h) * P(c | h')) / (C(h) + T(h)) where h' drops the oldest character of h, T(h) counts the distinct continuations of h and the recursion ends at a uniform distribution over vocabulary (which includes an end-of-word event). Every distribution therefore sums to one and unseen characters keep non-zero mass. Text is lowercased, so the model judges spelling while the optical model keeps choosing case.

Parameters:
  • counts (dict[str, dict[str, int]]) – {context: {char: count}} for contexts of length 0 .. order - 1.

  • order (int) – n-gram order (context length + 1).

classmethod from_texts(texts, order=5)[source]#

Count n-grams over texts (one word or line each).

Parameters:
Return type:

CharNGramLM

log_prob(history, char)[source]#

log P(char | history); char="" scores the end of the word.

Parameters:
Return type:

float

score(text)[source]#

Total log-probability of text as a complete word.

Parameters:

text (str)

Return type:

float

torchocr.decoders.build_decoder(weights, charset, kind='auto')[source]#

The decoder a recognizer checkpoint was benchmarked with.

weights.meta["decoder"] may declare {"type": "ctc_beam", "beam_width", "lm_weight", "lm_url"}; the language model is downloaded (hash-checked) on first use.

Parameters:
  • weights – A recognizer weights enum member.

  • charset (list[str]) – Its charset (load_charset(meta["charset"], meta["num_classes"])).

  • kind (str) – "auto" (the declared decoder, else greedy), "greedy" or "beam".

Return type:

CTCGreedyDecoder | CTCBeamDecoder

Losses#

class torchocr.DBLoss(k=50.0, threshold_weight=10.0, binary_weight=1.0, negative_ratio=3.0, eps=1e-06)[source]#

Bases: Module

Differentiable Binarization training loss.

Combines the three components of Liao et al. (AAAI 2020):

  1. Probability-map BCE with online hard example mining (OHEM). OHEM keeps every positive pixel plus the negative_ratio * n_pos hardest negatives, preventing the background class from drowning the loss.

  2. Threshold-map L1 loss restricted to the boundary band where the GT threshold is non-zero.

  3. Approximate-binary Dice loss computed on B_hat = sigmoid(k * (P - T)). The factor k controls how sharply this differentiable form approximates the true Heaviside step P > T: as k -> infinity B_hat becomes a step function, but its gradient also vanishes away from the decision boundary. The paper’s k = 50 is a tested compromise – a tight transition (~0.04 units in P - T) that still leaves backprop signal. Lower k flattens the step toward 0.5; higher k starves the gradients.

Total loss: Lp + binary_weight * Lb + threshold_weight * Lt.

Parameters:
  • k (float) – Steepness of the differentiable binarization. Default 50.

  • threshold_weight (float) – Multiplier on the threshold-map L1 loss. Default 10, matching the paper’s beta.

  • binary_weight (float) – Multiplier on the dice loss over the approximate binary map. Default 1, matching the paper’s alpha. Set to 0 to disable the binary term entirely.

  • negative_ratio (float) – Negatives-to-positives ratio for the OHEM sampling in the probability BCE. Default 3.0.

  • eps (float) – Numerical-stability epsilon for BCE clamping and Dice.

forward(prediction, gt_probability, gt_threshold, training_mask=None)[source]#

Compute DB loss components and their weighted sum.

Parameters:
  • prediction (DBNetOutput) – Output of DBNet for one batch.

  • gt_probability (Tensor) – (B, 1, H, W) 0/1 (or soft) text mask.

  • gt_threshold (Tensor) – (B, 1, H, W) float threshold-map target.

  • training_mask (Tensor | None) – Optional (B, 1, H, W) boolean tensor of valid pixels. False entries are excluded from all three losses (use this to ignore “don’t care” regions such as overlapping or illegible text).

Returns:

{"loss", "probability", "threshold", "binary"}. Call .backward() on the "loss" entry; log the others.

Return type:

dict[str, Tensor]

class torchocr.CRNNLoss(blank_index=0, reduction='mean', zero_infinity=True)[source]#

Bases: Module

Wrapper around torch.nn.CTCLoss for CRNN-style logits.

Applies log_softmax to the recognizer’s logits before delegating to nn.CTCLoss, which expects log probabilities. The blank index defaults to 0, matching CTCGreedyDecoder.

Parameters:
  • blank_index (int) – CTC blank class index. Default 0.

  • reduction (str) – Reduction passed to nn.CTCLoss – ‘mean’, ‘sum’, or ‘none’. Default ‘mean’.

  • zero_infinity (bool) – Whether to zero out infinite losses (caused by input lengths shorter than target lengths). Default True for training stability.

forward(logits, targets, input_lengths, target_lengths)[source]#

Compute the CTC loss.

Parameters:
  • logits (Tensor) – (T, B, num_classes) raw recognizer outputs.

  • targets (Tensor) – Either (B, S_max) padded or (sum(target_lengths),) concatenated label tensor, per nn.CTCLoss convention.

  • input_lengths (Tensor) – (B,) int tensor giving each sequence’s effective length in time-steps.

  • target_lengths (Tensor) – (B,) int tensor giving each target’s length.

Return type:

Tensor

Character sets#

torchocr.load_charset(name, num_classes)[source]#

Load a bundled charset by the name stored in weights.meta["charset"].

Example:

weights = CRNN_ResNet34_VD_Weights.DEFAULT
charset = load_charset(weights.meta["charset"], weights.meta["num_classes"])
Parameters:
  • name (str)

  • num_classes (int)

Return type:

list[str]