torchocr#
Pipeline#
- class torchocr.OCRPipeline(detector, recognizer, post_processor, decoder, *, crop_size=None, detector_transforms=None, recognizer_transforms=None, recognizer_batch_size=64, min_text_score=None, orientation=None)[source]#
Bases:
objectEnd-to-end OCR pipeline: image -> populated
DocumentTensor.Composes a detector, post-processor, recognizer, and decoder:
orientation(if given) sets a page scanned sideways or upside down upright; every later step sees the upright page.detector_transforms(if given) resizes/normalizes the image for the detector, e.g.detector_weights.transforms().The post-processor yields text regions. Rotated quads are used when it offers
quadrilaterals()(asDBPostProcessordoes); otherwise its(K, 5) [batch_idx, x1, y1, x2, y2]boxes are used. Regions are mapped back to input-image coordinates.recognizer_transforms.normalize(if given) normalizes the original-resolution image, and every region is rectified out of it withtorchocr.ops.crop_quads()– aspect-preserving, padded, one batched call on the model’s device.The recognizer and decoder turn crops into strings.
Use
from_pretrained()to assemble all four from published weights.- Parameters:
detector (Module) – Module returning a
DBNetOutput.recognizer (Module) – Module mapping
(K, C, H, W)crops to(T, K, num_classes)logits.post_processor (Callable[[DBNetOutput], Tensor]) – Callable mapping a
DBNetOutputto boxes (see above).decoder (Callable[[Tensor], list[str]]) – Callable mapping
(T, K, num_classes)logits to a list of K strings.crop_size (tuple[int, int] | None) –
(height, max_width)of recognizer crops. Default: fromrecognizer_transformswhen given, else(32, 128).detector_transforms (Callable[[Tensor], Tensor] | None) – Optional preset applied to the input image before the detector (e.g.
DetectionPreset).recognizer_transforms (RecognitionPreset | None) – Optional
RecognitionPresetwhosenormalizeis applied to the page before cropping.recognizer_batch_size (int) – Words cropped and recognized per forward pass, bounding memory on dense documents. Default 64.
min_text_score (float | None) – Drop regions whose recognition confidence (
DocumentTensor.scores) is below this value.None(default) keeps every region. 0.8 maximizes ICDAR-2015 training-split end-to-end hmean for the published scene-text models.orientation (PageOrientation | None) – Optional
PageOrientationrun before detection. DefaultNone(pages are read as given).
Without presets the input is fed to both models as-is, so it must already be normalized and sized for the detector (the v0.1 behavior).
- classmethod from_pretrained(detector_weights=None, recognizer_weights=None, device=None, decoder='auto', min_text_score=None, correct_orientation=False)[source]#
Assemble a pipeline from published weights, in eval mode.
Each stage gets its own preprocessing (
weights.transforms()); the post-processor usesdetector_weights.meta["postprocess"]and the decoder the charset named inrecognizer_weights.meta.- Parameters:
detector_weights (WeightsEnum | str | None) – DBNet weights enum member or
"Cls.NAME". DefaultDBNet_MobileNetV3_Large_05_Weights.DEFAULT.recognizer_weights (WeightsEnum | str | None) – CRNN weights enum member or
"Cls.NAME". DefaultCRNN_ResNet34_VD_Weights.DEFAULT.device (device | str | None) – Where to place both models. Default: CPU.
decoder (str) –
"auto"uses the decoder declared inrecognizer_weights.meta["decoder"](e.g. beam search with a language model), else greedy;"greedy"or"beam"force one.min_text_score (float | None) – See
OCRPipeline. DefaultNone(keep all).correct_orientation (bool) – Set pages upright first with
PageOrientation.from_pretrained(). DefaultFalse.
- Return type:
- class torchocr.DocumentTensor(pixels, text=<factory>, bounding_boxes=None, polygons=None, scores=None, rotation=0)[source]#
Bases:
objectContainer for OCR-ready document data.
- Parameters:
- pixels#
The input image, unmodified – or set upright when the pipeline corrects page orientation (see
rotation).- Type:
- bounding_boxes#
(K, 4)axis-aligned(x1, y1, x2, y2)boxes.- Type:
torch.Tensor | None
- polygons#
(K, 4, 2)rotated text quads in reading order (top-left, top-right, bottom-right, bottom-left), when the post-processor provides them.- Type:
torch.Tensor | None
- scores#
(K,)recognition confidence in[0, 1]: the mean probability of the characters read (0 if none).- Type:
torch.Tensor | None
- rotation#
Clockwise angle (0, 90, 180 or 270) the input page was turned by;
pixelsand every region are in the upright frame.- Type:
Page orientation#
- class torchocr.PageOrientation(classifier, transforms, min_confidence=0.0)[source]#
Bases:
ModulePredict how far each page is turned, and turn it back.
Wraps a 4-way classifier (0, 90, 180, 270 degrees) and its preprocessing. A page is only turned when the classifier is confident enough (
min_confidence): most real pages arrive upright, and turning an upright page costs more than leaving a turned one as it is.- Parameters:
classifier (Module) – Module mapping
(B, 3, h, w)preprocessed pages to(B, 4)logits, classcmeaning “turnedc * 90degrees clockwise”.transforms (Module) – Preprocessing for one
(3, H, W)uint8RGB page, e.g.weights.transforms().min_confidence (float) – Smallest softmax probability at which a page is turned; below it the page is kept as it is (angle 0).
- Forward:
(3, H, W)page,(B, 3, H, W)batch or list of pages (sizes may differ),uint8RGB ->(B,)int64clockwise angles in{0, 90, 180, 270}.
Example:
orientation = PageOrientation.from_pretrained() page, angle = orientation.upright(load_image("scan.jpg"))
- classmethod from_pretrained(weights=None, device=None, min_confidence=None)[source]#
Classifier, preset and threshold from published weights, in eval mode.
- Parameters:
weights (WeightsEnum | str | None) –
PPLCNet_x1_0_Weightsmember or name. DefaultPPLCNet_x1_0_Weights.DEFAULT.device (device | str | None) – Where to place the classifier. Default: CPU.
min_confidence (float | None) – Default
weights.meta["min_confidence"].
- Return type:
- forward(pages)[source]#
Define the computation performed at every call.
Should be overridden by all subclasses.
Note
Although the recipe for forward pass needs to be defined within this function, one should call the
Moduleinstance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.
Post-processing#
- class torchocr.DBPostProcessor(threshold=0.3, box_thresh=0.7, max_candidates=1000, unclip_ratio=1.5, min_size=3)[source]#
Bases:
objectExtract per-region bounding boxes from DBNet probability maps.
Implements PaddleOCR’s contour-based DB post-processing flow: threshold the probability map, run
cv2.findContours(), fit a rotated minimum-area rectangle to each contour, score it by averaging the probability map under the rectangle mask, expand the polygon outward byunclip_ratioviapyclipper, and refit the rectangle. Boxes whose score is belowbox_threshor whose minor side is belowmin_sizeare discarded.Calling the processor projects the rotated rectangles to axis-aligned
(x1, y1, x2, y2)boxes (the v0.1 contract);quadrilaterals()returns the rotated quads themselves.The output is shaped
(K, 5)with columns[batch_idx, x1, y1, x2, y2]so it composes directly withtorchvision.ops.roi_align().Kis the total number of accepted boxes across the batch (not capped at one per image). Coordinates are floats in input-pixel units, withx2/y2being inclusive (x2 = x1is a one-pixel-wide column) to match the convention oftorchvision.ops.masks_to_boxes().Note
cv2andpyclipperrun on CPU. GPU probability maps are moved to CPU for processing, then the resulting box tensor is placed back on the original device.- Parameters:
threshold (float) – Probability threshold for binarization. Default 0.3, matching PaddleOCR.
box_thresh (float) – Minimum mean probability under a candidate box for it to be kept. Default 0.7.
max_candidates (int) – Maximum number of contours examined per image. Default 1000.
unclip_ratio (float) – Polygon expansion ratio used by
pyclipper.PyclipperOffset.Execute(). Default 1.5 (matches PaddleOCR v2 weights). Larger values produce looser boxes.min_size (int) – Reject candidates whose minor rotated-rectangle side is below this many pixels. Default 3.
- quadrilaterals(output)[source]#
Return one
(N_b, 4, 2)tensor of rotated text quads per image.Regions are sorted in reading order (
torchocr.ops.reading_order(): lines top to bottom, words left to right). Vertices are(x, y)input-pixel coordinates clipped to the page, ordered top-left, top-right, bottom-right, bottom-left so they feedtorchocr.ops.crop_quads(). Unlike__call__(), the rotation of each region is preserved, which is what polygon-based benchmarks such as ICDAR-2015 score.- Parameters:
output (DBNetOutput)
- Return type:
Decoders#
- class torchocr.CTCGreedyDecoder(charset, blank_index=0)[source]#
Bases:
objectGreedy CTC decoder for CRNN-style logits.
Implements the canonical CTC decoding rule: take the per-step argmax, collapse consecutive duplicate indices (treating blanks as ordinary symbols during the collapse), then remove the remaining blank indices. Padding the recognizer’s input with blanks therefore encodes “same character twice” correctly.
- class torchocr.CTCBeamDecoder(charset, beam_width=10, lm=None, lm_weight=0.3, blank_index=0, top_k=8, blank_skip=0.999)[source]#
Bases:
objectCTC prefix beam search with optional character-LM shallow fusion.
Implements the prefix beam search of Hannun et al. (2014): each prefix tracks the probability of ending in blank and in its last character, so “ll” (separated by a blank) and “l” (one repeated character) stay distinct. When a character is appended its LM log-probability, scaled by
lm_weight, is added; at the end every beam also pays the LM’s end-of-word probability. Frames whose blank probability exceedsblank_skipare treated as pure blank and only thetop_kmost likely characters are expanded per frame, which keeps a pure-Python search fast on short words.- Parameters:
charset (list[str]) – Character strings indexed by class id.
beam_width (int) – Prefixes kept per frame. Default 10.
lm (CharNGramLM | None) – Optional
CharNGramLM.lm_weight (float) – LM scale in log space. Default 0.3.
blank_index (int) – CTC blank class. Default 0.
top_k (int) – Characters expanded per frame. Default 8.
blank_skip (float) – Blank probability above which a frame is skipped. Default 0.999.
- class torchocr.CharNGramLM(counts, order)[source]#
Bases:
objectCase-insensitive character n-gram language model (interpolated Witten-Bell).
P(c | h) = (C(h, c) + T(h) * P(c | h')) / (C(h) + T(h))whereh'drops the oldest character ofh,T(h)counts the distinct continuations ofhand the recursion ends at a uniform distribution overvocabulary(which includes an end-of-word event). Every distribution therefore sums to one and unseen characters keep non-zero mass. Text is lowercased, so the model judges spelling while the optical model keeps choosing case.- Parameters:
- classmethod from_texts(texts, order=5)[source]#
Count n-grams over
texts(one word or line each).- Parameters:
- Return type:
- torchocr.decoders.build_decoder(weights, charset, kind='auto')[source]#
The decoder a recognizer checkpoint was benchmarked with.
weights.meta["decoder"]may declare{"type": "ctc_beam", "beam_width", "lm_weight", "lm_url"}; the language model is downloaded (hash-checked) on first use.- Parameters:
- Return type:
Losses#
- class torchocr.DBLoss(k=50.0, threshold_weight=10.0, binary_weight=1.0, negative_ratio=3.0, eps=1e-06)[source]#
Bases:
ModuleDifferentiable Binarization training loss.
Combines the three components of Liao et al. (AAAI 2020):
Probability-map BCE with online hard example mining (OHEM). OHEM keeps every positive pixel plus the
negative_ratio * n_poshardest negatives, preventing the background class from drowning the loss.Threshold-map L1 loss restricted to the boundary band where the GT threshold is non-zero.
Approximate-binary Dice loss computed on
B_hat = sigmoid(k * (P - T)). The factorkcontrols how sharply this differentiable form approximates the true Heaviside stepP > T: ask -> infinityB_hatbecomes a step function, but its gradient also vanishes away from the decision boundary. The paper’sk = 50is a tested compromise – a tight transition (~0.04units inP - T) that still leaves backprop signal. Lowerkflattens the step toward0.5; higherkstarves the gradients.
Total loss:
Lp + binary_weight * Lb + threshold_weight * Lt.- Parameters:
k (float) – Steepness of the differentiable binarization. Default 50.
threshold_weight (float) – Multiplier on the threshold-map L1 loss. Default 10, matching the paper’s beta.
binary_weight (float) – Multiplier on the dice loss over the approximate binary map. Default 1, matching the paper’s alpha. Set to 0 to disable the binary term entirely.
negative_ratio (float) – Negatives-to-positives ratio for the OHEM sampling in the probability BCE. Default 3.0.
eps (float) – Numerical-stability epsilon for BCE clamping and Dice.
- forward(prediction, gt_probability, gt_threshold, training_mask=None)[source]#
Compute DB loss components and their weighted sum.
- Parameters:
prediction (DBNetOutput) – Output of
DBNetfor one batch.gt_probability (Tensor) –
(B, 1, H, W)0/1 (or soft) text mask.gt_threshold (Tensor) –
(B, 1, H, W)float threshold-map target.training_mask (Tensor | None) – Optional
(B, 1, H, W)boolean tensor of valid pixels.Falseentries are excluded from all three losses (use this to ignore “don’t care” regions such as overlapping or illegible text).
- Returns:
{"loss", "probability", "threshold", "binary"}. Call.backward()on the"loss"entry; log the others.- Return type:
- class torchocr.CRNNLoss(blank_index=0, reduction='mean', zero_infinity=True)[source]#
Bases:
ModuleWrapper around
torch.nn.CTCLossfor CRNN-style logits.Applies
log_softmaxto the recognizer’s logits before delegating tonn.CTCLoss, which expects log probabilities. The blank index defaults to 0, matchingCTCGreedyDecoder.- Parameters:
- forward(logits, targets, input_lengths, target_lengths)[source]#
Compute the CTC loss.
- Parameters:
logits (Tensor) –
(T, B, num_classes)raw recognizer outputs.targets (Tensor) – Either
(B, S_max)padded or(sum(target_lengths),)concatenated label tensor, pernn.CTCLossconvention.input_lengths (Tensor) –
(B,)int tensor giving each sequence’s effective length in time-steps.target_lengths (Tensor) –
(B,)int tensor giving each target’s length.
- Return type: