Getting started#

Installation#

pip install torchocr

or, from source, git clone https://github.com/BryanBradfo/torchOCR.git && pip install -e torchOCR.

torchocr needs PyTorch 2 and torchvision 0.23 or later. pip install torchocr[lmdb] adds the LMDB word-corpus reader; from source, .[test] adds the test suite and .[docs] this site.

Image to text#

OCRPipeline.from_pretrained takes a detector and a recognizer weights entry, downloads both (SHA-256 checked, cached by torch.hub), and wires each model’s preprocessing, post-processing and decoder from its weights.meta:

import torch
from torchocr import OCRPipeline
from torchocr.io import load_image

device = "cuda" if torch.cuda.is_available() else "cpu"
pipeline = OCRPipeline.from_pretrained(
    "DBNet_ResNet18_VD_Weights.ICDAR2015",
    "CRNN_ResNet34_VD_Weights.MJSYNTH_ICDAR2015_TOTALTEXT_MLT19",
    device=device,
    min_text_score=0.8,  # drop regions read with low confidence
)
document = pipeline(load_image("street.jpg"))

document.text            # list[str], one per region, in reading order
document.polygons        # (K, 4, 2) rotated quads
document.bounding_boxes  # (K, 4) xyxy
document.scores          # (K,) recognition confidence

Building blocks#

Every stage can be used, replaced or fine-tuned on its own. The detection half of the pipeline:

import torch
from torchocr import DBPostProcessor
from torchocr.io import load_image
from torchocr.models import DBNet, DBNet_ResNet18_VD_Weights

weights = DBNet_ResNet18_VD_Weights.ICDAR2015
detector = DBNet(weights=weights).eval()
images = weights.transforms()(load_image("street.jpg"))[None]  # (1, 3, H, W), H and W multiples of 32
with torch.no_grad():
    output = detector(images)                                  # DBNetOutput(probability, threshold)
postprocess = DBPostProcessor(**weights.meta["postprocess"])
quads = postprocess.quadrilaterals(output)[0]                  # (K, 4, 2), in the resized image's pixels

Models are FX-traceable and work with torch.compile and torch.export.

Training#

The references/ directory holds the scripts that trained and evaluated every published checkpoint, in the spirit of torchvision’s reference scripts: references/detection, references/recognition and references/end2end. Each script’s --help documents a complete example command.