.. DO NOT EDIT. .. THIS FILE WAS AUTOMATICALLY GENERATED BY SPHINX-GALLERY. .. TO MAKE CHANGES, EDIT THE SOURCE PYTHON FILE: .. "auto_examples/plot_pipeline_anatomy.py" .. LINE NUMBERS ARE GIVEN BELOW. .. only:: html .. note:: :class: sphx-glr-download-link-note :ref:`Go to the end ` to download the full example code. .. rst-class:: sphx-glr-example-title .. _sphx_glr_auto_examples_plot_pipeline_anatomy.py: Anatomy of the pipeline ======================= :class:`~torchocr.OCRPipeline` is four swappable stages: ``detector → post-processor → crop_quads → recognizer → decoder`` This example runs them one by one, so each can be inspected or replaced. .. GENERATED FROM PYTHON SOURCE LINES 13-27 .. code-block:: Python from pathlib import Path import matplotlib.pyplot as plt import torch from torchocr import CTCGreedyDecoder, DBPostProcessor, load_charset from torchocr.decoders import build_decoder from torchocr.io import load_image from torchocr.models import CRNN, DBNet, CRNN_ResNet34_VD_Weights, DBNet_ResNet18_VD_Weights from torchocr.ops import crop_quads ASSETS = Path("../examples") # run from the gallery/ directory, as sphinx-gallery does image = load_image(ASSETS / "english_doc.jpg") .. GENERATED FROM PYTHON SOURCE LINES 28-32 1. Detection ------------ The detector predicts a per-pixel text probability map. Its weights entry provides the preprocessing it was trained with (resize to multiples of 32, normalization). .. GENERATED FROM PYTHON SOURCE LINES 32-47 .. code-block:: Python det_weights = DBNet_ResNet18_VD_Weights.ICDAR2015 detector = DBNet(weights=det_weights).eval() detector_input = det_weights.transforms()(image)[None] with torch.inference_mode(): detection = detector(detector_input) # DBNetOutput(probability, threshold) fig, (left, right) = plt.subplots(1, 2, figsize=(12, 3.5)) left.imshow(image.permute(1, 2, 0)) right.imshow(detection.probability[0, 0], cmap="magma") for ax, title in ((left, "input"), (right, "detection.probability")): ax.set_title(title) ax.set_axis_off() plt.tight_layout() .. image-sg:: /auto_examples/images/sphx_glr_plot_pipeline_anatomy_001.png :alt: input, detection.probability :srcset: /auto_examples/images/sphx_glr_plot_pipeline_anatomy_001.png :class: sphx-glr-single-img .. GENERATED FROM PYTHON SOURCE LINES 48-53 2. Post-processing ------------------ :class:`~torchocr.DBPostProcessor` turns the probability map into rotated quadrilaterals, sorted in reading order (:func:`torchocr.ops.reading_order`). They are in the resized image's pixels; scale them back to the original. .. GENERATED FROM PYTHON SOURCE LINES 53-60 .. code-block:: Python postprocess = DBPostProcessor(**det_weights.meta["postprocess"]) quads = postprocess.quadrilaterals(detection)[0] scale = torch.tensor([image.shape[-1] / detector_input.shape[-1], image.shape[-2] / detector_input.shape[-2]]) quads = quads * scale print(f"{len(quads)} regions; first quad:\n{quads[0]}") .. rst-class:: sphx-glr-script-out .. code-block:: none 10 regions; first quad: tensor([[ 29.0000, 18.4091], [108.0000, 18.4091], [108.0000, 47.0455], [ 29.0000, 47.0455]]) .. GENERATED FROM PYTHON SOURCE LINES 61-65 3. Rectified crops ------------------ :func:`torchocr.ops.crop_quads` warps every quad into an axis-aligned strip of fixed height with one batched ``grid_sample`` call, undoing rotation and perspective. .. GENERATED FROM PYTHON SOURCE LINES 65-78 .. code-block:: Python rec_weights = CRNN_ResNet34_VD_Weights.MJSYNTH_ICDAR2015_TOTALTEXT_MLT19 preset = rec_weights.transforms() crops, widths = crop_quads(preset.normalize(image)[None].float(), quads, torch.zeros(len(quads), dtype=torch.long), height=32, max_width=320) print(crops.shape) # (K, 3, 32, W) raw, _ = crop_quads(image[None].float(), quads, torch.zeros(len(quads), dtype=torch.long), height=32, max_width=320) fig, axes = plt.subplots(len(raw), 1, figsize=(6, 0.5 * len(raw))) for ax, crop in zip(axes, raw): ax.imshow(crop.permute(1, 2, 0).clamp(0, 255).byte()) ax.set_axis_off() .. image-sg:: /auto_examples/images/sphx_glr_plot_pipeline_anatomy_002.png :alt: plot pipeline anatomy :srcset: /auto_examples/images/sphx_glr_plot_pipeline_anatomy_002.png :class: sphx-glr-single-img .. rst-class:: sphx-glr-script-out .. code-block:: none torch.Size([10, 3, 32, 320]) .. GENERATED FROM PYTHON SOURCE LINES 79-83 4. Recognition and decoding --------------------------- The recognizer emits per-column character logits for CTC. The weights entry declares its decoder: beam search with a character language model. Greedy decoding is the cheap baseline. .. GENERATED FROM PYTHON SOURCE LINES 83-92 .. code-block:: Python recognizer = CRNN(weights=rec_weights).eval() with torch.inference_mode(): logits = recognizer(crops) # (T, K, num_classes) charset = load_charset(rec_weights.meta["charset"], rec_weights.meta["num_classes"]) greedy = CTCGreedyDecoder(charset)(logits) beam = build_decoder(rec_weights, charset)(logits) # what the pipeline uses for g, b in zip(greedy, beam): print(f"{g:>14} {b:>14}" + (" <- the language model changed this word" if g != b else "")) .. rst-class:: sphx-glr-script-out .. code-block:: none Please Please lower lower your your volume volume when when you you pass pass by by residential residential creas creas .. rst-class:: sphx-glr-timing **Total running time of the script:** (0 minutes 2.829 seconds) .. _sphx_glr_download_auto_examples_plot_pipeline_anatomy.py: .. only:: html .. container:: sphx-glr-footer sphx-glr-footer-example .. container:: sphx-glr-download sphx-glr-download-jupyter :download:`Download Jupyter notebook: plot_pipeline_anatomy.ipynb ` .. container:: sphx-glr-download sphx-glr-download-python :download:`Download Python source code: plot_pipeline_anatomy.py ` .. container:: sphx-glr-download sphx-glr-download-zip :download:`Download zipped: plot_pipeline_anatomy.zip ` .. only:: html .. rst-class:: sphx-glr-signature `Gallery generated by Sphinx-Gallery `_