torchocr.metrics#

class torchocr.metrics.DetectionHmean(iou_threshold=0.5, ignore_threshold=0.5)[source]#

Bases: object

ICDAR-2015 text-detection precision / recall / hmean.

Reproduces the official RRC “IoU” evaluation (the protocol behind every published ICDAR-2015 detection number, and PaddleOCR’s DetectionIoUEvaluator):

  1. Polygons that are not valid (e.g. self-intersecting) are dropped.

  2. A prediction is don’t care if more than ignore_threshold of its own area lies inside a single ignored ground-truth region (### transcriptions in ICDAR files).

  3. Remaining pairs are matched greedily and one-to-one in (ground truth, prediction) order whenever IoU > iou_threshold.

  4. Matches and care counts are summed over all images before the ratios are taken, so large images are not averaged away.

Usage follows the stateful-metric idiom:

metric = DetectionHmean()
for preds, target in loader:
    metric.update(preds, target.polygons, target.ignore)
result = metric.compute()  # HmeanResult
Parameters:
  • iou_threshold (float) – Minimum IoU (exclusive) for a match. Default 0.5.

  • ignore_threshold (float) – Fraction (exclusive) of a prediction’s area that must overlap an ignored region to discard it. Default 0.5.

update(preds, targets, ignore=None)[source]#

Accumulate one image.

Parameters:
  • preds (Tensor) – Predicted regions, either (N, P, 2) polygons with P >= 3 vertices or (N, 4) xyxy boxes.

  • targets (Tensor) – Ground-truth regions in the same formats.

  • ignore (Tensor | None) – Optional (M,) bool mask over targets marking don’t-care regions. Default: nothing ignored.

Return type:

None

class torchocr.metrics.EndToEndHmean(iou_threshold=0.5, ignore_threshold=0.5, case_sensitive=False, alphanumeric_only=True)[source]#

Bases: DetectionHmean

End-to-end text spotting hmean: a detection counts only if it is also read correctly.

Regions are matched exactly as in DetectionHmean; a matched pair is a true positive only when the normalized transcriptions are equal. This follows the ICDAR-2015 end-to-end task with a generic vocabulary (no lexicon), simplified: text equality uses RecognitionAccuracy’s normalization instead of the official script’s special-character rules.

Parameters:
  • iou_threshold (float) – As in DetectionHmean.

  • ignore_threshold (float) – As in DetectionHmean.

  • case_sensitive (bool) – Text normalization before comparison. Defaults follow the scene-text convention (case-insensitive, letters and digits only).

  • alphanumeric_only (bool) – Text normalization before comparison. Defaults follow the scene-text convention (case-insensitive, letters and digits only).

update(preds, targets, ignore, pred_texts, target_texts)[source]#

Accumulate one image; pred_texts / target_texts align with the regions.

Parameters:
Return type:

None

class torchocr.metrics.HmeanResult(precision, recall, hmean, num_matched, num_gt, num_pred)[source]#

Bases: object

Dataset-level detection scores under the ICDAR IoU protocol.

num_gt and num_pred exclude don’t-care regions and invalid polygons; they are the denominators of recall and precision.

Parameters:
class torchocr.metrics.RecognitionAccuracy(case_sensitive=True, alphanumeric_only=False)[source]#

Bases: object

Word accuracy and character error rate over (prediction, target) string pairs.

case_sensitive=False, alphanumeric_only=True is the usual scene-text protocol (ICDAR / IIIT5K / SVT report “case-insensitive word accuracy” over letters and digits).

Parameters:
  • case_sensitive (bool) – Compare strings as-is. Default True.

  • alphanumeric_only (bool) – Drop every character that is not a letter or a digit before comparing. Default False.

class torchocr.metrics.RecognitionResult(word_accuracy, char_error_rate, num_words)[source]#

Bases: object

Dataset-level recognition scores.

char_error_rate is the summed edit distance divided by the summed target length, so long words weigh more than short ones.

Parameters: