nessie.detectors.weighted_discrepancy

Module Contents

Classes

WeightedDiscrepancy

Inconsistency detection in semantic annotation
class nessie.detectors.weighted_discrepancy.WeightedDiscrepancy

Bases: nessie.detectors.error_detector.Detector

Inconsistency detection in semantic annotation Nora Hollenstein, Nathan Schneider, Bonnie Webber In: Proceedings of LREC 2016

error_detector_kind(self) → nessie.detectors.error_detector.DetectorKind
score(self, texts: nessie.types.StringArray, labels: nessie.types.StringArray, **kwargs) → numpy.typing.NDArray[float]

Label entropy is computed by collecting labels for instances with the same surface form and then computing the the weighted discrepancy. The lower, the more likely is it that the majority label is correct and that instances with minority labels are wrong. We assign instances with the majority label for its surface form a score of 0.0, as it is likely not wrong.

Parameters

  • texts – a (num_instances, ) string sequence containing the text/surface form of each instance

  • labels – a (num_instances, ) string sequence containing the noisy label for each instance

Returns

a (num_samples, ) numpy array containing the scores for each instance