nessie.detectors.label_entropy

Module Contents

Classes

LabelEntropy

Hollenstein, N, Schneider, N & Webber, B 2016, Innessie detection in semantic annotation. in 10th
class nessie.detectors.label_entropy.LabelEntropy

Bases: nessie.detectors.error_detector.ModelBasedDetector

Hollenstein, N, Schneider, N & Webber, B 2016, Innessie detection in semantic annotation. in 10th edition of the Language Resources and Evaluation Conference. pp. 3986-3990, 10th edition of the Language Resources and Evaluation Conference, Portorož , Slovenia,

error_detector_kind(self) → nessie.detectors.error_detector.DetectorKind
score(self, texts: nessie.types.StringArray, labels: nessie.types.StringArray, **kwargs) → numpy.typing.NDArray[float]

Label entropy is computed by collecting labels for instances with the same surface form and then computing the entropy over this distribution. The lower, the more likely is it that the majority label is correct and that instances with minority labels are wrong. We assign instances with the majority label for its surface form a score of 0.0, as it is likely not wrong.

Parameters

  • texts – a (num_instances, ) string sequence containing the text/surface form of each instance

  • labels – a (num_instances, ) string sequence containing the noisy label for each instance

Returns

a (num_samples, ) numpy array containing the scores for each instance