nessie.detectors.label_aggregation

Module Contents

Classes

LabelAggregation

Uses crowdsourcing aggregation tools to adjudicate labels obtained via Monte-Carlo dropout.
class nessie.detectors.label_aggregation.LabelAggregation

Bases: nessie.detectors.error_detector.ModelBasedDetector

Uses crowdsourcing aggregation tools to adjudicate labels obtained via Monte-Carlo dropout.

Spotting Spurious Data with Neural Networks Hadi Amiri, Timothy A. Miller, Guergana Savova https://aclanthology.org/N18-1182.pdf

correct(self, labels: nessie.types.StringArray, repeated_probabilities: nessie.types.FloatArray2D, le: sklearn.preprocessing.LabelEncoder, **kwargs) → numpy.typing.NDArray[str]
error_detector_kind(self)
needs_multiple_probabilities(self) → bool
score(self, labels: nessie.types.StringArray, repeated_probabilities: nessie.types.FloatArray2D, le: sklearn.preprocessing.LabelEncoder, **kwargs) → numpy.typing.NDArray[bool]

Uses crowdsourcing aggregation tools to adjudicate labels obtained via Monte-Carlo dropout. Flags instances that then disagree with the adjudicated predicions.

Parameters

  • labels – a (num_instances, ) string sequence containing the noisy label for each instance

  • repeated_probabilities – A float array of (num_instances, T, num_classes) which has T label distributions

  • instance (per) –

  • inference. (e.g. as obtained by using different dropout for each prediction run during) –

  • le – the label encoder that allows converting the probabilities back to labels

Returns

a (num_instances,) numpy array of bools containing the flags

supports_correction(self) → bool
uses_probabilities(self) → bool