nessie.detectors.error_detector

Module Contents

Classes

Detector

DetectorKind

Generic enumeration.

DetectorType

Generic enumeration.

ModelBasedDetector

Helper class that provides a standard way to create an ABC using
class nessie.detectors.error_detector.Detector
__repr__(self)

Return repr(self).

__str__(self)

Return str(self).

correct(self, *args, **kwargs)
abstract error_detector_kind(self) → DetectorKind
name(self) → str
needs_multiple_probabilities(self) → bool
abstract score(self, *args, **kwargs)
supports_correction(self) → bool
uses_probabilities(self) → bool
class nessie.detectors.error_detector.DetectorKind

Bases: enum.Enum

Generic enumeration.

Derive from this class to define new enumerations.

FLAGGER
SCORER
class nessie.detectors.error_detector.DetectorType

Bases: enum.Enum

Generic enumeration.

Derive from this class to define new enumerations.

BASELINE_MAJORITY_LABEL
BASELINE_MAJORITY_LABEL_PER_SURFACE_FORM
BASELINE_RANDOM_FLAGGER
BASELINE_RANDOM_SCORER
BORDA_COUNT
CLASSIFICATION_ENTROPY
CLASSIFICATION_UNCERTAINTY
CONFIDENT_LEARNING
CROSS_WEIGH
CURRICULUM_SPOTTER
DATAMAP_CONFIDENCE
DATAMAP_CONFIDENCE_SEQUENCE
DIVERSE_ENSEMBLE
DROPOUT_UNCERTAINTY
ITEM_RESPONSE_THEORY
KNN_ENTROPY
KNN_FLAGGER
LABEL_AGGREGATION
LABEL_ENTROPY
LEITNER_SPOTTER
MEAN_DISTANCE
PREDICTION_MARGIN
PROJECTION_ENSEMBLE
RETAG
UNIFORM_ENSEMBLE
VARIATION_PRINCIPLE
VARIATION_PRINCIPLE_SPAN
WEIGHTED_DISCREPANCY
class nessie.detectors.error_detector.ModelBasedDetector

Bases: Detector, abc.ABC

Helper class that provides a standard way to create an ABC using inheritance.

abstract score(self, texts: nessie.types.StringArray, labels: nessie.types.StringArray, predictions: nessie.types.StringArray, probabilities: nessie.types.FloatArray2D, repeated_probabilities: nessie.types.FloatArray2D, confidences_over_time: numpy.typing.NDArray[float], le: sklearn.preprocessing.LabelEncoder, **kwargs) → numpy.ndarray
Parameters

  • texts –

  • labels – Array of strings, these are not encoded

  • predictions – Array of strings, these are not encoded

  • probabilities – ndarray of shape (n_predictions, n_classes)

  • repeated_probabilities – ndarray of shape (n_predictions, T, n_classes) repeatedly sampled probabilities

  • confidences_over_time – ndarray of shape (n_predictions, T, n_classes)

  • le – label encoder that can be used to map the numeric labels in probabilities to string labels

  • **kwargs –

Returns: