nessie.detectors.classification_uncertainty
Module Contents
Classes
Given a distribution over labels for each instance in form of a (num_instances, num_labels) |
- class nessie.detectors.classification_uncertainty.ClassificationUncertainty
Bases:
nessie.detectors.error_detector.ModelBasedDetectorGiven a distribution over labels for each instance in form of a (num_instances, num_labels) numpy array, the resulting score is just 1 - the probability of the noisy label specified.
Halteren, Hans van. “The Detection of Innessie in Manually Tagged Text.” In: Proceedings of the COLING-2000 Workshop on Linguistically Interpreted Corpora, 48–55. Centre Universitaire, Luxembourg: International Committee on Computational Linguistics, 2000. https://www.aclweb.org/anthology/W00-1907.
See also
Dan Hendrycks and Kevin Gimpel A Baseline for Detecting Misclassified and Out-of-Distribution Examples in Neural Networks In: Proceedings of International Conference on Learning Representations
- error_detector_kind(self)
- score(self, labels: nessie.types.StringArray, probabilities: numpy.typing.NDArray[float], le: sklearn.preprocessing.LabelEncoder, **kwargs) numpy.typing.NDArray[float]
Scores the input according to their classification uncertainty.
- Parameters
labels – a (num_instances, ) string sequence containing the noisy label for each instance
probabilities – a (num_instances, num_classes) numpy array obtained from a machine learning model
le – the label encoder that allows converting the probabilities back to labels
- Returns
a (num_instances,) numpy array containing the resulting scores
- Return type
scores
- uses_probabilities(self) bool