nessie.detectors.dropout_uncertainty

Module Contents

Classes

DropoutUncertainty

Compute Uncertainty via Monte Carlo Dropout.
class nessie.detectors.dropout_uncertainty.DropoutUncertainty

Bases: nessie.detectors.error_detector.Detector

Compute Uncertainty via Monte Carlo Dropout. This first has been proposed in:

Spotting Spurious Data with Neural Networks MHadi Amiri, Timothy A. Miller, Guergana Savova https://aclanthology.org/N18-1182.pdf https://arxiv.org/abs/1703.00410

Also see: How Certain is Your Transformer? Artem Shelmanov, Evgenii Tsymbalov, Dmitri Puzyrev, Kirill Fedyanin, Alexander Panchenko, Maxim Panov EACL 2021

error_detector_kind(self)
needs_multiple_probabilities(self) → bool
score(self, repeated_probabilities: nessie.types.FloatArray2D, **kwargs) → numpy.typing.NDArray[float]

Given probabilities obtained via Monte Carlo Dropout, compute the score via the entropy of the model predictions.

Parameters

  • repeated_probabilities – A float array of (num_instances, T, num_classes) which has T label distributions

  • instance (per) –

  • inference. (e.g. as obtained by using different dropout for each prediction run during) –

Returns

a (num_instances,) numpy array of bools containing the scores after running DU

Return type

scores

uses_probabilities(self) → bool