nessie.detectors.datamap_confidence

Swayamdipta, Swabha, Roy Schwartz, Nicholas Lourie, Yizhong Wang, Hannaneh Hajishirzi, Noah A. Smith, Yejin Choi. “Dataset Cartography: Mapping and Diagnosing Datasets with Training Dynamics.” Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), 2020.

https://doi.org/10.18653/v1/2020.emnlp-main.746

Module Contents

Classes

DataMapConfidence

DataMapConfidenceCallback

Callback that is called during certain events (starting/ending training, ...)
class nessie.detectors.datamap_confidence.DataMapConfidence(model: nessie.models.model.CallbackableModel, needs_flattening: bool = False)

Bases: nessie.detectors.error_detector.Detector

error_detector_kind(self)
score(self, X: Union[nessie.types.StringArray, nessie.types.StringArray2D], y: Union[nessie.types.StringArray, nessie.types.StringArray2D], **kwargs) → Union[numpy.typing.NDArray[float], awkward.Array]
class nessie.detectors.datamap_confidence.DataMapConfidenceCallback(model: nessie.models.model.Model, X: List[List[str]])

Bases: transformers.TrainerCallback

Callback that is called during certain events (starting/ending training, …)

on_epoch_end(self, args: transformers.TrainingArguments, state: transformers.TrainerState, control: transformers.TrainerControl, **kwargs)
on_train_begin(self, args: transformers.TrainingArguments, state: transformers.TrainerState, control: transformers.TrainerControl, **kwargs)