nessie.detectors.leitner_spotter
Module Contents
Classes
Spotting Spurious Data with Neural Networks |
|
Callback that is called during certain events (starting/ending training, ...) |
|
Attributes
- nessie.detectors.leitner_spotter.Queues
- class nessie.detectors.leitner_spotter.LeitnerSpotter(verbose: bool = True, max_epochs: int = 48)
Bases:
nessie.detectors.error_detector.Detector
Spotting Spurious Data with Neural Networks
Hadi Amiri, Timothy A. Miller, Guergana Savova https://aclanthology.org/N18-1182.pdf
- error_detector_kind(self) nessie.detectors.error_detector.DetectorKind
- score(self, texts: nessie.types.StringArray, labels: nessie.types.StringArray, **kwargs) numpy.typing.NDArray[float]
Apply curriculum learning via Zettelkasten and score the items by the perceived difficulty during the curriculum training.
- Parameters
texts – a (num_instances, ) string sequence containing the texts for each instance
labels – a (num_instances, ) string sequence containing the noisy label for each instance
- Returns
a (num_instances,) numpy array of bools containing the flags after using CS
- Return type
scores
- class nessie.detectors.leitner_spotter.LeitnerSpotterDataset(tokenized_texts: Dict, encoded_labels: List[int])
Bases:
torch.utils.data.Dataset
- property X(self)
- __getitem__(self, idx: int)
- __len__(self)
- property true_len(self)
- update_mapping(self, new_dataset_mask: numpy.ndarray)
- property y(self)
- class nessie.detectors.leitner_spotter.LeitnerSpotterDatasetCallback(model: nessie.models.text.transformer_text_classifier.TransformerTextClassifier, number_of_queues: int = 5)
Bases:
transformers.TrainerCallback
Callback that is called during certain events (starting/ending training, …)
- on_epoch_begin(self, args: transformers.TrainingArguments, state: transformers.TrainerState, control: transformers.TrainerControl, **kwargs)
- on_epoch_end(self, args: transformers.TrainingArguments, state: transformers.TrainerState, control: transformers.TrainerControl, **kwargs)
- on_train_begin(self, args: transformers.TrainingArguments, state: transformers.TrainerState, control: transformers.TrainerControl, **kwargs)
- on_train_end(self, args: transformers.TrainingArguments, state: transformers.TrainerState, control: transformers.TrainerControl, **kwargs)
- class nessie.detectors.leitner_spotter.LeitnerSpotterTransformerTextClassifier
Bases:
nessie.models.text.transformer_text_classifier.TransformerTextClassifier