nessie.detectors.borda_count

Module Contents

Classes

BordaCount

Aggregate ranking scores via Borda count. Given a matrix of kxn instances,

class nessie.detectors.borda_count.BordaCount

Bases: nessie.detectors.error_detector.Detector

Aggregate ranking scores via Borda count. Given a matrix of kxn instances, where k is the number of scorers and n the number of instances, for each scorer, assign the highest rank a score of n, the second largest n-1 and so on. Then sum up the scores and rank for the newly computed scores.

This has been described first in

Inconsistencies in Crowdsourced Slot-Filling Annotations: A Typology and Identification Methods Stefan Larson, Adrian Cheung, Anish Mahendran, Kevin Leach, Jonathan K. Kummerfeld COLING 2020

error_detector_kind(self) → nessie.detectors.error_detector.DetectorKind

score(self, ensemble_scores: numpy.typing.NDArray[float]) → numpy.typing.NDArray[float]

Aggregates the given ensemble scores obtained previously from several scorers into one by means of Borda count.

Parameters

ensemble_scores – a (num_scorers, num_samples) numpy array

Returns

a (num_samples,) numpy array containing the aggregated scores

Return type

scores

uses_probabilities(self) → bool