medkit.audio.metrics.diarization#

This module needs extra-dependencies not installed as core dependencies of medkit. To install them, use pip install medkit-lib[metrics-diarization].

Classes:

DiarizationEvaluator([turn_label, ...])

Diarization Error Rate (DER) computation based on pyannote.

DiarizationEvaluatorResult(der, confusion, ...)

Results returned by DiarizationEvaluator
class DiarizationEvaluator(turn_label='turn', speaker_label='speaker', collar=0.0)[source]#

Diarization Error Rate (DER) computation based on pyannote.

The DER is the ratio of time that is not attributed correctly because of one of the following errors:

  • detected as non-speech when there was speech (missed detection);

  • detected as speech where there was none (false alarm);

  • attributed to the wrong speaker (confusion).

This component expects as input reference documents containing the reference speech turn segments as well as corresponding predicted speech turn segments. The speech turn segments must each have a speaker attribute.

Note that values of the reference and predicted speaker attributes (ie speaker labels) don’t have to be the same, since they will be optimally remapped using the Hungarian algorithm.

Parameters

  • turn_label (str) – Label of the turn segments on the reference documents

  • speaker_label (str) – Label of the speaker attributes on the reference and predicted turn segments

  • collar (float) – Margin of error (in seconds) around start and end of reference segments

Methods:

compute(reference, predicted)

Compute and return the DER for predicted speech turn segments, against reference annotated documents.
compute(reference, predicted)[source]#

Compute and return the DER for predicted speech turn segments, against reference annotated documents.

Parameters

  • reference (Sequence[AudioDocument]) – Reference documents containing speech turns segments with turn_label as label, each of them containing a speaker attribute with speaker_label as label.

  • predicted (Sequence[Sequence[Segment]]) – Predicted segments containing each a speaker attribute with speaker_label as label. This is a list of list that must be of the same length and ordering as reference.

Return type

DiarizationEvaluatorResult

Returns

DiarizationEvaluatorResult – Computed metrics

class DiarizationEvaluatorResult(der, confusion, false_alarm, missed_detection, total_speech, support)[source]#

Results returned by DiarizationEvaluator

Variables

  • der (float) – Diarization Error Rate, combination of confusion, false alarm and missed detection

  • confusion (float) – Ratio of time detected as speech but attributed to a wrong speaker (over total_speech)

  • false_alarm (float) – Ratio of time corresponding to non-speech mistakenly detected as speech (over total_speech)

  • missed_detection (float) – Ratio of time corresponding to undetected speech (over total_speech)

  • total_speech (float) – Total duration of speech in the reference

  • support (float) – Total duration of audio