sadda.align¶
Phone-level forced alignment: time-align a transcript to audio, producing Word and Phone results with IPA labels. PROVISIONAL tier.
For a task-oriented walk-through, see the Forced alignment guide.
Requires the espeak-ng system binary (G2P) and pip install "sadda[align]"
(ONNX Runtime + huggingface_hub); the acoustic model is fetched from the Hub on
first use.
Alignment¶
align ¶
align(audio: ndarray, sample_rate: int, transcript: str, *, model: AcousticModel, voice: str = 'en-us', detector: Optional[str] = 'blank', min_silence_seconds: float = 0.2) -> Alignment
Force-align transcript to audio with model.
Phonemizes the transcript (espeak-ng, voice), gets per-frame posteriors
from model, tokenizes each word against the model vocab, runs the CTC
forced-align DP, and returns time-aligned words and phones.
Silence (detector): "blank" (default) marks CTC-blank runs at least
min_silence_seconds long as silence; "vad" uses Silero VAD
(:mod:sadda.ml); None disables it. Detected silence becomes
empty-labeled intervals — the Word and Phone results stay contiguous (a
full partition of the recording), with pauses and edge silence left empty
rather than absorbed into neighbouring words.
The min_silence_seconds default (0.20 s) is grounded in the pause
literature: above typical stop-closure durations, between Praat's 0.1 s
silence-detector default (To TextGrid (silences),
https://www.fon.hum.uva.nl/praat/manual/Sound__To_TextGrid__silences____.html)
and Goldman-Eisler's (1968) 0.25 s articulatory-vs-hesitation-pause boundary
(Psycholinguistics: Experiments in Spontaneous Speech, Academic Press — no
stable weblink available).
Alignment ¶
A forced alignment: aligned words and the flat phone sequence.
__annotations__
class-attribute
¶
dict() -> new empty dictionary dict(mapping) -> new dictionary initialized from a mapping object's (key, value) pairs dict(iterable) -> new dictionary initialized as if via: d = {} for k, v in iterable: d[k] = v dict(**kwargs) -> new dictionary initialized with the name=value pairs in the keyword argument list. For example: dict(one=1, two=2)
__dataclass_fields__
class-attribute
¶
__dataclass_fields__ = {'words': Field(name='words',type='tuple[TimedWord, ...]',default=<dataclasses._MISSING_TYPE object at 0x7f0dcc0ee060>,default_factory=<dataclasses._MISSING_TYPE object at 0x7f0dcc0ee060>,init=True,repr=True,hash=None,compare=True,metadata=mappingproxy({}),kw_only=False,_field_type=_FIELD), 'phones': Field(name='phones',type='tuple[TimedPhone, ...]',default=<dataclasses._MISSING_TYPE object at 0x7f0dcc0ee060>,default_factory=<dataclasses._MISSING_TYPE object at 0x7f0dcc0ee060>,init=True,repr=True,hash=None,compare=True,metadata=mappingproxy({}),kw_only=False,_field_type=_FIELD)}
dict() -> new empty dictionary dict(mapping) -> new dictionary initialized from a mapping object's (key, value) pairs dict(iterable) -> new dictionary initialized as if via: d = {} for k, v in iterable: d[k] = v dict(**kwargs) -> new dictionary initialized with the name=value pairs in the keyword argument list. For example: dict(one=1, two=2)
__doc__
class-attribute
¶
str(object='') -> str str(bytes_or_buffer[, encoding[, errors]]) -> str
Create a new string object from the given object. If encoding or errors is specified, then the object must expose a data buffer that will be decoded using the given encoding and error handler. Otherwise, returns the result of object.str() (if defined) or repr(object). encoding defaults to sys.getdefaultencoding(). errors defaults to 'strict'.
__match_args__
class-attribute
¶
Built-in immutable sequence.
If no argument is given, the constructor returns an empty tuple. If iterable is specified the tuple is initialized from iterable's items.
If the argument is a tuple, the return value is the same object.
__module__
class-attribute
¶
str(object='') -> str str(bytes_or_buffer[, encoding[, errors]]) -> str
Create a new string object from the given object. If encoding or errors is specified, then the object must expose a data buffer that will be decoded using the given encoding and error handler. Otherwise, returns the result of object.str() (if defined) or repr(object). encoding defaults to sys.getdefaultencoding(). errors defaults to 'strict'.
TimedWord ¶
One aligned word: its span and the phones inside it.
__annotations__
class-attribute
¶
__annotations__ = {'text': 'str', 'start_seconds': 'float', 'end_seconds': 'float', 'phones': 'tuple[TimedPhone, ...]'}
dict() -> new empty dictionary dict(mapping) -> new dictionary initialized from a mapping object's (key, value) pairs dict(iterable) -> new dictionary initialized as if via: d = {} for k, v in iterable: d[k] = v dict(**kwargs) -> new dictionary initialized with the name=value pairs in the keyword argument list. For example: dict(one=1, two=2)
__dataclass_fields__
class-attribute
¶
__dataclass_fields__ = {'text': Field(name='text',type='str',default=<dataclasses._MISSING_TYPE object at 0x7f0dcc0ee060>,default_factory=<dataclasses._MISSING_TYPE object at 0x7f0dcc0ee060>,init=True,repr=True,hash=None,compare=True,metadata=mappingproxy({}),kw_only=False,_field_type=_FIELD), 'start_seconds': Field(name='start_seconds',type='float',default=<dataclasses._MISSING_TYPE object at 0x7f0dcc0ee060>,default_factory=<dataclasses._MISSING_TYPE object at 0x7f0dcc0ee060>,init=True,repr=True,hash=None,compare=True,metadata=mappingproxy({}),kw_only=False,_field_type=_FIELD), 'end_seconds': Field(name='end_seconds',type='float',default=<dataclasses._MISSING_TYPE object at 0x7f0dcc0ee060>,default_factory=<dataclasses._MISSING_TYPE object at 0x7f0dcc0ee060>,init=True,repr=True,hash=None,compare=True,metadata=mappingproxy({}),kw_only=False,_field_type=_FIELD), 'phones': Field(name='phones',type='tuple[TimedPhone, ...]',default=<dataclasses._MISSING_TYPE object at 0x7f0dcc0ee060>,default_factory=<dataclasses._MISSING_TYPE object at 0x7f0dcc0ee060>,init=True,repr=True,hash=None,compare=True,metadata=mappingproxy({}),kw_only=False,_field_type=_FIELD)}
dict() -> new empty dictionary dict(mapping) -> new dictionary initialized from a mapping object's (key, value) pairs dict(iterable) -> new dictionary initialized as if via: d = {} for k, v in iterable: d[k] = v dict(**kwargs) -> new dictionary initialized with the name=value pairs in the keyword argument list. For example: dict(one=1, two=2)
__doc__
class-attribute
¶
str(object='') -> str str(bytes_or_buffer[, encoding[, errors]]) -> str
Create a new string object from the given object. If encoding or errors is specified, then the object must expose a data buffer that will be decoded using the given encoding and error handler. Otherwise, returns the result of object.str() (if defined) or repr(object). encoding defaults to sys.getdefaultencoding(). errors defaults to 'strict'.
__match_args__
class-attribute
¶
Built-in immutable sequence.
If no argument is given, the constructor returns an empty tuple. If iterable is specified the tuple is initialized from iterable's items.
If the argument is a tuple, the return value is the same object.
__module__
class-attribute
¶
str(object='') -> str str(bytes_or_buffer[, encoding[, errors]]) -> str
Create a new string object from the given object. If encoding or errors is specified, then the object must expose a data buffer that will be decoded using the given encoding and error handler. Otherwise, returns the result of object.str() (if defined) or repr(object). encoding defaults to sys.getdefaultencoding(). errors defaults to 'strict'.
TimedPhone ¶
One aligned phone with its time span and (optional) confidence.
score is the aligner's per-phone confidence where it provides one (the
neural DP does); it is None for backends that don't emit one (e.g. MFA's
TextGrid output).
__annotations__
class-attribute
¶
__annotations__ = {'label': 'str', 'start_seconds': 'float', 'end_seconds': 'float', 'score': 'Optional[float]'}
dict() -> new empty dictionary dict(mapping) -> new dictionary initialized from a mapping object's (key, value) pairs dict(iterable) -> new dictionary initialized as if via: d = {} for k, v in iterable: d[k] = v dict(**kwargs) -> new dictionary initialized with the name=value pairs in the keyword argument list. For example: dict(one=1, two=2)
__dataclass_fields__
class-attribute
¶
__dataclass_fields__ = {'label': Field(name='label',type='str',default=<dataclasses._MISSING_TYPE object at 0x7f0dcc0ee060>,default_factory=<dataclasses._MISSING_TYPE object at 0x7f0dcc0ee060>,init=True,repr=True,hash=None,compare=True,metadata=mappingproxy({}),kw_only=False,_field_type=_FIELD), 'start_seconds': Field(name='start_seconds',type='float',default=<dataclasses._MISSING_TYPE object at 0x7f0dcc0ee060>,default_factory=<dataclasses._MISSING_TYPE object at 0x7f0dcc0ee060>,init=True,repr=True,hash=None,compare=True,metadata=mappingproxy({}),kw_only=False,_field_type=_FIELD), 'end_seconds': Field(name='end_seconds',type='float',default=<dataclasses._MISSING_TYPE object at 0x7f0dcc0ee060>,default_factory=<dataclasses._MISSING_TYPE object at 0x7f0dcc0ee060>,init=True,repr=True,hash=None,compare=True,metadata=mappingproxy({}),kw_only=False,_field_type=_FIELD), 'score': Field(name='score',type='Optional[float]',default=None,default_factory=<dataclasses._MISSING_TYPE object at 0x7f0dcc0ee060>,init=True,repr=True,hash=None,compare=True,metadata=mappingproxy({}),kw_only=False,_field_type=_FIELD)}
dict() -> new empty dictionary dict(mapping) -> new dictionary initialized from a mapping object's (key, value) pairs dict(iterable) -> new dictionary initialized as if via: d = {} for k, v in iterable: d[k] = v dict(**kwargs) -> new dictionary initialized with the name=value pairs in the keyword argument list. For example: dict(one=1, two=2)
__doc__
class-attribute
¶
__doc__ = "One aligned phone with its time span and (optional) confidence.\n\n ``score`` is the aligner's per-phone confidence where it provides one (the\n neural DP does); it is ``None`` for backends that don't emit one (e.g. MFA's\n TextGrid output).\n "
str(object='') -> str str(bytes_or_buffer[, encoding[, errors]]) -> str
Create a new string object from the given object. If encoding or errors is specified, then the object must expose a data buffer that will be decoded using the given encoding and error handler. Otherwise, returns the result of object.str() (if defined) or repr(object). encoding defaults to sys.getdefaultencoding(). errors defaults to 'strict'.
__match_args__
class-attribute
¶
Built-in immutable sequence.
If no argument is given, the constructor returns an empty tuple. If iterable is specified the tuple is initialized from iterable's items.
If the argument is a tuple, the return value is the same object.
__module__
class-attribute
¶
str(object='') -> str str(bytes_or_buffer[, encoding[, errors]]) -> str
Create a new string object from the given object. If encoding or errors is specified, then the object must expose a data buffer that will be decoded using the given encoding and error handler. Otherwise, returns the result of object.str() (if defined) or repr(object). encoding defaults to sys.getdefaultencoding(). errors defaults to 'strict'.
tokenize ¶
Greedy longest-match an IPA string into vocab class ids.
At each position, the longest vocab key that matches wins (so dʒ beats
d when both are in the vocab). Whitespace is skipped. Raises
:class:ValueError naming the offending substring if no key matches — which
is how a phone the acoustic model doesn't cover surfaces.
Grapheme-to-phoneme (espeak-ng)¶
phonemize ¶
Phonemize a transcript to per-word IPA phones via espeak-ng.
Words are phonemized individually (a clean 1:1 word→phones mapping, at the
cost of cross-word coarticulation — acceptable for an alignment target).
Edge punctuation is stripped; stress marks are removed. voice is an
espeak-ng language id (e.g. "en-us", "de", "cmn").
Utterance ¶
A phonemized transcript: an ordered list of :class:Word\ s.
Word ¶
One word and its phonemization.
Attributes:
-
text–The word as written (edge punctuation removed).
-
ipa–The espeak-ng IPA string for the word, stress marks stripped.
-
phones–ipasegmented into individual IPA phones (base character + its trailing length / tie / diacritic modifiers). The aligner reconciles these against the acoustic model's vocabulary.
__annotations__
class-attribute
¶
dict() -> new empty dictionary dict(mapping) -> new dictionary initialized from a mapping object's (key, value) pairs dict(iterable) -> new dictionary initialized as if via: d = {} for k, v in iterable: d[k] = v dict(**kwargs) -> new dictionary initialized with the name=value pairs in the keyword argument list. For example: dict(one=1, two=2)
__dataclass_fields__
class-attribute
¶
__dataclass_fields__ = {'text': Field(name='text',type='str',default=<dataclasses._MISSING_TYPE object at 0x7f0dcc0ee060>,default_factory=<dataclasses._MISSING_TYPE object at 0x7f0dcc0ee060>,init=True,repr=True,hash=None,compare=True,metadata=mappingproxy({}),kw_only=False,_field_type=_FIELD), 'ipa': Field(name='ipa',type='str',default=<dataclasses._MISSING_TYPE object at 0x7f0dcc0ee060>,default_factory=<dataclasses._MISSING_TYPE object at 0x7f0dcc0ee060>,init=True,repr=True,hash=None,compare=True,metadata=mappingproxy({}),kw_only=False,_field_type=_FIELD), 'phones': Field(name='phones',type='tuple[str, ...]',default=<dataclasses._MISSING_TYPE object at 0x7f0dcc0ee060>,default_factory=<dataclasses._MISSING_TYPE object at 0x7f0dcc0ee060>,init=True,repr=True,hash=None,compare=True,metadata=mappingproxy({}),kw_only=False,_field_type=_FIELD)}
dict() -> new empty dictionary dict(mapping) -> new dictionary initialized from a mapping object's (key, value) pairs dict(iterable) -> new dictionary initialized as if via: d = {} for k, v in iterable: d[k] = v dict(**kwargs) -> new dictionary initialized with the name=value pairs in the keyword argument list. For example: dict(one=1, two=2)
__doc__
class-attribute
¶
__doc__ = "One word and its phonemization.\n\n Attributes:\n text: The word as written (edge punctuation removed).\n ipa: The espeak-ng IPA string for the word, stress marks stripped.\n phones: ``ipa`` segmented into individual IPA phones (base character +\n its trailing length / tie / diacritic modifiers). The aligner\n reconciles these against the acoustic model's vocabulary.\n "
str(object='') -> str str(bytes_or_buffer[, encoding[, errors]]) -> str
Create a new string object from the given object. If encoding or errors is specified, then the object must expose a data buffer that will be decoded using the given encoding and error handler. Otherwise, returns the result of object.str() (if defined) or repr(object). encoding defaults to sys.getdefaultencoding(). errors defaults to 'strict'.
__match_args__
class-attribute
¶
Built-in immutable sequence.
If no argument is given, the constructor returns an empty tuple. If iterable is specified the tuple is initialized from iterable's items.
If the argument is a tuple, the return value is the same object.
__module__
class-attribute
¶
str(object='') -> str str(bytes_or_buffer[, encoding[, errors]]) -> str
Create a new string object from the given object. If encoding or errors is specified, then the object must expose a data buffer that will be decoded using the given encoding and error handler. Otherwise, returns the result of object.str() (if defined) or repr(object). encoding defaults to sys.getdefaultencoding(). errors defaults to 'strict'.
split_phones ¶
Segment an IPA string into phones.
A phone is a base character plus any trailing modifiers that bind to it:
combining diacritics (Unicode Mn, including the affricate/diphthong tie),
and modifier letters (Lm — length ː, aspiration ʰ, palatalization
ʲ, …). A tie makes the following base join the current phone (t͡ʃ → one
phone); untied sequences split (tʃ → t, ʃ), matching how espeak-ng
emits them.
Acoustic model¶
The AcousticModel protocol¶
Any object with an emissions(audio, sample_rate) method returning
Emissions is an acoustic model — a structural,
runtime-checkable protocol, so no subclassing is required (a custom model or a
test mock both qualify).
class AcousticModel(Protocol):
def emissions(self, audio: np.ndarray, sample_rate: int) -> Emissions:
"""Per-frame CTC log-probabilities over a phone vocabulary."""
Emissions ¶
Per-frame acoustic posteriors for one audio clip.
Attributes:
-
log_probs–(T, C)float32 array of log-probabilities over theCclasses (phones + blank), one row per frame. -
vocab–Maps each phone label to its class id (column of
log_probs). -
frame_rate–Emission frames per second (e.g. 50.0 for a wav2vec2 CTC head with a 20 ms stride) — used to convert frame spans to seconds.
-
blank_id–The CTC blank class id.
__annotations__
class-attribute
¶
__annotations__ = {'log_probs': 'np.ndarray', 'vocab': 'Mapping[str, int]', 'frame_rate': 'float', 'blank_id': 'int'}
dict() -> new empty dictionary dict(mapping) -> new dictionary initialized from a mapping object's (key, value) pairs dict(iterable) -> new dictionary initialized as if via: d = {} for k, v in iterable: d[k] = v dict(**kwargs) -> new dictionary initialized with the name=value pairs in the keyword argument list. For example: dict(one=1, two=2)
__dataclass_fields__
class-attribute
¶
__dataclass_fields__ = {'log_probs': Field(name='log_probs',type='np.ndarray',default=<dataclasses._MISSING_TYPE object at 0x7f0dcc0ee060>,default_factory=<dataclasses._MISSING_TYPE object at 0x7f0dcc0ee060>,init=True,repr=True,hash=None,compare=True,metadata=mappingproxy({}),kw_only=False,_field_type=_FIELD), 'vocab': Field(name='vocab',type='Mapping[str, int]',default=<dataclasses._MISSING_TYPE object at 0x7f0dcc0ee060>,default_factory=<dataclasses._MISSING_TYPE object at 0x7f0dcc0ee060>,init=True,repr=True,hash=None,compare=True,metadata=mappingproxy({}),kw_only=False,_field_type=_FIELD), 'frame_rate': Field(name='frame_rate',type='float',default=<dataclasses._MISSING_TYPE object at 0x7f0dcc0ee060>,default_factory=<dataclasses._MISSING_TYPE object at 0x7f0dcc0ee060>,init=True,repr=True,hash=None,compare=True,metadata=mappingproxy({}),kw_only=False,_field_type=_FIELD), 'blank_id': Field(name='blank_id',type='int',default=<dataclasses._MISSING_TYPE object at 0x7f0dcc0ee060>,default_factory=<dataclasses._MISSING_TYPE object at 0x7f0dcc0ee060>,init=True,repr=True,hash=None,compare=True,metadata=mappingproxy({}),kw_only=False,_field_type=_FIELD)}
dict() -> new empty dictionary dict(mapping) -> new dictionary initialized from a mapping object's (key, value) pairs dict(iterable) -> new dictionary initialized as if via: d = {} for k, v in iterable: d[k] = v dict(**kwargs) -> new dictionary initialized with the name=value pairs in the keyword argument list. For example: dict(one=1, two=2)
__doc__
class-attribute
¶
__doc__ = 'Per-frame acoustic posteriors for one audio clip.\n\n Attributes:\n log_probs: ``(T, C)`` float32 array of log-probabilities over the ``C``\n classes (phones + blank), one row per frame.\n vocab: Maps each phone label to its class id (column of ``log_probs``).\n frame_rate: Emission frames per second (e.g. 50.0 for a wav2vec2 CTC\n head with a 20 ms stride) — used to convert frame spans to seconds.\n blank_id: The CTC blank class id.\n '
str(object='') -> str str(bytes_or_buffer[, encoding[, errors]]) -> str
Create a new string object from the given object. If encoding or errors is specified, then the object must expose a data buffer that will be decoded using the given encoding and error handler. Otherwise, returns the result of object.str() (if defined) or repr(object). encoding defaults to sys.getdefaultencoding(). errors defaults to 'strict'.
__match_args__
class-attribute
¶
Built-in immutable sequence.
If no argument is given, the constructor returns an empty tuple. If iterable is specified the tuple is initialized from iterable's items.
If the argument is a tuple, the return value is the same object.
__module__
class-attribute
¶
str(object='') -> str str(bytes_or_buffer[, encoding[, errors]]) -> str
Create a new string object from the given object. If encoding or errors is specified, then the object must expose a data buffer that will be decoded using the given encoding and error handler. Otherwise, returns the result of object.str() (if defined) or repr(object). encoding defaults to sys.getdefaultencoding(). errors defaults to 'strict'.
Wav2Vec2EspeakModel ¶
Wav2Vec2EspeakModel(model_path: Union[str, Path], vocab_path: Union[str, Path], *, frame_rate: float = 50.0, blank_token: str = '<pad>')
espeak-IPA wav2vec2 CTC acoustic model (ONNX) — an AcousticModel.
Parameters:
-
model_path(Union[str, Path]) –ONNX model file. Input: a mono 16 kHz waveform
(1, N); output:(1, T, C)logits. -
vocab_path(Union[str, Path]) –vocab.jsonmapping each phone to its class id. -
frame_rate(float, default:50.0) –Emission frames per second — 50.0 for wav2vec2's 320× stride at 16 kHz (used to turn frame spans into seconds).
-
blank_token(str, default:'<pad>') –The vocab key that is the CTC blank (default
"<pad>").
from_pretrained
classmethod
¶
from_pretrained(repo_id: str = 'sadda-speech/wav2vec2-espeak-ctc', *, revision: str | None = None, model_file: str = 'model.onnx', vocab_file: str = 'vocab.json', **kwargs) -> Wav2Vec2EspeakModel
Fetch the ONNX model + vocab from a Hugging Face repo, then build it.
Downloads model_file and vocab_file from repo_id (default the
sadda HF org) via huggingface_hub (cached locally), then constructs
the model. pip install "sadda[align]" provides both onnxruntime and
huggingface_hub. Remaining kwargs pass through to the constructor.
emissions ¶
Run the model over mono audio → per-frame log-probs.
Audio not already at the model's :data:SAMPLE_RATE (16 kHz) is
resampled first (:func:_resample_to_model_rate), so recordings at any
rate align without the caller resampling by hand.