StreamPA

Frozen StreamPA pronunciation-assessment inference: fine-tuned Whisper Base N-best -> G2P / prefix-local Charsiu / PCN / prosody -> slot and utterance scores. This is a downloadable, multi-component Python system, not a hosted endpoint or a standard Transformers AutoModel/pipeline implementation.

Published identity

• Experiment: M_stress_scalar_gate_capacity64 (Ours-M), primary seed 1337.
• Original checkpoint SHA256: 870df8dbe3c787487f094e0c4d4ade6bed8fcc5d86b578c4edc57581e81023b4.
• All original tensors are retained losslessly in scorer/model.safetensors. Strict loading retains the historical inactive utt_head; the active utterance path is the frozen gru_visible head.
• 64 hidden dimensions, three local-attention blocks, four heads, scalar reliability gate, detached stress branch, 64-dimensional recurrent state. Inputs: 40 phone columns including epsilon, 14 global prosody features, 17 slot prosody features, and at most 72 PCN slots.
• The included Whisper weights are the actual fine-tuned Base checkpoint, SHA256 8917c9612b44f73bb5ef2ffec48cf44a4ee0dfa5a94af25e9e6710b9fd66a17f. They must not be replaced with generic Whisper Base. Beam size 8, N-best 5, maximum 128 decoded tokens; original hypothesis-score computation retained.
• The paper's four-seed aggregate (1337, 202601, 202602, 202603) is not the performance of this single checkpoint. No unverified paper metric is copied into this card. Interface tests do not establish assessment PCC.

Download and install

Verified platform: Python 3.10 on Ubuntu in local Windows WSL2, CPU FP32. Native Windows and GPU inference are not part of the verified platform. Use a new Python 3.10 environment. The fixed release tag below is created only after fresh Hub-download inference verification succeeds.

python3.10 -m venv .venv-streampa
source .venv-streampa/bin/activate
python -m pip install huggingface_hub==0.36.0
python -c "from huggingface_hub import snapshot_download; snapshot_download('faeea/StreamPA', revision='streampa-v1.0.0', local_dir='./StreamPA')"
cd StreamPA
python -m pip install --extra-index-url https://download.pytorch.org/whl/cpu -r requirements-lock.txt
python -m pip install --no-deps --no-build-isolation .
python scripts/setup_assets.py
python scripts/verify_bundle.py

The public bundle contains the scorer and fine-tuned Whisper weights. Setup downloads Charsiu and G2P resources from official, fixed revisions, checks their SHA256s, and installs them inside this bundle. There is no runtime reference to a training checkout or private server. See THIRD_PARTY_NOTICES.md for licenses and third-party weight terms. Internet is needed for initial installation/setup, not for subsequent inference. No author token is required for the public release or its official runtime resources.

One WAV and streaming CLI

Input: your own English mono 16,000-Hz WAV, no transcript or labels. The frontend supports at most 30 seconds per recording and 72 current PCN slots; longer inputs return an explicit error/status, never hidden truncation. Output files must not already exist.

python -m streampa.infer --audio /path/to/demo.wav --bundle-dir . --device cpu --output scores.json
python -m streampa.stream --audio /path/to/demo.wav --bundle-dir . --device cpu --output prefixes.jsonl

The single-file CLI returns all prefixes plus the EOF result. The streaming CLI actually reads and appends waveform blocks, then signals EOF. Both run the same causal implementation. CLI block size does not change the fixed 0.64-second schedule and 0.16-second lookahead.

For an appropriate CUDA installation, select one device externally and use --device cuda. Do not use a broken/busy device. The inherited frontend uses FP16 Whisper on CUDA, FP32 Charsiu/scorer; CPU is FP32 throughout. CUDA is untested in this release validation, and changing precision can change ASR decoding. The CPU lockfile deliberately selects CPU PyTorch.

Python sessions

import soundfile as sf
from streampa import StreamPAPipeline

pipeline = StreamPAPipeline('./StreamPA', device='cpu', threads=2)
session = pipeline.create_session()
with sf.SoundFile('demo.wav') as f:
    assert f.samplerate == 16000 and f.channels == 1
    while True:
        block = f.read(1600, dtype='float32')  # 100 ms received audio
        if len(block) == 0:
            break
        for result in session.append(block):
            print(result['audio_end'], result['status'], result['scores'])
print(session.finish())  # process the exact EOF, including a short final block
session.reset()          # required before a new recording in this session
assert session.current() is None

The pipeline owns frozen models; sessions have separate audio and recurrent state. Use sessions sequentially (concurrent GPU service scheduling is not provided). StreamPAPipeline.from_pretrained(repo_id, revision=...) is also implemented, but its downloaded directory must have setup_assets.py run before loading; use the explicit local-bundle example for first setup.

JSON and state semantics

• commit_time: scheduling boundary; audio_end: exact prefix waveform end supplied to every frontend; received_audio_end: audio received by the caller so far, which may be later when a block spans multiple boundaries. A normal prefix is processed only after commit_time + 0.16 is received. EOF alone defines the final endpoint. No reference boundary is consulted.
• slots: raw independent phone-head and word-head predictions per current PCN slot, ASR word assignment, slot time, validity and commitment masks. Epsilon/gap slots are retained; valid_phone=false means their phone score is not a localized phone assessment. asr_word_id=-1 has no word assignment.
• words: mean word-head predictions over visible slots assigned to each current ASR word. This is a presentation aggregation, not mean phone accuracy. It is not the paper's offline reference-word-identity evaluation. ASR words and scores may be revised at later prefixes, including committed words.
• new_commit_mask: the frozen legacy boundary-aware state-update rule. Only newly committed word representations are consumed by the GRU. With no new words the persistent state remains unchanged, but visible features still enter the sentence head and may change current utterance scores. This heuristic has no formal exactly-once guarantee for every possible mixed predecessor merge. It is not silently replaced with a corrected rule.
• scores.utterance_network_raw retains native model outputs, including the untrained completeness dimension for audit only. Displayed utterance scores omit completeness. ASR confidence is not a probability of correct pronunciation.
• Historical target normalization divides all source scores by 5: native word/utterance source scores span 0--10; phone source scores span 0--2. Thus displayed 0--5 word/utterance = network output x 2.5, and phone = network output x 12.5. Outputs are not clipped; out-of-range estimates remain visible. Stress has a trained head but is not a calibrated correctness probability.
• status != 'ok' returns scores=null, never a false zero grade. Statuses include empty/too-short/silent audio, no usable hypothesis, G2P/alignment/ acoustic failures, nonfinite values and unsupported slot length. Failed prefixes remain in the output; previously successful state is preserved.
• stages and wall_time_sec measure actual local computation, not capture waiting or a universal real-time guarantee. Model loading is separate.

Offline operation and checks

python scripts/setup_assets.py --offline
HF_HUB_OFFLINE=1 TRANSFORMERS_OFFLINE=1 python -m streampa.infer --audio /path/to/demo.wav --bundle-dir . --device cpu --output offline_scores.json
python -m pytest tests -q

Keep the entire prepared bundle, including charsiu/ and assets/nltk_data/. Do not delete those setup-generated directories when preparing an offline copy. For hash mismatch, stop and re-download the specified revision rather than replacing a resource with another model. Missing G2P/tagger resources require setup_assets.py; inference does not intentionally download them. Proxy configuration may be needed during setup in restricted networks. Use a fresh output filename for each command. Convert sample rate/channels before opening a session; the inference entry point does not resample a hidden full recording.

Scope and limitations

Inference accepts audio only. Training used human annotations and teacher supervision. Historical evaluation data were selected by preprocessing filters and predominantly contain read speech. Deployment removes the historical reference-phone-count ratio filter and uses received-audio/EOF scheduling; it does not claim to reproduce historical test membership or PCC on arbitrary WAVs. The frozen dictionary is a corpus-derived pronunciation vocabulary, not an utterance-specific reference transcript. Original PCN gap semantics, alignment, normalization, repetition filtering and legacy commitment are retained.

No claim of validated spontaneous/open-ended speech generalization, formal exactly-once commitment, GRU-induced stability improvement, or end-to-end real-time performance is made. Synthetic validation clips test computation and interfaces, not pronunciation-assessment accuracy. See validation_summary.json for actually completed and untested checks.

Observed interface-test limitation: several short prefixes produce repeated Whisper hypotheses and are explicitly unscorable. A synthetic "Good morning" clip was unscorable at all three prefixes; three separately selected licensed read-speech clips all produced valid EOF scores, with 9 valid prefixes out of 15 attempted. These failures were not hidden or repaired by changing decoding. CPU frontend computation was much slower than the audio duration in these tests.

The old v6 47-feature system remains historical only at faeea/StreamPA, revision legacy-v6-before-streampa. It is not this checkpoint and must not be mixed with these configurations or entry points.