| | --- |
| | license: apache-2.0 |
| | datasets: |
| | - amphion/Emilia-Dataset |
| | language: |
| | - de |
| | - en |
| | base_model: |
| | - neuphonic/neucodec |
| | tags: |
| | - audio |
| | - speech |
| | --- |
| | |
| | ## NeuCodec decoder fine-tuned for German speech |
| |
|
| | This is just the decoder of [neuphonic/neucodec](https://huggingface.co/neuphonic/neucodec), fine-tuned on equal amounts of German and English speech data from Emilia-Yodas, to enhance decoding quality of German speech. |
| | Since we only fine-tuned the decoder, the codebook is identical to the base model, meaning this model can be used with the regular NeuCodec encoder. |
| |
|
| | We supply a compact class `NeuCodecDecoder.py` to easily run inference with this decoder since the NeuCodec codebase doesn't easily allow loading model files from foreign HuggingFace repos. |
| |
|
| | ### Inference Example |
| |
|
| | ```python |
| | import torch |
| | import torchaudio |
| | |
| | from NeuCodecDecoder import NeuCodecDecoder |
| | |
| | decoder_model = NeuCodecDecoder.from_pretrained("DigitalLearningGmbH/neucodec-decoder-ft-de") |
| | decoder_model = decoder_model.eval().cuda() |
| | |
| | with torch.no_grad(): |
| | decoded = decoder_model.decode_code(torch.tensor(tokens).unsqueeze(0).unsqueeze(0).to('cuda')).cpu() |
| | |
| | torchaudio.save("decoded.wav", decoded[0, :, :], 24_000) |
| | ``` |
| |
|
| | For more information please refer to [the original model card](https://huggingface.co/neuphonic/neucodec). |
