ASLP-lab commited on
Commit
da2d00a
·
verified ·
1 Parent(s): 67accac

update README

Browse files
Files changed (1) hide show
  1. README.md +105 -0
README.md CHANGED
@@ -1,3 +1,108 @@
1
  ---
2
  license: apache-2.0
3
  ---
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
  ---
2
  license: apache-2.0
3
  ---
4
+ ---
5
+ license: apache-2.0
6
+ ---
7
+
8
+ # SphereVAE
9
+
10
+ SphereVAE: Hyperspherical Latent Autoencoders for Robust Autoregressive Speech Representation Modeling
11
+
12
+ [arXiv](https://arxiv.org/pdf/2609.09903v1) · [Demo](https://haoyuzhang3.github.io/SphereVAE_Demo/) · [GitHub](https://github.com/ASLP-lab/SphereVAE)
13
+
14
+
15
+
16
+ ## Model
17
+
18
+ SphereVAE combines a SEANet audio codec with causal Transformers and a
19
+ Power-spherical latent space for robust speech representation modeling.
20
+
21
+ ## Installation
22
+
23
+ Use Python 3.10 or newer. Install a PyTorch and torchaudio pair appropriate for
24
+ your machine, then install the dependencies:
25
+
26
+ ```bash
27
+ python -m pip install -r requirements.txt
28
+ ```
29
+
30
+ GPU training uses Accelerate. Configure it for your machine before launching:
31
+
32
+ ```bash
33
+ accelerate config
34
+ ```
35
+
36
+ ## Training data
37
+
38
+ Place a Kaldi-style SCP file at `data/train.scp`, or update
39
+ `data.train_scp_path` in `configs/config_Sphere_VAE.yaml`:
40
+
41
+ ```text
42
+ utt_0001 /path/to/audio_0001.flac
43
+ utt_0002 /path/to/audio_0002.wav
44
+ ```
45
+
46
+ Each line contains an utterance ID and an audio path. Audio is processed as
47
+ mono 24 kHz input and prepared as training segments by the dataset pipeline.
48
+
49
+ ## Training
50
+
51
+ ```bash
52
+ CUDA_VISIBLE_DEVICES=0 accelerate launch train.py -c configs/config_Sphere_VAE.yaml
53
+ ```
54
+
55
+ Adjust training settings in `configs/config_Sphere_VAE.yaml`. Checkpoints and
56
+ TensorBoard logs are written to the configured experiment directory. For
57
+ multi-GPU training, configure Accelerate and set the visible devices as needed.
58
+
59
+ ## Inference
60
+
61
+ ```bash
62
+ python infer.py \
63
+ --checkpoint exps/Sphere_VAE/G_400000.pth \
64
+ --input data/test \
65
+ --output output/reconstructed \
66
+ --device cuda:0
67
+ ```
68
+
69
+ `--input` accepts an audio file or directory. Supported extensions are WAV,
70
+ FLAC, MP3, OGG, and M4A. Use `--config` to select another configuration.
71
+ Inference uses posterior sampling, so repeated runs can produce different
72
+ reconstructions. Output is trimmed to the input length.
73
+
74
+ ## Layout
75
+
76
+ ```text
77
+ models/model_Sphere_VAE.py Sphere_VAE model and builder
78
+ modules/ SEANet, Transformer, spherical distribution, utilities
79
+ train.py Training entrypoint
80
+ infer.py Audio reconstruction entrypoint
81
+ dataset.py SCP-listed audio dataset
82
+ configs/ Sphere_VAE configuration
83
+ discriminators/ STFT discriminator
84
+ losses/ Spectral and adversarial losses
85
+ utils/ Configuration, checkpoints, and compilation utilities
86
+ ```
87
+
88
+ ## Citation
89
+
90
+ ```bibtex
91
+ @misc{zhang2026spherevaehypersphericallatentautoencoders,
92
+ title={SphereVAE: Hyperspherical Latent Autoencoders for Robust Autoregressive Speech Representation Modeling},
93
+ author={Haoyu Zhang and Jingbin Hu and Hanke Xie and Qirui Zhan and Wenhao Li and Ziyu Zhang and Xiaming Ren and Yue Li and Xunyu Zhu and Zhipeng Chen and Lei Xie},
94
+ year={2026},
95
+ eprint={2609.09903},
96
+ archivePrefix={arXiv},
97
+ primaryClass={eess.AS},
98
+ url={https://arxiv.org/abs/2609.09903}
99
+ }
100
+ ```
101
+
102
+ ## Acknowledgments
103
+
104
+ Parts of the code are adapted from Kyutai Mimi and Meta AudioCraft/EnCodec,
105
+ as indicated by the original notices retained in the source files. This
106
+ export does not assign a new license to that third-party code. The source
107
+ checkout did not include the root license files referenced by those notices;
108
+ applicable upstream license texts still need to be supplied before release.