nyuuzyou (nyuuzyou)

posted an update 2 days ago

Post

234

Earlier I asked for a storage grant for some new datasets. One of those, the Google Code Archive nyuuzyou/google-code-archive, is now in trending. Thanks to Hugging Face and the community for the support. 🤗

1 reply

·

posted an update 4 days ago

Post

2590

🏛️ Microsoft CodePlex Archive Dataset - nyuuzyou/ms-codeplex-archive

Following the strong response to the Google Code Archive nyuuzyou/google-code-archive (thanks!), this release preserves another major historical repository: the Microsoft CodePlex Archive.

CodePlex served as Microsoft’s primary open-source hosting platform from 2006 to 2017. This dataset captures the distinct .NET and Windows-centric development ecosystem that flourished before the industry standardizing on GitHub.

Key Stats:

- 5,043,730 files from 38,087 repositories
- 3.6 GB compressed Parquet
- 91 programming languages (Heavily featuring C#, ASP.NET, and C++)
- Cleaned of binaries, build artifacts, and vendor directories (node_modules, packages)
- Includes platform-specific license metadata (Ms-PL, Ms-RL)

reacted to raincandy-u's post with 😎 5 days ago

Post

2849

Introducing Rain-v2: Democratizing LLM training on gaming GPUs! ⚡

Following Rain-100M, we’re scaling up. Rain-v2 features a larger training dataset.

We’ve published a comprehensive blog covering the end-to-end journey—from raw data collection to rigorous evaluation and safety testing.

HF Repo: 🤗 raincandy-u/Rain-v2

Blog: 📚
https://angelkawaii.xyz/2026/01/29/rain-v2/

Special thanks to the open-source community and the SmolLM2 team for their foundational work! 🚀

HuggingFaceTB
SmolLM2: When Smol Goes Big -- Data-Centric Training of a Small Language Model (2502.02737)

posted an update 10 days ago

Post

1882

🌐 NNTP Discussion Archives - 387M Messages from Public Newsgroups - nyuuzyou/nntp-text-387m

Here's something different from the code datasets: 20+ years of public discussion archives from NNTP newsgroups. Clean Parquet format, but this time it's conversations instead of code.

Key Stats:
- 386,629,949 messages from 159,345 newsgroups
- 191 GB compressed Parquet storage
- Spans 2002-2026
- Multilingual: English, German, French, Italian, Dutch, Polish, Russian, and others
- Email addresses redacted for privacy

The data is messy in the way real discussions are messy. Spam wasn't filtered out - you get the advertisements, the arguments, the off-topic threads, all of it. If you want sanitized text, this isn't it. If you want to see how people actually talked online before Discord and Reddit took over, here you go.

Processing kept it simple: convert everything to UTF-8, remove exact duplicates, strip binary attachments, redact emails. Legacy character encodings were a nightmare - had to handle Windows-1252, ISO-8859 variants, KOI8-R, Shift-JIS, GBK, and others just to get readable text. At least it was fun to do, and I think the result turned out pretty well. I hope someone else will also be able to have fun or gain something useful from this project.

reacted to raincandy-u's post with 🔥 11 days ago

Post

5305

🤗 Just released Rain-100M, an experimental ~97M-parameter Qwen3-style language model trained from random initialization.

Repo: raincandy-u/Rain-100M

Data: HuggingFaceFW/fineweb-edu, ~3B tokens, English only

Tokenizer: custom 16k BPE, context length 4096

Architecture: 12 Transformer layers, hidden size 768, 12 heads, MLP 2048, SiLU, bf16

Rain-100M is a raw base model (not instruction-tuned or safety-aligned), aimed at small-scale research, debugging training pipelines, and CPU/edge experiments. If you run evaluations, finetunes, or visualizations with it, I would be very interested in your results!

3 replies

·

reacted to ZennyKenny's post with 🚀 15 days ago

Post

3229

😎 My new personal website is live! Check out https://kennethhamilton.me to chat with an LLM about my professional skills and personal projects.

🙈 Think of it like a really, really vain version of ChatGPT.

6 replies

·

posted an update 15 days ago

Post

1452

🏛️ Google Code Archive Dataset - nyuuzyou/google-code-archive

Expanding beyond the modern code series, this release presents a massive historical snapshot from the Google Code Archive. This dataset captures the open-source landscape from 2006 to 2016, offering a unique time capsule of software development patterns during the era before GitHub's dominance.

Key Stats:

- 65,825,565 files from 488,618 repositories
- 47 GB compressed Parquet storage
- 454 programming languages (Heavily featuring Java, PHP, and C++)
- Extensive quality filtering (excluding vendor code and build artifacts)
- Rich historical metadata: original repo names, file paths, and era-specific licenses

This is one of those releases that I'm most interested in getting feedback on. Would you like to see more old code datasets?

reacted to ZennyKenny's post with 🔥 21 days ago

Post

3201

🎉 Wow. Congratulations @bfirsh and the Replicate team on the CloudFlare acquisition!

✌️ You've really built an incredible ecosystem and product offering and should be super proud.

posted an update 21 days ago

Post

270

🇨🇳 GitCode Dataset - Continuing the Chinese Code Series nyuuzyou/gitcode-code

Following up on the Gitee release, here's another major Chinese code dataset from GitCode (CSDN's code hosting platform). Same pipeline, same clean format, more valuable data from China's developer ecosystem.

Key Stats:
- 48,142,567 files from 85,632 repositories
- 40 GB compressed Parquet storage
- 537 programming languages
- Extensive quality filtering applied
- Rich metadata: repo names, file paths, licenses, and sizes

The final dataset in the Chinese code series is also available: nyuuzyou/jihulab-code. It's smaller in size but shares the same pipeline and formatting.

replied to Ujjwal-Tyagi's post 22 days ago

Glad you are finding it useful! You should also check out these datasets:

https://huggingface.co/datasets/nyuuzyou/gitcode-code
https://huggingface.co/datasets/nyuuzyou/jihulab-code

They use the same data processing pipeline and format, but they are sourced from different Chinese services.

reacted to Ujjwal-Tyagi's post with 🤗 22 days ago

Post

2586

I am very excited to see the release of nyuuzyou/gitee-code. This is exactly what I have been looking for. Thank you to @nyuuzyou for his hard work on this.

3 replies

·

reacted to codelion's post with 🔥 23 days ago

Post

6077

Introducing Dhara-70M: A diffusion language model that achieves 3.8x higher throughput than autoregressive models!

Key findings from our research on optimal architectures for small language models:

→ Depth beats width: 32 layers outperforms 12 layers at the same parameter count
→ Best-in-class factuality: 47.5% on TruthfulQA
→ 10x training efficiency using WSD (Warmup-Stable-Decay) conversion
→ Canon layers add only 0.13% parameters but improve reasoning

We trained on 1B tokens using the optimal 50-30-20 dataset mix (PDFs + filtered web + educational content), then converted to diffusion with just 100M additional tokens.

Blog: https://huggingface.co/blog/codelion/optimal-model-architecture
Model: codelion/dhara-70m

1 reply

·

posted an update 23 days ago

Post

1671

🇨🇳 Gitee Code Dataset - The Missing Piece of the Stack
nyuuzyou/gitee-code

Gitee is not included in the Software Heritage archive, meaning it is currently missing from datasets like The Stack. This release fills that massive gap, serving as the largest Chinese code dataset and one of the largest code corpuses overall.

- 819,472,785 files from 3,105,923 repositories
- 536 GB compressed Parquet storage
- 554 programming languages
- Extensive quality filtering: Removed vendor code, artifacts, and generated files
- Rich Chinese language understanding: High volume of Chinese comments and docs

Huge thanks to Hugging Face for the storage grant that made hosting this (and all my other datasets) possible!

I have also already dropped several other new code datasets and rolled out QoL improvements for older ones. I will be dropping posts on those throughout the week.

1 reply

·

reacted to hypothetical's post with 🔥 27 days ago

Post

2028

We have updated our transcription model: TheStageAI/thewhisper-large-v3-turbo

– 6.00 WER on the English Open ASR Leaderboard
– 4.74 WER on the Multilingual Open ASR Leaderboard
– Beats NVIDIA Parakeet (6.34 WER) and Whisper-large-v3-turbo (7.8 WER)
– Strong improvements in Arabic, Hindi, Chinese
– Maintains quality with background and environmental noise
– Optimized inference engines for NVIDIA and Apple
– Hugging Face Transformers interface for easy use
– Best-in-class speed on NVIDIA GPUs and power efficiency on Apple devices
– NVIDIA Jetson Thor support

2 replies

·

reacted to Reality123b's post with 🤗 27 days ago

Post

2107

Happy birthday to me!!!

2 replies

·

reacted to Fishtiks's post with 🔥 27 days ago

Post

1661

Have extra processing power in downtime? Have old devices you haven't used in years? I recommend primarily Folding@home to do protein folding on your GPUs, but also BOINC, particularly on Android and Apple devices, because of the lower power usage. I've been doing this, and get about 14,000 hours a week in, primarily for mapping cancer markers on BOINC on an Aiyara cluster of Androids. I also hold a sign out by the highway encouraging people to join BOINC. It was Dylan Bucci, a young promoter of BOINC on school's computers, who wished before he died to get as many people on as possible to do this, and in his honor, the Dylan Bucci challenge was implemented. No reason to wait for a challenge. If you care about such things, there is an associated cryptocurrency for such processing, but it's worth it to save lives.

I look forward to AI-related endeavors like this, and only know of NATIX Drive&, Acurast, and HYRA AI, all of which use Androids I'd rather devote to BOINC. However, they also allow one to be paid, and to totally devote old devices to the processing. On the same topic, DexPOINT monetizes your Android's Internet connection.

BOINC runs on Android, Apple, PCs of all sorts, Pi devices, Chrome devices, Fire Sticks, TV boxes, Android watches with the full OS, and all sorts of things that have Android or the ability to run Linux, although it will also run on Windows. Folding@home works best on PCs with modern NVIDIA GPUs, and in a cool room. You can also run BOINC on modern computers, but they must be throttled, because they often get too hot.

4 replies

·

reacted to mlabonne's post with 🚀 28 days ago

Post

5440

New family of 1B models just dropped!

> LiquidAI/LFM2.5-1.2B-Base: 10T → 28T tokens
> LiquidAI/LFM2.5-1.2B-Instruct: new large-scale multi-stage RL
> LiquidAI/LFM2.5-1.2B-JP: our most polite model
> LiquidAI/LFM2.5-VL-1.6B: multi-image multilingual
> LiquidAI/LFM2.5-Audio-1.5B: 8x times faster, no quality loss

Super proud of this release 🤗

3 replies

·

reacted to ronantakizawa's post with 👍 about 1 month ago

Post

5620

Thank you @clem (Co-Founder & CEO of Hugging Face) for sharing my dataset on X / Twitter!

ronantakizawa/github-top-developers

#github #dataset

4 replies

·

reacted to omarkamali's post with 👍 about 2 months ago

Post

3487

Hello picomon! AMD GPU Monitoring made easy

Just run uvx picomon and behold:

┌──────────────────────────────────────────┐  ┌──────────────────────────────────────────┐
│ GPU 0  GFX  42%  UMC  21%                │  │ GPU 1  GFX  78%  UMC  66%                │
│ PWR 135/250W (54%)  VRAM 10.0/16.0GB 62% │  │ PWR 210/250W (84%)  VRAM 14.5/16.0GB 90% │
│                                          │  │                                          │
│ GFX ▁▂▂▃▄▄▅▆▆▇█▇▆▅▄▃▂▁                   │  │ GFX ▂▃▄▅▆▇██▇▆▅▄▂▂▃▅▆                    │
│ PWR ▁▁▂▂▃▄▄▅▆▇██▇▆▅▄▂▁                   │  │ PWR ▂▂▃▄▅▆▇██▇▆▅▄▃▂▂▃                    │
│ VRM ▁▁▂▂▃▄▄▅▆▇███▇▆▅▄▂                   │  │ VRM ▂▃▄▅▆▆▇███▇▆▅▄▃▂▂                    │
└──────────────────────────────────────────┘  └──────────────────────────────────────────┘

Repo at https://github.com/omarkamali/picomon
Or pypi at https://pypi.org/project/picomon

reacted to melvindave's post with 🚀 2 months ago

Post

2443

Deployed my first Space!

Moved my PDF to Images Converter app from streamlit cloud to Spaces

Upload a PDF and get a zip file of pages as PNGs or JPEGs, perfect for posts or decks

Hope it's useful!

melvindave/pdf-to-images

3 replies

·

nyuuzyou PRO

AI & ML interests

Recent Activity

Organizations

nyuuzyou PRO

AI & ML interests

Recent Activity

Organizations

nyuuzyou's activity