Hugging Face's logo
Daniel Wayner's picture

Daniel Wayner

danwayner

AI & ML interests

None yet

Recent Activity

replied to kanaria007's post 1 day ago
✅ New Article: *Evaluation as a Goal Surface* (v0.1) Title: 🧪 Evaluation as a Goal Surface: Experiments, Learning Boundary, and ETH-Aware A/B 🔗 https://huggingface.co/blog/kanaria007/evaluation-as-a-goal-surface --- Summary: Most “evaluation” quietly collapses into a single number—and then we optimize the wrong thing. This article reframes evaluation as a *goal surface*: multi-objective, role-aware, and ethics-bounded. In SI-Core terms, experiments become *first-class Jumps (E-Jumps)* with explicit contracts, traces, and gates—so you can run A/B tests, shadow evals, and adaptive rollouts *without violating ETH, confusing principals/roles, or learning from unsafe data*. > Don’t optimize a metric. > Optimize a goal surface—under explicit constraints. --- Why It Matters: • Prevents Goodhart failures by treating evaluation as *multi-goal + constraints*, not a scalar leaderboard • Makes experimentation auditable: *EvalTrace* answers “what changed, for whom, why, and under what policy” • Enables *ETH-aware A/B*: assignment, exposure, and stopping rules respect safety/fairness boundaries • Connects experiments to governance: *Learning Boundary (LB)* + rollout control (PoLB) instead of “ship and pray” --- What’s Inside: • What EVAL is in SI-Core, and *who* is being evaluated (agents / roles / principals) • “Experiments as Jumps”: *E-Jump request/draft* patterns and contracts • *ETH-aware variant testing* (including ID/role constraints at assignment time) • Shadow evaluation + off-policy evaluation (how to learn without unsafe intervention) • Role & persona overlays for EVAL (role-aware scoring, persona-aware reporting) • *EvalTrace* for audits + incident review, plus “evaluate the evaluators” test strategies • Practical experiment design: power/sample size, early stopping, multi-objective bandits, causal inference --- 📖 Structured Intelligence Engineering Series this is the *how-to-design / how-to-run experiments safely* layer.
reacted to nyuuzyou's post with 👍 1 day ago
🏛️ Microsoft CodePlex Archive Dataset - https://huggingface.co/datasets/nyuuzyou/ms-codeplex-archive Following the strong response to the Google Code Archive https://huggingface.co/datasets/nyuuzyou/google-code-archive (thanks!), this release preserves another major historical repository: the Microsoft CodePlex Archive. CodePlex served as Microsoft’s primary open-source hosting platform from 2006 to 2017. This dataset captures the distinct .NET and Windows-centric development ecosystem that flourished before the industry standardizing on GitHub. Key Stats: - 5,043,730 files from 38,087 repositories - 3.6 GB compressed Parquet - 91 programming languages (Heavily featuring C#, ASP.NET, and C++) - Cleaned of binaries, build artifacts, and vendor directories (node_modules, packages) - Includes platform-specific license metadata (Ms-PL, Ms-RL)
reacted to yuriyvnv's post with 👍 1 day ago
🎯 WAVe: 1B Multimodal Embedding Model for Word-Level Speech Quality Multimodal embeddings for speech + transcript that verify quality at the word level, not just sentence level. Catches mispronunciations, timing errors, and prosody issues that sentence-level filters miss. 📊 Impact on Portuguese ASR: • 34% reduction in training steps • 50% better cross-domain generalization • 30% less synthetic data needed • Word-aligned attention finds errors other methods miss 🏗️ Architecture: • Text: XLM-RoBERTa (278M params) • Audio: Wav2Vec2-BERT 2.0 (581M params) • Word Alignment: Multi-head attention + GLU (14M params) • Total: 1B parameters ``` from transformers import AutoModel, AutoProcessor processor = AutoProcessor.from_pretrained( "yuriyvnv/WAVe-1B-Multimodal-PT", trust_remote_code=True ) model = AutoModel.from_pretrained( "yuriyvnv/WAVe-1B-Multimodal-PT", trust_remote_code=True ) ``` # Assess speech-transcript alignment ``` inputs = processor(text="Olá, como está?", audio=audio_array, sampling_rate=16000, return_tensors="pt") quality = model(**inputs).quality_score.item() ``` Perfect for filtering synthetic speech datasets before ASR training. Model: https://huggingface.co/yuriyvnv/WAVe-1B-Multimodal-PT Code to create WAVe : https://github.com/yuriyvnv/WAVe #multimodal #speech #embeddings #asr #syntheticdata #qualityassessment
View all activity

Organizations

None yet