Upload plugins/mlintern/skills/hf-dataset-search/SKILL.md with huggingface_hub

plugins/mlintern/skills/hf-dataset-search/SKILL.md

@@ -0,0 +1,61 @@
+---
+name: hf-dataset-search
+description: "Search Hugging Face Hub for datasets, inspect schema, splits, sample rows, and training-method compatibility."
+disable-model-invocation: false
+---
+
+# hf-dataset-search — Hugging Face Dataset Discovery
+
+## Purpose
+
+Find and validate Hugging Face datasets before using them in training or evaluation. Prevent schema hallucinations and incompatible data.
+
+## Tools
+
+- `dataset_search`: Search HF Hub datasets by query, tags, or task.
+- `hub_repo_details`: Get dataset metadata and README.
+
+## Dataset Viewer Inspection
+
+The plugin does not expose a direct `hf_inspect_dataset` tool. To inspect schema, splits, and sample rows, use the dataset inspection script:
+
+```bash
+python skills/ml-intern-harness/scripts/inspect_dataset.py <dataset-id> --split train --sample-rows 3
+```
+
+This queries the Hugging Face Dataset Viewer API for:
+- Validity status
+- Configs and splits
+- Schema (column names and types)
+- Representative rows
+- Parquet file availability
+- SFT/DPO/GRPO compatibility notes
+
+## Workflow
+
+1. Search for candidate datasets with `dataset_search`.
+2. Inspect metadata with `hub_repo_details` (set `repo_type="dataset"`).
+3. Run `inspect_dataset.py` for schema and sample row details.
+4. Verify training-method compatibility:
+   - SFT: needs `messages`, `text`, or `prompt`/`completion`
+   - DPO: needs `prompt`, `chosen`, `rejected`
+   - GRPO: needs `prompt`
+5. Surface class imbalance, missing values, unexpected formats, or unsafe substitutions.
+
+## Example
+
+```
+_dataset_search(query="instruction following", sort="downloads")
+_hub_repo_details(repo_ids=["HuggingFaceH4/ultrachat_200k"], repo_type="dataset")
+python skills/ml-intern-harness/scripts/inspect_dataset.py HuggingFaceH4/ultrachat_200k --split train_sft --sample-rows 3
+```
+
+## Validation Checklist
+
+Before using a dataset:
+- [ ] Dataset is valid and has Dataset Viewer coverage.
+- [ ] Configs/splits match expectations.
+- [ ] Column names and types are compatible with the trainer.
+- [ ] Sample rows look reasonable.
+- [ ] Row count is sufficient for the task.
+- [ ] License and gating are acceptable.