Organization Card

🤗 FineEnvs: Open RL Environments

FineEnvs is a home for end-to-end RL environment recipes, built to make it easier to explore, reproduce, train, and evaluate agent systems.

Explore complete and reproducible environment projects from us and the community, including:

🌍 Open RL environments
🧩 End-to-end environment recipes
💻 Complete implementations
📦 Models, datasets, and artifacts
🧪 Training and evaluation setups
🚀 Demos and Spaces
📚 Tutorials and guides

All the reproducible code — environments, rollouts, training configs, notebooks, article and slide sources — lives in one repo: github.com/adithya-s-k/FineEnvs. The artifacts those produce live here on the Hub.

FineEnvs Projects

A growing collection of open projects, environments, resources, and artifacts.

Project	What it is	Explore
FineEnvs Academy	Articles, guides, tutorials, slides, and hands-on resources for learning how to build RL environments and agent systems.	Explore →
Data Agent	Training SLMs for data science with multi-harness RL environments.	Explore →
MiMo-V2.6-RL in Harbor	All 7,780 of Xiaomi's MiMo-V2.6 RL environments as Harbor tasks, plus an explorer to browse them and run graded rollouts.	Explore → · Explorer →
Repo2RLEnv	Verifiable coding and terminal RL environments in Harbor format, with per-task quality labels and provenance.	Explore →

Each numbered project below is a self-contained recipe: an environment, a training run, and every artifact it produced.

#	Project	What it is	Explore
00	RL Environments 101	Three environments implemented six times over, one per framework. Same logic, six dialects.	Source → · Collection →
01	LaTeX OCR	Qwen3-VL-2B trained to read rendered math into LaTeX, scored by a reward served from a live Space.	Collection →
02	Watercolour	Qwen3.5-35B-A3B trained to paint watercolours by writing p5.brush sketches, rewarded by taste rather than correctness.	Collection →
03	GeoGuesser	A multi-turn visual geolocation environment, and the 4B trained on it until it outscored `gpt-5.4-mini` and `claude-haiku-4.5`.	Collection →
04	SmolDataEnvs	5.5K+ data-analysis tasks for hill-climbing small models, graded deterministically with no LLM judge: plain prompts, verified SFT traces, and Harbor task suites.	Collection →
05	SmolDataEnvs: Multi-harness RL	One small model trained with GRPO inside four unmodified coding agents (OpenCode, Claude Code, Codex, Mini-SWE-Agent), with LFM2.5-2.6B and Qwen3.5-2B checkpoints.	Article → · Slides → · Collection →
06	Multilingual	Two OpenEnv servers: a million document pages in 22 languages with Sarvam Indic OCR Bench, and read speech in all 102 FLEURS languages. Gemma 4 trained on each to read and to hear Kannada.	Collection →
07	PortSimEnv	Re-plan a broken week of container-ship dockings at the Port of Barcelona, built from the port's real 2024 records and graded against a plan CP-SAT proves optimal.	Article → · Play → · Collection →
08	Math Environments	Three ways to grade maths: one exact arithmetic answer (Scott Wu Math), a Lean 4 proof the kernel checks (openai/math), and an olympiad proof graded 0–7 against its rubric (FineProofEnvs). Qwen3.5-2B trained on FineProofEnvs from the base model: 0.22 → 0.41 on held-out proofs.	Model → · Play → · Collection →

Articles & Talks

	What it covers	Read / Watch
📖 The Ultimate Guide to RL Environments	Building and scaling RL environments in the LLM era — how frameworks are built, how rewards are wired, how they scale to thousands of concurrent sessions.	Read →
🎞️ RL Environments 101	From "what is an env?" to training your own: RL fundamentals → environment anatomy → OpenEnv → training with TRL.	Watch →
📈 Scaling RL for LLMs	RL environments and RL training — what an environment is, how reward hacking happens, how to train against your own. AMD AI Dev Day.	Watch →
🔀 Multi-Harness Training	OpenEnv × Harbor — why an environment's failure model decides whether it can be trained against.	Watch →
🧭 The Ultimate Guide to Multi-Harness RL	Training small models on SmolDataEnvs with the same tasks and reward but a different tool loop each time (TRL, native OpenCode, Harbor), and what changes.	Read →
🎤 Training a Coding Agent Through a Harness You Did Not Write	Multi-harness RL talk by Sergio Paniego Blanco: one model, four unmodified coding agents, GRPO.	Watch →
🌍 How to turn a game into an RL environment	The technical intuition, end to end: curating the data, designing the environment, shipping it with OpenEnv, and training a 4B against it with TRL.	Read →
🚢 Simulation RL Environments	Turning real-world work into RL environments. Part 1 builds PortSimEnv from the Port of Barcelona's 2024 records and grades every plan against a proven optimum.	Read →

Environments

Three reference environments, each implemented across six frameworks — openenv, ors, nemo_gym, verifiers, skyrl_gym, gem. Same logic, six dialects. Source → · RL Envs 101 collection →

Environment	Tools	OpenEnv	ORS	NeMo Gym
Jupyter agent — real code execution in an E2B sandbox	4	Space	Space	Space
Wordle — multi-turn, pure Python, no backend	1	Space	Space	Space
Desktop — computer-use, vision-driven Linux desktop	19	Space	Space	—

Build your own

Five agent skills turn a plain-English description into a runnable RL environment across four frameworks — works with Claude Code, Cursor, Codex, OpenCode, Gemini CLI and others.

npx skills add adithya-s-k/FineEnvs

We're looking for new end-to-end recipes — a task, an environment, a training run, and honest results. Contributing guide →

Citation

@misc{fineenvs,
  author = {Kolavi, Adithya S},
  title  = {FineEnvs: Open Source RL Environments for LLM Agents},
  year   = {2026},
  url    = {https://github.com/adithya-s-k/FineEnvs}
}