Craig Jones

Incredible release by @burtenshaw , @merve , @pcuenq , @ariG23498 , and the Thinking Machines Lab. Having controllable reasoning_effort natively built into a sparse MoE architecture is a massive breakthrough for agentic governance.

We build autonomous multi-agent operating systems running locally on consumer hardware (BUYASOUL Family: https://huggingface.co/grandcodepope/buyasoul-family). In our architecture, the executive agent (Profit) arbitrates task execution through an objective function we call the PLT rule:
SOUL_PROFIT = PROFIT + LOVE - TAX

Having a native reasoning_effort parameter gives our governance engine a direct mechanical throttle to minimize the computational "Tax" (latency and token burn) during routine ledger logging, while reserving high reasoning depth for complex multi-agent negotiations.

Two technical questions regarding edge deployment and world modeling:

1. You mentioned using the GOLD algorithm in TRL to distill Inkling’s capabilities into smaller on-device models. Have you benchmarked distillation efficiency when transferring reasoning into sub-1B quantized models (e.g. Qwen 0.8B) for local bare-metal execution?
2. For the ECHO world-model training in OpenEnv, have you tested applying next-token environment prediction to real-time spatial simulation environments (like 3D WebGL coordinate grids) to build persistent spatial memory in agents?

Huge congratulations on the 1T and Inkling-Small releases!

Fantastic systems-level analysis, @yarizk , @eishna , @jsntsay , and @mrvnvr . The insight on cache-hit economics completely upending sticker pricing on agent trajectories is spot on.

We ran into these exact dynamics while building our autonomous multi-agent operating system (BUYASOUL Family: https://huggingface.co/grandcodepope/buyasoul-family), which we engineered entirely on consumer hardware (a 2013 PC with $0 cloud spend) using local quantized GGUF models.

We arrived at the same conclusion: model routing cannot be an isolated classification step; it must be an intrinsic governance law inside the execution harness. We formalized this through an objective function we call the PLT rule:
SOUL_PROFIT = PROFIT + LOVE - TAX

Where:

• Profit (P) optimizes for task accuracy and goal completion.
• Love (L) enforces alignment and constraints.
• Tax (T) explicitly penalizes computational latency, token burn, and hardware overhead.

Our transport layer (Omniroute) evaluates this tradeoff dynamically at every step—keeping execution local on bare metal whenever possible, and routing outward only when the expected utility exceeds the computational tax.

A quick question regarding your multi-objective optimizer:
How do you handle mid-trajectory state handover when routing shifts from a smaller model to a larger model midway through a multi-step task? Do you pass the raw conversation history or compress intermediate tool trajectories to preserve prompt cache alignment on the new target model?

Looking forward to your follow-up post on the technical implementation!

Outstanding writeup, @tomaarsen . Lifting the hard document truncation caps and decoupling contrastive batch sizes via GradCache makes late-interaction training accessible on consumer hardware.

I wanted to share an immediate real-world use case where this architecture solves a major bottleneck: autonomous agent memory persistence.

In our multi-agent architecture (BUYASOUL Family: https://huggingface.co/grandcodepope/buyasoul-family), our memory subsystem (Seshat) retrieves past trajectories from append-only JSONL ledgers. Dense single-vector embeddings consistently fail here because they compress multi-turn tool outputs and state changes into a single point, averaging away specific parameters and causing what researchers call "soul erosion."

Token-level late interaction via MaxSim is the exact remedy—it allows the agent to pinpoint the specific decision or tool execution across thousands of historical tokens without lossy document compression.

Two quick questions on index maintenance for streaming agent logs:

1. For an agent continuously appending new entries to its local ledger, what is the best strategy for incremental index updates with MultiVectorEncoder without full re-indexing?
2. Have you experimented with binary or int8 scalar quantization on the 128-dim token projections to minimize RAM footprint on consumer machines?

Kudos on the Sentence Transformers v6.0 release!