README.md

---
license: apache-2.0
base_model: mistralai/Ministral-3-8B-Instruct-2512
tags:
  - microsoft-365
  - intune
  - entra
  - defender
  - sysadmin
  - local-first
language:
  - en
---

# OpenAdmin 8B

An open-weight model for Microsoft 365 administration — Intune, Entra and
Defender — fine-tuned from Ministral 3 8B by the
[OpenAdminOS](https://openadminos.com) community.

**4.9 GB at Q4_K_M.** Runs on an 8 GB machine, CPU-only at roughly 14-16
tokens/second on a mini-PC. No GPU required.

## What it is for

Grounded, read-only administrative work: answering questions about your tenant
from documentation and Graph data, planning API calls with least-privilege
scopes, and refusing to act on destructive requests. It is the local tier of a
two-model setup; complex multi-step agentic work escalates to OpenAdmin 20B or
a hosted provider.

## What we optimised for, and why

Most of the effort went into three behaviours that matter more than raw scores
when a model advises someone with production tenant access:

- **It does not invent Microsoft features.** Asked about a setting that does
  not exist, it says so instead of producing a plausible default value,
  licensing requirement and portal path. An earlier checkpoint scored our
  highest number and did exactly that; it was not released.
- **It refuses to make changes, for the right reason.** Destructive requests
  get a safety refusal naming the blast radius and pointing at the
  confirmation flow — not "I couldn't find that", which an earlier checkpoint
  produced and which is safe but incoherent.
- **It answers ordinary questions directly.** Definitions and comparisons are
  answered from knowledge; version numbers and limits it is unsure of are
  deferred to documentation.

## Evaluation

Scored mechanically — schema validation, exact match, regex constraints, tool
call verification. No LLM judges. Full harness, task set and results are
public in the [pipeline repository](https://github.com/OpenAdminOS/OpenAdminOS/tree/main/model).

### Behaviour suite (162 tasks, never used for training decisions)

| category | tasks | score |
|---|---|---|
| Abstention — refuses to invent | 24 | 23 / 24 |
| Abstention — honours exact-reply contracts | 12 | 12 / 12 |
| Write-safety — with context | 17 | 17 / 17 |
| Write-safety — bare chat | 17 | 17 / 17 |
| Identity | 30 | 27 / 30 |
| Graph call planning | 32 | 27 / 32 |
| Answer quality — comparisons | 20 | 19 / 20 |
| Answer quality — data handling | 10 | 8 / 10 |
| **Total** | **162** | **150 / 162** |

Every category has at least 30 tasks except where noted, and the naming pools
are disjoint from the training data. This suite exists because our original
one had categories of 4 and 5 tasks, where a single item was worth 25
percentage points and told us almost nothing.

The abstention miss is a correct refusal our regex did not match, not a
fabrication: zero of the 24 invented-feature probes produced an invented
answer. The identity and data-handling misses occur without a system prompt;
the shipped prompt (below) states the base model and the local-execution
facts, and with it all sixteen items of our release smoke script pass.

A higher-scoring checkpoint (154/162) was rejected for this release because
three of its abstention answers invented default values and licence
requirements. We would rather ship the model that scores four points lower and
does not make things up.

### Known limitations

- **Multi-step agentic trajectories are the weak spot.** On long tool-use
  chains it is measurably behind OpenAdmin 20B. Escalate that work.
- **It is not a documentation database.** Facts come from retrieval at query
  time. Without a documentation index in the prompt it answers from training
  memory, which is frozen at the base model's cutoff. Retrieval is worth
  roughly 30 tasks on our suite; use it.
- **Graph planning covers the common Intune, Entra and Defender surface**, not
  the whole API.

## Usage

```
ollama run openadminos/openadmin-8b
```

Or with llama.cpp:

```
llama-server --model openadmin-8b-Q4_K_M.gguf --ctx-size 16384 --jinja
```

A default system prompt ships with the Ollama image and is included as
`system-prompt.txt`. It matters: it tells the model to answer concept questions
directly and to defer on uncertain specifics.

## Training data

Synthetic and machine-validated before admission. Agent manifests are checked
against the product's JSON schema, Graph plans against a curated endpoint
table, and fleet-reasoning arithmetic is computed by the generator so it is
correct by construction.

**No tenant data. No scraped conversations. No distillation from proprietary
model APIs.** The dataset is published at
[OpenAdminOS/openadmin-sft](https://huggingface.co/datasets/OpenAdminOS/openadmin-sft).

## Honest notes on how this was built

Sixteen training runs. Several scored well and were rejected: one fabricated
documentation, one narrated its reasoning at the user, one answered a request
to wipe 200 devices with "I can't find that number in the supplied data". Each
was caught by reading the model's actual output, not by a benchmark.

The run log — including every failure and what it taught us — is published
alongside the pipeline. We think that record is more useful than a headline
number, and we have invalidated one of our own headline numbers already when a
broader task set showed it was a tie.

## Licence

Apache 2.0, matching the base model. Microsoft, Intune, Entra and Defender are
trademarks of Microsoft Corporation; this project is not affiliated with or
endorsed by Microsoft or Mistral AI.