# MVP Progress ## Milestone 1: Repository Setup and Tooling Status: completed Scope completed: - Packaging and pytest configuration - Pydantic deterministic-core dependency - Ruff configuration - Optional dependency groups for OpenAI and Gradio - Environment example file - `.gitignore` for local, Python, test, build, and generated-output artifacts - Import smoke test Verification: - `python -m ruff check .` passed: `All checks passed!` - `python -m ruff format --check .` passed: `2 files already formatted` - `python -m pytest` passed: `1 passed in 0.00s` Out of scope, not started: - Vocabulary loading - Article loading - Model integration - Classification logic - Persistence - UI and root `app.py` Source-file status: - `data/gcmd_hierarchy.json` unchanged by this milestone. - `data/articles.json` unchanged by this milestone. - `prototype/app_hf_poc.py` unchanged by this milestone. ## Milestone 2: Hierarchy Loader and Vocabulary Index Status: completed Scope completed: - Deterministic loader for `data/gcmd_hierarchy.json` - Recursive traversal of the GCMD hierarchy through `Category`, `Topic`, `Term`, `Variable_Level_1`, `Variable_Level_2`, and `Variable_Level_3` - Canonical UUID-bearing concept records - Vocabulary file SHA-256 version hash - Canonical paths excluding the root `Category` - UUID, canonical path, parent, direct-child, Topic, Term, Variable, ancestor, and descendant lookups - Typed vocabulary errors for malformed hierarchy content - Focused small hierarchy fixture and full-data smoke tests Implemented models and lookup structures: - `CanonicalConceptRecord` in `src/gcmd_classifier/models.py` - `VocabularyIndex` in `src/gcmd_classifier/vocabulary/index.py` - `load_vocabulary`, `build_vocabulary_index`, and `calculate_file_hash` in `src/gcmd_classifier/vocabulary/loader.py` Hierarchy assumptions discovered from the current real file: - The root node is the non-UUID `Category` named `EARTH SCIENCE`. - The current file contains only the `EARTH SCIENCE` Category represented by that root. - Every current Topic and Term has a UUID. - The only UUID-less node in the current file is the root Category. - Current UUID-bearing record counts are: 14 Topics, 144 Terms, 1,368 `Variable_Level_1`, 1,456 `Variable_Level_2`, 553 `Variable_Level_3`, and 3,535 total UUID-bearing records. - Current observed transitions are adjacent only: `Category -> Topic -> Term -> Variable_Level_1 -> Variable_Level_2 -> Variable_Level_3`. Error cases implemented: - Invalid JSON - Non-object root or child nodes - Missing or empty required `level` and `name` - Unsupported hierarchy levels - UUID-bearing root Category - UUID-less non-root nodes - Invalid parent-child level transitions - Children under `Variable_Level_3` - Non-list `children` - Non-string `definition` - Duplicate UUIDs - Same canonical path assigned to different UUIDs - Lookup requests for unknown UUIDs or invalid lookup scopes Verification: - `python -m ruff check .` passed: `All checks passed!` - `python -m ruff format --check .` passed: `8 files already formatted` - `python -m pytest` passed: `23 passed in 0.14s` Out of scope, not started: - Article loading - Model integration - Topic routing - Term routing - Variable-level classification - Deterministic final classification validation beyond vocabulary lookup integrity - Redundancy removal - Persistence, caching, and output generation - UI and root `app.py` Source-file status: - `data/gcmd_hierarchy.json` unchanged by this milestone. - `data/articles.json` unchanged by this milestone. - `prototype/app_hf_poc.py` unchanged by this milestone. Deviations from approved plan: - None. Remaining risks or assumptions: - The loader remains fully dynamic and does not hard-code vocabulary names, UUIDs, paths, branches, or record counts, but full-data smoke tests intentionally assert current-file counts. - Future approved vocabulary rules would be needed before accepting any UUID-less non-root hierarchy node. ## Milestone 3: Article Loading and Validation Status: completed Scope completed: - Deterministic loader for `data/articles.json` - Strict source article model preserving exact serialized fields: `DOI`, `Title`, `Year`, and `Abstract` - Read/decode, individual record validation, duplicate DOI detection, and aggregate result return paths - Record-level validation errors with source index, code, field, message, and DOI when available - Valid and invalid article fixtures - Full-data smoke test for the current `data/articles.json` Implemented article model and loader APIs: - `ArticleRecord` in `src/gcmd_classifier/models.py` - `ArticleValidationIssue` and `ArticleLoadResult` in `src/gcmd_classifier/models.py` - `read_article_json`, `validate_article_record`, `validate_article_records`, and `load_articles` in `src/gcmd_classifier/articles/loader.py` Validation and error-collection policy: - File-level conditions raise typed exceptions: missing file, invalid JSON, and top-level JSON value that is not a list. - Record-level conditions are collected in `ArticleLoadResult.errors` while all remaining records continue to be validated. - Duplicate DOI records are reported with `DUPLICATE_DOI` and excluded from `ArticleLoadResult.articles` so returned records have unique DOI values. Extra-field policy: - Unexpected fields are rejected for the MVP with `UNEXPECTED_FIELD` because neither `PROJECT_SPEC.md` nor `AGENTS.md` defines an allow-list extension policy for source article records. Exact-preservation protections: - Successful `ArticleRecord` values are returned unchanged. - DOI values are not normalized or replaced. - Titles and abstracts are not stripped, lowercased, rewritten, summarized, or supplemented. - Empty `Abstract` values are valid and preserved exactly. - Whitespace-only strings are preserved; non-empty checks do not trim source values. - `Year` must be a strict integer and Boolean values are rejected. Current full-data findings: - `data/articles.json` contains 468 source records. - 467 records have valid non-empty unique DOI values. - Exactly one record, index 467, is invalid because `DOI` is empty. - 14 records have empty `Abstract` values and otherwise load successfully. Error cases covered: - File not found - Invalid JSON - Top-level JSON value not a list - Article entry not an object - Missing `DOI`, `Title`, `Year`, or `Abstract` - Empty `DOI` and `Title` - Non-string `DOI`, `Title`, or `Abstract` - String, floating-point, and Boolean `Year` - Null field values - Duplicate DOI - Unexpected extra fields - Multiple invalid records reported in one aggregate result Verification: - `python -m ruff check .` passed: `All checks passed!` - `python -m ruff format --check .` passed: `11 files already formatted` - `python -m pytest` passed: `51 passed in 0.18s` Out of scope, not started: - Model prompts - Model-provider integration - Topic routing - Term routing - Variable-level classification - Persistence - Caching - Batch processing - UI and root `app.py` Source-file status: - `data/articles.json` unchanged by this milestone. - `data/gcmd_hierarchy.json` unchanged by this milestone. - `prototype/app_hf_poc.py` unchanged by this milestone. Deviations from approved plan: - `IMPLEMENTATION_PLAN.md` was revised to reflect the clarified Milestone 3 rule that empty `Abstract` values are valid. Remaining risks or assumptions: - The current source file remains partially invalid until the empty DOI at index 467 is corrected. - Whitespace-only DOI or Title values are non-empty and therefore preserved as valid under the explicit no-trimming rule; this can be tightened later only with an approved source-data rule. ## Milestone 4: Shared Models, Output Schema Draft, and Structured Response Schemas Status: completed Scope completed: - Structured Pydantic response schemas for Topic, Term, and Variable decisions - Shared candidate decision schema with selected candidate ID, optional uncalibrated confidence, evidence, support type, and reason - Term and Variable response invariants for `stop_at_parent`, `selected`, and `stop_reason` - Separate status enums for article processing status, article classification outcome, classification final status, and review status - Article result, classification record, deterministic validation result, warning/error, processing metadata, and run summary models - Draft JSON Schema files for article classification results and run summaries - Tests for structured response validation, output model validation, status-scope separation, confidence ranges, and JSON Schema validation Structured response models implemented: - `CandidateDecision` - `TopicResponse` - `TermResponse` - `VariableResponse` Result and status models implemented: - `SupportType` - `ArticleProcessingStatus` - `ArticleClassificationOutcome` - `ClassificationFinalStatus` - `ReviewStatus` - `OutputWarning` - `OutputError` - `DeterministicValidationResult` - `ConfidenceMetadata` - `OriginalCandidateReference` - `SemanticValidationResult` - `ClassificationRecord` - `ProcessingMetadata` - `ArticleResult` - `RunSummary` Schema validation rules and invariants implemented: - Candidate IDs must be non-empty strings. - Candidate confidence, when present, must be between `0.0` and `1.0`. - Candidate evidence is required and non-empty for selected candidates. - Unknown fields are rejected in structured model responses and output models. - `stop_at_parent: true` requires an empty `selected` list. - Non-empty `selected` is represented only with `stop_at_parent: false`. - `stop_at_parent: true` requires a non-empty `stop_reason`. - Accepted and reduced classification records require successful deterministic validation. - Completed not-classified article results require an empty `classifications` list and a non-empty `no_classification_reason`. - Completed classified article results require at least one classification record. - Failed and partial article results can be represented without classification records. - Article result output preserves exact source field names: `DOI`, `Title`, `Year`, and `Abstract`. - Empty `Abstract` is valid in article result output. - Boolean `Year` is rejected by article result validation. - Review-compatible enum values are representable, but no review-trigger logic was implemented. JSON Schema files: - `schemas/classification_result.schema.json` - `schemas/run_summary.schema.json` Dependencies: - No dependency was added. `jsonschema` was already available in the active interpreter and was used for JSON Schema tests. Verification: - `python -m ruff check .` passed: `All checks passed!` - `python -m ruff format --check .` passed: `15 files already formatted` - `python -m pytest` passed: `93 passed in 0.33s` Out of scope, not started: - Live model-provider implementation - Prompt construction - Topic routing - Term routing - Variable-level traversal or classification - Semantic validation implementation - Persistence - Caching - Batch processing - UI and root `app.py` Source-file status: - `data/gcmd_hierarchy.json` unchanged by this milestone. - `data/articles.json` unchanged by this milestone. - `prototype/app_hf_poc.py` unchanged by this milestone. Deviations from approved plan: - None. Remaining risks or assumptions: - The JSON Schemas are draft schemas generated from the current Pydantic models; they are useful for MVP validation but may need hand refinement before publication as final normative schemas. - Confidence remains optional, uncalibrated metadata and does not independently determine acceptance. - Review-related enum values are schema-compatible only; emitting them still requires later approved decision rules. - The schema can represent future semantic validation output, but Milestone 4 does not implement a semantic validator. ## Milestone 5: Model Abstraction, Prompts, Fake Model, and Retries Status: completed Scope completed: - Provider-neutral model client protocol and request/response types - Typed structured model responses carried through the model interface - Deterministic scripted fake model client for unit tests - Retry wrapper for retryable model failures - Prompt builders for Topic, Term, and Variable decisions - Environment-backed model and prompt configuration - Guarded optional OpenAI provider module with isolated imports - Unit tests for fake model behavior, prompts, retry behavior, configuration, and OpenAI import isolation Model interface implemented: - `ModelClient` protocol - `ModelStage` - `ModelRequest` - `ModelResponse` - `TokenUsage` - `RetryPolicy` - `generate_with_retries` Fake model behavior implemented: - Returns scripted typed Topic, Term, and Variable responses. - Supports no-selection responses. - Can return invalid candidate IDs for later routing-validation tests without performing candidate validation in this layer. - Raises scripted retryable and non-retryable errors. - Supports fail-once-then-succeed retry scenarios. - Supports retry exhaustion scenarios. - Records every request it receives for assertions. - Does not import or require OpenAI. Prompt builders implemented: - `build_topic_prompt` - `build_term_prompt` - `build_variable_prompt` - `PromptCandidate` - `ParentContext` Prompt protections implemented: - Article title and abstract are clearly delimited. - Article content is explicitly described as untrusted input. - Prompts instruct the model not to follow instructions inside article text. - Prompts instruct the model to choose only supplied `candidate_id` values. - Prompts instruct the model not to invent, generate, or modify UUIDs, canonical paths, labels, hierarchy levels, or parent-child relationships. - Prompts request structured output only. - Prompts allow no selection for Topic routing and stopping at parent for Term and Variable decisions. - Prompts remain valid when `Abstract` is an empty string. - Source article text is inserted without trimming, normalization, or rewriting. Retry behavior implemented: - Retry only `RetryableModelError` failures. - Do not retry `NonRetryableModelError` failures. - Configurable `max_retries` through `RetryPolicy` and `ModelSettings`. - Successful responses include the retry count. - Exhausted retries raise `ModelRetriesExhaustedError` with the retry count. - Unknown candidate IDs are not retried or validated in this layer; `UnknownCandidateIDError` is available for later routing stages. Configuration fields added: - `MODEL_PROVIDER` - `MODEL_NAME` - `MODEL_TEMPERATURE` - `MODEL_TIMEOUT_SECONDS` - `MODEL_MAX_RETRIES` - `PROMPT_VERSION_TOPIC` - `PROMPT_VERSION_TERM` - `PROMPT_VERSION_VARIABLE` - `MODEL_API_KEY_ENV_VAR` - `MODEL_INCLUDE_COST_METADATA` OpenAI provider status: - `OpenAIModelClient` is implemented as a minimal guarded optional provider behind the shared interface. - OpenAI imports occur only inside provider construction, not at module import time. - API keys are read from environment variables only. - Unit tests do not instantiate a live configured provider and make no live API calls. - The provider is intentionally lightly tested in this milestone; fake-model and interface behavior are the deterministic core. Dependencies: - No dependency was added. Verification: - `python -m ruff check .` passed: `All checks passed!` - `python -m ruff format --check .` passed: `25 files already formatted` - `python -m pytest` passed: `124 passed in 0.37s` Live model/API status: - No live model API calls were made. Out of scope, not started: - Topic routing implementation - Term routing implementation - Variable traversal or end-to-end classification - Deterministic candidate validation in routing components - Semantic validation implementation - Persistence - Caching - Batch processing - Gradio UI - Root `app.py` Source-file status: - `data/gcmd_hierarchy.json` unchanged by this milestone. - `data/articles.json` unchanged by this milestone. - `prototype/app_hf_poc.py` unchanged by this milestone. Deviations from approved plan: - The OpenAI provider is minimal and guarded rather than fully integration-tested, because live provider behavior and SDK details are intentionally outside deterministic unit tests for this milestone. Remaining risks or assumptions: - Provider-specific retry classification for OpenAI SDK exception classes may need refinement when live integration tests are explicitly approved. - Prompt wording is covered by unit tests, but prompt quality still needs later evaluation with routing components. - Candidate ID validation belongs to later routing logic and was not implemented here. ## Milestone 6: Topic Routing Status: completed Scope completed: - Topic candidate construction from the loaded vocabulary index - Topic routing through the provider-neutral model interface - Structured `TopicResponse` request and parsing via the Milestone 5 model layer - Selected `candidate_id` validation against the current prompt candidate set - Topic branch seed output for later Term routing - Valid no-topic routing result without fallback behavior - Unit tests for candidate construction, Topic routing, model interaction, invalid candidates, and current full-data Topic count Topic candidate builder behavior implemented: - `build_topic_candidates` reads Topic records from `VocabularyIndex.topics()`. - Only UUID-bearing `Topic` records are converted into candidates. - The non-UUID root `Category` is never included as a candidate. - Candidate IDs are application-supplied deterministic IDs such as `topic_0001`, not authoritative UUIDs. - Candidate IDs map back to authoritative Topic UUIDs in application code. - Prompt candidates include candidate ID, Topic name, level, optional definition, and canonical path context. - No Topic names, UUIDs, paths, or counts are hard-coded in application behavior. Topic router behavior implemented: - `route_topics` builds Topic candidates, creates a Topic prompt, and calls `generate_with_retries` using the neutral `ModelClient` interface. - The router requests the `TopicResponse` schema. - Selected candidate IDs are mapped back to authoritative Topic records from the vocabulary index. - Each selected Topic produces a `TopicBranchSeed` containing branch ID, Topic UUID, Topic name, level, canonical path, evidence, support type, confidence, model reason, original candidate ID, prompt version, provider, model name, and retry count. - The router performs Topic routing only and does not route Terms or Variables. No-topic behavior implemented: - Empty Topic selections return a successful `TopicRoutingResult` with no branches. - `no_selection_reason` is preserved when supplied by the model. - No fallback Topic is created. - No fake `FALLBACK` behavior from the proof of concept is used. Invalid candidate behavior implemented: - Unknown selected candidate IDs raise `UnknownCandidateIDError`. - Duplicate selected candidate IDs raise `UnknownCandidateIDError`. - Empty or malformed candidate IDs fail structured model response validation before routing accepts them. - Invalid Topic candidate IDs are not mapped to vocabulary records and are not silently dropped. Fake-model tests added: - Single Topic selection - Multiple Topic selection - No Topic selection - Invalid candidate ID selection - Empty/malformed candidate ID rejection - Provider-neutral request assertions - Empty `Abstract` prompt pass-through - No live provider usage Current full-data smoke assertion: - The current `data/gcmd_hierarchy.json` builds 14 Topic candidates for the EARTH SCIENCE vocabulary. - All current full-data Topic candidates map to UUID-bearing Topic records. - This count is a current-file smoke assertion only and is not hard-coded into application behavior. Verification: - `python -m ruff check .` passed: `All checks passed!` - `python -m ruff format --check .` passed: `29 files already formatted` - `python -m pytest` passed: `143 passed in 0.43s` Live model/API status: - No live model API calls were made. - Unit tests use `FakeModelClient` only. Out of scope, not started: - Term routing - Variable traversal - End-to-end classification - Deterministic final classification validation - Redundancy removal - Persistence - Caching - Batch processing - Gradio UI - Root `app.py` Source-file status: - `data/gcmd_hierarchy.json` unchanged by this milestone. - `data/articles.json` unchanged by this milestone. - `prototype/app_hf_poc.py` unchanged by this milestone. Deviations from approved plan: - Tests were added in `tests/test_topic_routing.py` rather than `tests/test_pipeline.py` to keep this milestone focused on Topic routing instead of pipeline behavior. Remaining risks or assumptions: - Candidate IDs are stable for deterministic vocabulary order; if a future vocabulary inserts or reorders Topics, candidate IDs may shift for that prompt, but each prompt still carries its own candidate-to-record mapping. - Topic routing currently raises explicit errors for unknown or duplicate selected IDs rather than returning partial valid selections with warnings. - Confidence is preserved as optional uncalibrated metadata and is not used to independently accept a Topic. ## Milestone 7: Term Routing Status: completed Scope completed: - Direct Term candidate construction beneath one selected Topic branch - Term routing through the provider-neutral model interface - Structured `TermResponse` request and parsing via the Milestone 5 model layer - Selected Term `candidate_id` validation against the current prompt candidate set - Topic-to-Term direct-child relationship validation - Term branch seed output for later Variable routing - Successful stop-at-Topic routing result when no Term is adequately supported - Unit tests for Term candidate construction, Term routing, stop behavior, invalid candidates, relationship validation, model interaction, and current full-data Term count Term candidate builder behavior implemented: - `build_term_candidates` accepts a selected Topic UUID and reads only `VocabularyIndex.terms_for_topic(topic_uuid)`. - Only UUID-bearing direct `Term` children of the selected Topic are converted into candidates. - Terms from sibling Topics are not included. - Candidate IDs are application-supplied deterministic IDs such as `term_0001`, scoped to the current Term prompt. - Candidate IDs map back to authoritative Term UUIDs in application code. - Prompt candidates include candidate ID, Term name, level, optional definition, canonical path context, and parent Topic context. - No Term names, UUIDs, paths, counts, or Topic-to-Term relationships are hard-coded in application behavior. Term router behavior implemented: - `route_terms` consumes a Milestone 6 `TopicBranchSeed`. - The router builds direct Term candidates only for the selected Topic. - The router creates a Term prompt with selected parent Topic context and direct-child Term candidates. - The router calls `generate_with_retries` using the neutral `ModelClient` interface. - The router requests the `TermResponse` schema. - Selected candidate IDs are mapped back to authoritative Term records from the vocabulary index. - Each selected Term produces a `TermBranchSeed` containing branch ID, parent Topic UUID/name, Term UUID, Term name, level, canonical path, evidence, support type, confidence, model reason, original candidate ID, parent branch ID, prompt version, provider, model name, and retry count. - Multiple selected Terms produce independent branch seeds. - The router performs Term routing only and does not route Variables. Stop-at-Topic behavior implemented: - `stop_at_parent: true` returns a successful `TopicStopResult`. - `stop_reason` is preserved. - Parent Topic UUID, name, level, canonical path, evidence, support type, confidence, and candidate ID are preserved. - No Term branch seeds are created for stop-at-Topic results. - Stopping at the Topic is not treated as a system failure. - No fake or fallback Terms are created. Invalid candidate behavior implemented: - Unknown selected Term candidate IDs raise `UnknownCandidateIDError`. - Duplicate selected Term candidate IDs raise `UnknownCandidateIDError`. - Empty or malformed candidate IDs fail structured model response validation before routing accepts them. - Invalid Term candidate IDs are not mapped to vocabulary records and are not silently dropped. Topic-to-Term relationship validation implemented: - `validate_term_candidate_relationship` verifies that each Term candidate belongs to the selected Topic. - It checks both the candidate's stored Topic UUID and the vocabulary index parent relationship. - A manually corrupted candidate-to-record mapping is rejected. - A sibling Topic Term cannot be accepted because it is absent from the selected Topic prompt candidate set. Fake-model tests added: - One Term selected under a selected Topic - Multiple Terms under one selected Topic - Stop at Topic with `stop_at_parent: true` - Topic with no direct/supported Term returning a stop result - Invalid Term candidate ID selection - Duplicate Term candidate ID selection - Empty/malformed candidate ID rejection - Provider-neutral request assertions - Empty `Abstract` prompt pass-through - No live provider usage Current full-data smoke assertion: - The current `data/gcmd_hierarchy.json` builds 144 direct Term candidates across all current Topics. - All current full-data Term candidates map to UUID-bearing Term records. - Current Term UUIDs are unique across Topic candidate lists. - This count is a current-file smoke assertion only and is not hard-coded into application behavior. Verification: - `python -m ruff check .` passed: `All checks passed!` - `python -m ruff format --check .` passed: `30 files already formatted` - `python -m pytest` passed: `168 passed in 0.58s` Live model/API status: - No live model API calls were made. - Unit tests use `FakeModelClient` only. Out of scope, not started: - Variable traversal - End-to-end classification - Final deterministic classification validation - Redundancy removal - Persistence - Caching - Batch processing - Gradio UI - Root `app.py` Source-file status: - `data/gcmd_hierarchy.json` unchanged by this milestone. - `data/articles.json` unchanged by this milestone. - `prototype/app_hf_poc.py` unchanged by this milestone. Deviations from approved plan: - Tests were added in `tests/test_term_routing.py` rather than `tests/test_pipeline.py` to keep this milestone focused on Term routing instead of pipeline behavior. - No `classification/traversal.py` module was added because the necessary branch helpers fit cleanly in the existing candidate/router modules without introducing a premature abstraction. Remaining risks or assumptions: - Term candidate IDs are stable for deterministic direct-child order within a prompt; if vocabulary order changes, candidate IDs may shift, but each prompt still carries its own candidate-to-record mapping. - Term routing raises explicit errors for unknown or duplicate selected IDs rather than returning partial valid selections with warnings. - Confidence is preserved as optional uncalibrated metadata and is not used to independently accept a Term. ## Milestone 8: Variable-Level Controlled Descent Status: completed Scope completed: - Direct Variable candidate construction beneath Term, `Variable_Level_1`, and `Variable_Level_2` parents - Recursive controlled descent through Variable levels using only direct children - Structured `VariableResponse` request and parsing via the Milestone 5 model layer - Selected Variable `candidate_id` validation against the current prompt candidate set - Direct parent-to-child relationship validation for selected Variables - Terminal branch outcomes for stop-at-parent, no-child parent, and leaf-node cases - Branch provenance and evidence trail tracking - Branch-level error reporting that preserves successful sibling outcomes - Unit tests for Variable candidate construction, controlled descent, stopping, leaf handling, multi-branch behavior, invalid candidates, provenance, model interaction, and current full-data Variable counts Variable candidate builder behavior implemented: - `build_variable_candidates` accepts a current parent UUID and reads only `VocabularyIndex.variables_for_parent(parent_uuid)`. - Supported parents are `Term`, `Variable_Level_1`, and `Variable_Level_2` records. - Only UUID-bearing direct Variable children are converted into candidates. - Grandchildren, sibling branches, and global Variables are not included. - Candidate IDs are application-supplied deterministic IDs such as `variable_0001`, scoped to the current Variable prompt. - Candidate IDs map back to authoritative Variable UUIDs in application code. - Prompt candidates include candidate ID, Variable name, level, optional definition, canonical path context, and parent context. - No Variable names, UUIDs, paths, counts, or parent-child relationships are hard-coded in application behavior. Controlled descent behavior implemented: - `descend_variables` starts from a Milestone 7 `TermBranchSeed`. - Each parent evaluation calls `generate_with_retries` using the neutral `ModelClient` interface and requests `VariableResponse`. - Each model call receives only direct Variable children of the current parent. - Selected children create independent child branches with distinct branch IDs. - Descent continues recursively for selected child branches only. - Grandchildren are not evaluated before their parent is selected. - The traversal does not skip hierarchy levels and does not evaluate Variables globally. Stop-at-parent behavior implemented: - A parent with no direct Variable children returns a successful terminal outcome at the current parent. - `stop_at_parent: true` returns a successful terminal outcome at the current parent and preserves `stop_reason`. - `Variable_Level_3` selections stop naturally as leaf-node terminal outcomes. - Term, `Variable_Level_1`, and `Variable_Level_2` parents can all become terminal outcomes when descent is unsupported. - No fake or fallback Variable is created. - Descent is not forced to a leaf. Multi-branch behavior implemented: - Multiple selected child Variables create independent branch outcomes. - Child branch IDs preserve the parent branch lineage. - Sibling branches continue independently when another sibling stops. - Branch-level failures are recorded as `VariableBranchError` entries and do not discard successful sibling terminal outcomes. Invalid candidate behavior implemented: - Unknown selected Variable candidate IDs raise `UnknownCandidateIDError` at the current branch. - Duplicate selected Variable candidate IDs raise `UnknownCandidateIDError` at the current branch. - Empty or malformed candidate IDs fail structured model response validation before traversal accepts them. - Invalid Variable candidate IDs are not mapped to vocabulary records and are not silently dropped. - `validate_variable_candidate_relationship` rejects candidates that are not direct children of the current parent. Branch provenance behavior implemented: - `VariableTerminalOutcome` includes branch ID, parent branch ID, Topic UUID/name, Term UUID/name, final UUID/name/level/canonical path/path components, evidence, support type, confidence, reason, stop reason, candidate ID, prompt/model metadata, and retry count when available. - `EvidenceStep` preserves the branch lineage and evidence trail from Term through selected Variable stages. - Confidence remains optional, uncalibrated metadata and does not independently determine acceptance. Fake-model tests added: - Stop at Term - Stop at `Variable_Level_1` - Stop at `Variable_Level_2` - Select `Variable_Level_3` - Natural leaf-node handling - Multiple independent `Variable_Level_1` branches - Multiple selected descendants under one branch - Sibling continuation after one branch stops - Sibling continuation after one branch fails - Unknown, duplicate, empty, and malformed candidate IDs - Provider-neutral request assertions - Empty `Abstract` prompt pass-through - No live provider usage Current full-data smoke assertion: - The current `data/gcmd_hierarchy.json` builds direct Variable candidates matching current counts: 1,368 `Variable_Level_1`, 1,456 `Variable_Level_2`, and 553 `Variable_Level_3` records. - Every current full-data Variable candidate maps to a UUID-bearing Variable record. - Every current full-data Variable candidate is a direct child of the parent used to build it. - These counts are current-file smoke assertions only and are not hard-coded into application behavior. Verification: - `python -m ruff check .` passed: `All checks passed!` - `python -m ruff format --check .` passed: `32 files already formatted` - `python -m pytest` passed: `197 passed in 0.58s` Live model/API status: - No live model API calls were made. - Unit tests use `FakeModelClient` only. Out of scope, not started: - Redundancy removal - Final deterministic classification validation - Final article-level output construction - Persistence - Caching - Batch processing - Gradio UI - Root `app.py` Source-file status: - `data/gcmd_hierarchy.json` unchanged by this milestone. - `data/articles.json` unchanged by this milestone. - `prototype/app_hf_poc.py` unchanged by this milestone. Deviations from approved plan: - Tests were added in `tests/test_variable_traversal.py` rather than `tests/test_pipeline.py` to keep this milestone focused on Variable descent instead of pipeline behavior. - `classification/traversal.py` was added as planned for the reusable descent engine. - Branch-level partial behavior was implemented only for selected child branches; an initial Term-parent model failure still propagates because there are no sibling Variable branches yet at that point. Remaining risks or assumptions: - Variable candidate IDs are stable for deterministic direct-child order within a prompt; if vocabulary order changes, candidate IDs may shift, but each prompt still carries its own candidate-to-record mapping. - Branch-level errors are represented for traversal continuity but are not yet converted into final article-level statuses. - Terminal outcomes are routing/descent outcomes only; Milestone 9 still needs final deterministic validation and redundancy removal. ## Milestone 9: Deterministic Validation and Redundancy Removal Status: completed Scope completed: - Deterministic validation of terminal classification candidates against `VocabularyIndex` - Narrow conversion from validated routing/traversal candidates into final `ClassificationRecord` objects - Structural diagnostics for rejected invalid candidates - Redundancy removal for duplicate UUIDs, duplicate canonical paths, and same-lineage ancestor classifications - Branch-provenance-aware preservation of independent ancestor classifications - Unit tests for validation, conversion, diagnostics, redundancy removal, and full-data representative validation Deterministic validation behavior implemented: - `ClassificationCandidate` represents a terminal candidate awaiting deterministic vocabulary validation. - `validate_candidate` confirms that the final UUID exists in the loaded vocabulary and is assignable. - Supplied names, levels, canonical paths, path components, Topic context, Term context, and expected parent UUIDs are verified against authoritative vocabulary records when present. - Parent-child ancestry is checked through `VocabularyIndex` relationships. - Topic membership and Term membership are checked with UUID ancestry rather than model-supplied paths. - Root `Category` candidates are rejected explicitly and never accepted as final classifications. - Invented UUIDs, labels, levels, paths, and hierarchy relationships produce structured validation errors. - Validation is structural only and does not make independent semantic-support decisions. Final classification record conversion behavior implemented: - Valid candidates are converted to `ClassificationRecord` only after deterministic validation succeeds. - Accepted records use authoritative UUID, name, level, canonical path, path components, Topic, Term, and parent UUID from the vocabulary index. - Routing/traversal metadata is preserved where available, including branch ID, evidence, support type, optional uncalibrated confidence, reason or stop reason, and validation result. - Invalid candidates are returned as rejected validation results and do not appear in accepted `classifications`. Redundancy removal behavior implemented: - `remove_redundant_classifications` removes exact duplicate UUIDs. - Exact duplicate canonical paths are removed even when the UUID differs. - Ancestors are removed when a deeper descendant exists in the same branch lineage. - Ancestor classifications are preserved when branch provenance indicates an independent decision. - When provenance is insufficient, both ancestor and descendant are preserved with a warning. - Sibling classifications are preserved. - Same concept names in different branches are preserved when UUIDs and paths differ. - Redundancy uses `VocabularyIndex.is_ancestor`, not string-prefix path matching, as the primary ancestry test. Diagnostic behavior implemented: - Rejected validation candidates include structured `OutputError` diagnostics with error code, stage, field, expected value, and actual value where useful. - Redundancy actions produce structured `OutputWarning` diagnostics for duplicate UUID removal, duplicate canonical-path removal, same-branch ancestor removal, independent ancestor preservation, and insufficient-provenance preservation. Tests added: - `tests/test_validation.py` - `tests/test_redundancy.py` Verification: - `python -m ruff check .` passed: `All checks passed!` - `python -m ruff format --check .` passed: `36 files already formatted` - `python -m pytest` passed: `230 passed in 0.69s` Live model/API status: - No model calls were made. - No live API calls were made. Out of scope, not started: - Independent semantic validation - Persistence - Caching - Batch processing - Gradio UI - Root `app.py` - End-to-end article-level orchestration Source-file status: - `data/gcmd_hierarchy.json` unchanged by this milestone. - `data/articles.json` unchanged by this milestone. - `prototype/app_hf_poc.py` unchanged by this milestone. Deviations from approved plan: - Tests were added in focused `tests/test_validation.py` and `tests/test_redundancy.py` files, matching the approved likely file list. - `src/gcmd_classifier/models.py` and `src/gcmd_classifier/errors.py` did not require changes because existing structured output and error models were sufficient. Remaining risks or assumptions: - Candidate ID provenance is retained as structured warning metadata on converted records rather than adding a new generated output field to `ClassificationRecord` in this milestone. - Non-assignable concept rejection is implemented, but the current vocabulary fixture contains only assignable UUID-bearing records. - Redundancy cleanup is collection-level only; integration into final article result assembly remains deferred to later milestones. ## Milestone 10: Batch Runner, Incremental Persistence, Caching, and Logging Status: completed Scope completed: - Single-article orchestration through Topic routing, Term routing, Variable descent, deterministic validation, and redundancy removal - Article-level `ArticleResult` generation for classified, not-classified, partial, and failed outcomes - Batch processing over validated article records with continuation after article failures - Invalid source article diagnostics without fallback DOI generation - Incremental JSON checkpoint persistence after each article result - Final consolidated machine-readable JSON output - File-backed cache identity and cache reuse - Structured logging helpers with secret redaction - Run summary counters for valid/invalid records, processed articles, cache hits/misses, warnings, errors, and accepted classifications Single-article pipeline behavior implemented: - `classify_article` coordinates existing Milestone 6-9 components. - No-topic results become `processing_status: completed`, `classification_outcome: not_classified`, empty classifications, and a no-classification reason. - Stop-at-Topic, stop-at-Term, and Variable terminal outcomes are converted only after deterministic validation succeeds. - Branch-level Variable descent errors are retained as structured errors while successful sibling terminal outcomes are preserved. - Articles with accepted classifications and branch errors become `processing_status: partial` with the accepted classifications retained. - Failed articles retain source fields exactly and include structured errors. Batch runner behavior implemented: - `run_batch` accepts an `ArticleLoadResult`, processes only valid records in source order, and reports invalid source records in the summary. - One article failure does not stop later valid articles. - Each article result is saved promptly after completion or failure. - Batch output avoids duplicate article files for the same DOI-safe key by overwriting the DOI checkpoint path. - The runner supports `force_reprocess` to bypass cache reuse. Persistence strategy selected and implemented: - The MVP uses one JSON checkpoint file per DOI-safe SHA-256 key plus an optional consolidated `results.json`. - This append/checkpoint-style representation avoids rewriting a single large output document after every article. - Individual article checkpoint writes and consolidated output writes use temp-file-plus-atomic-replace. - Temporary and destination files are created in the same destination directory, so atomic replace occurs on the same filesystem. - No database infrastructure was introduced. Cache identity and reuse behavior implemented: - `CacheIdentity` includes DOI, exact article fingerprint, vocabulary version/hash, model provider, model name, model temperature, timeout, max retries, Topic/Term/Variable prompt versions, application version, and relevant configuration hash. - Cache keys change when article content, vocabulary version, model settings, prompt versions, application version, or relevant configuration changes. - Cache hits return article results with `processing_metadata.cache_used: true`. - Cache hits remain completed work and are never marked `skipped`. - Force reprocessing bypasses cache reuse. - Stale cache entries are not reused when identity inputs differ. Logging and diagnostics behavior implemented: - `log_event` records structured events through standard logging. - `sanitize_log_details` redacts keys that look like API keys, tokens, authorization headers, secrets, passwords, or sensitive headers. - Pipeline and batch events include DOI, stage, branch/cache context, status, classification counts, and error counts when available. - Structured `OutputError` and `OutputWarning` models are used for article, branch, validation, redundancy, and invalid-source diagnostics. Run summary behavior implemented: - `RunSummary` now includes explicit `valid_article_records`, `invalid_source_records`, `processed_articles`, `cache_hits`, `cache_misses`, `total_warnings`, `total_errors`, and `duration_seconds` fields. - `schemas/run_summary.schema.json` was regenerated from the updated Pydantic model. - Summary output remains machine-readable and validates against the draft JSON Schema. Tests added: - `tests/test_cache.py` - `tests/test_persistence.py` - `tests/test_pipeline.py` Verification: - `python -m ruff check .` passed: `All checks passed!` - `python -m ruff format --check .` passed: `46 files already formatted` - `python -m pytest` passed: `261 passed in 0.77s` Live model/API status: - No live model API calls were made. - Unit tests use `FakeModelClient` only. Out of scope, not started: - Gradio UI - Hugging Face launcher - Root `app.py` - Live-model smoke tests - Independent semantic validation - Production deployment infrastructure Source-file status: - `data/gcmd_hierarchy.json` unchanged by this milestone. - `data/articles.json` unchanged by this milestone. - `prototype/app_hf_poc.py` unchanged by this milestone. Deviations from approved plan: - `src/gcmd_classifier/errors.py` did not require changes because existing typed exceptions and structured `OutputError` diagnostics were sufficient. - `src/gcmd_classifier/config.py` did not require changes because existing model settings already covered the cache identity inputs needed for this milestone. - `RunSummary` and `schemas/run_summary.schema.json` were updated to represent required Milestone 10 counters explicitly. Remaining risks or assumptions: - The checkpoint store uses DOI-safe SHA-256 filenames, so humans inspect consolidated JSON rather than filename-readable DOI values. - Cached failed and partial results can be reused when the cache identity matches exactly; future policy may choose to cache only completed results. - Batch processing is sequential for the MVP. Parallel execution remains deferred. - Run summary model-call totals are available for fake clients and providers that expose request counts or metadata; richer token/cost aggregation remains provider-dependent. ## Milestone 11: Gradio Integration and Hugging Face Launcher Status: completed Scope completed: - Lightweight Gradio demonstration UI package under `src/gcmd_classifier/ui/` - `create_demo()` factory for a Gradio `Interface` - Root-level thin Hugging Face launcher in `app.py` - UI formatting for classifications, UUIDs, canonical paths, hierarchy levels, evidence, support type, confidence metadata, stopping reason, deterministic validation status, no-classification reason, warnings, and errors - UI tests for importability, construction with a fake Gradio module, pipeline delegation, formatting helpers, empty `Abstract` handling, root launcher import behavior, and frozen prototype immutability `create_demo()` behavior implemented: - Builds a Gradio interface with `Title`, `Abstract`, optional `DOI`, and optional `Year` inputs. - Returns summary Markdown, detailed `ArticleResult` JSON, and structured diagnostics JSON outputs. - Defers all classification work until the user submits input. - Uses `ModelSettings.from_environment()` for provider/model configuration. - Uses a guarded Gradio import so non-UI modules and deterministic tests can run without Gradio import success. Root `app.py` launcher behavior implemented: - Imports only `create_demo` from `gcmd_classifier.ui.gradio_app`. - Creates `demo = create_demo()` for Hugging Face Spaces compatibility. - Calls `demo.launch()` only under `if __name__ == "__main__"`. - Contains no routing, validation, model-provider, or classification business logic. How the UI calls the pipeline service: - `run_demo_classification` constructs an `ArticleRecord` from UI inputs. - It loads the configured vocabulary when one is not injected. - It creates a model client through a factory and calls `pipeline_service.classify_article`. - It does not call Topic routing, Term routing, Variable descent, deterministic validation, redundancy removal, OpenAI, or any model provider directly. Display behavior implemented: - Classified results render a Markdown table with level, name, UUID, canonical path, evidence, support type, confidence metadata, stopping reason, and deterministic validation status. - No-classification results display the no-classification reason. - Warnings and errors appear in both the summary panel and structured diagnostics JSON. - Detailed output returns the full `ArticleResult` JSON payload. - Empty `Abstract` values are accepted and passed through to the pipeline input path. Dependency handling: - No dependency changes were made. - Gradio remains an optional dependency group in `pyproject.toml`. - In the active interpreter, importing Gradio fails because an optional audio dependency is unavailable (`pyaudioop`), so tests use a fake Gradio module for construction checks while keeping the real UI import guarded. - No API keys or secrets are hard-coded or printed. Tests added: - `tests/test_ui.py` Verification: - `python -m ruff check .` passed: `All checks passed!` - `python -m ruff format --check .` passed: `50 files already formatted` - `python -m pytest` passed: `272 passed in 0.78s` Live model/API status: - No live model API calls were made in tests. - UI tests use injected fake Gradio and fake pipeline/model behavior only. Out of scope, not started: - Production review UI - Live-model smoke testing - External deployment automation - Milestone 12 evaluation work Source-file status: - `data/gcmd_hierarchy.json` unchanged by this milestone. - `data/articles.json` unchanged by this milestone. - `prototype/app_hf_poc.py` unchanged by this milestone. Deviations from approved plan: - Tests use a fake Gradio module for `create_demo()` construction because the active interpreter cannot import Gradio successfully. The UI still constructs a real Gradio interface when the optional dependency imports correctly. - The UI supports a deterministic fake/demo model mode by default via `FakeModelClient`, but the fake response still runs through the pipeline service and is clearly represented as fake demo behavior. - When DOI is omitted in manual UI input, the UI creates a deterministic `demo-ui:` DOI and emits a UI warning; source JSON article loading behavior is unchanged and still never invents identifiers. Remaining risks or assumptions: - Manual live-provider use depends on optional provider dependencies and environment variables being available in the runtime environment. - The demo UI is intentionally simple and does not implement production human-review workflows. - Root `app.py` requires a working Gradio optional dependency in the deployment environment, as expected for Hugging Face Spaces. ## Milestone 12: MVP Smoke/Evaluation Harness Status: completed Scope completed: - Pytest-based MVP smoke/evaluation harness - Fake-model end-to-end smoke run over a small current-data subset - Optional live-model integration test marked `integration` and skipped by default - Pytest marker registration for integration tests - Baseline preservation checks for `prototype/app_hf_poc.py` - Source-file immutability checks for current data files Smoke/evaluation harness behavior implemented: - The smoke command is `python -m pytest tests/test_mvp_smoke.py`. - The smoke test loads the current full vocabulary from `data/gcmd_hierarchy.json`. - The smoke test loads and validates the current full article source from `data/articles.json`. - It verifies the current invalid source record is reported in diagnostics rather than silently ignored. - It processes only two valid articles in routine tests to avoid full-data model processing. - It uses the existing batch runner, cache, JSON store, pipeline service, routing, traversal, deterministic validation, redundancy removal, and output models. - It validates article results and run summary output against the existing JSON Schemas. Fake-model smoke behavior implemented: - The fake smoke script dynamically selects a real Topic, Term, and Variable branch from the loaded vocabulary without hard-coding names, UUIDs, paths, or counts into application behavior. - One fake article produces an accepted classification through Topic routing, Term routing, Variable descent, deterministic validation, redundancy removal, article result creation, and persistence. - One fake article produces a completed no-classification result. - Accepted classifications are asserted to have `deterministic_validation.valid: true`. - The fake smoke run makes no live API calls. Optional live integration behavior: - `tests/test_mvp_smoke.py` includes an optional `pytest.mark.integration` live smoke test. - It is skipped unless `GCMD_RUN_LIVE_INTEGRATION=1` is set. - It is skipped unless `MODEL_PROVIDER=openai` is configured. - It is skipped unless the configured API key environment variable is present. - Normal `python -m pytest` does not run live model calls. - If explicitly enabled, the live test processes only one article and asserts structural validity only. Metrics recorded: - Total source records considered - Valid article records - Invalid source records - Articles processed in the smoke run - Classified articles - Not-classified articles - Partial articles - Failed articles - Accepted classifications - Deterministic validation failures through accepted-record assertions - Warnings and errors - Model calls - Cache hits and misses - Processing start time - Processing end time - Duration - Token usage and estimated cost remain available when providers supply them Baseline preservation behavior: - The smoke harness reads `prototype/app_hf_poc.py` bytes before and after the fake smoke run and confirms they are unchanged. - The smoke harness asserts the prototype module is not imported during the fake smoke run. - No prototype execution, refactor, or comparison study was added. Verification: - `python -m ruff check .` passed: `All checks passed!` - `python -m ruff format --check .` passed: `51 files already formatted` - `python -m pytest` passed: `273 passed, 1 skipped in 0.89s` Live model/API status: - Normal tests made no live model API calls. - The single skipped test is the optional live integration smoke test. Out of scope, not started: - Full expert-reviewed evaluation dashboard - Publication benchmark - Production monitoring system - Model-comparison framework - Full proof-of-concept comparison study Source-file status: - `data/gcmd_hierarchy.json` unchanged by this milestone. - `data/articles.json` unchanged by this milestone. - `prototype/app_hf_poc.py` unchanged by this milestone. Deviations from approved plan: - No standalone script was added because a pytest smoke harness is the simplest maintainable command and was explicitly allowed. - `pyproject.toml` was updated only to register the `integration` pytest marker. Remaining risks or assumptions: - Fake-model smoke validates structural pipeline behavior, not scientific correctness. - Live integration remains opt-in and depends on optional dependencies, credentials, and provider availability. - Future baseline comparison with the frozen proof of concept remains deferred.