Spaces:

igerasimov
/

GCMD_Keyword_Classifier_MVP

Running on Zero

Deterministic loader for data/gcmd_hierarchy.json
Recursive traversal of the GCMD hierarchy through Category, Topic, Term, Variable_Level_1, Variable_Level_2, and Variable_Level_3
Canonical UUID-bearing concept records
Vocabulary file SHA-256 version hash
Canonical paths excluding the root Category
UUID, canonical path, parent, direct-child, Topic, Term, Variable, ancestor, and descendant lookups
Typed vocabulary errors for malformed hierarchy content
Focused small hierarchy fixture and full-data smoke tests

Implemented models and lookup structures:

CanonicalConceptRecord in src/gcmd_classifier/models.py
VocabularyIndex in src/gcmd_classifier/vocabulary/index.py
load_vocabulary, build_vocabulary_index, and calculate_file_hash in src/gcmd_classifier/vocabulary/loader.py

Hierarchy assumptions discovered from the current real file:

The root node is the non-UUID Category named EARTH SCIENCE.
The current file contains only the EARTH SCIENCE Category represented by that root.
Every current Topic and Term has a UUID.
The only UUID-less node in the current file is the root Category.
Current UUID-bearing record counts are: 14 Topics, 144 Terms, 1,368 Variable_Level_1, 1,456 Variable_Level_2, 553 Variable_Level_3, and 3,535 total UUID-bearing records.
Current observed transitions are adjacent only: Category -> Topic -> Term -> Variable_Level_1 -> Variable_Level_2 -> Variable_Level_3.

Error cases implemented:

Invalid JSON
Non-object root or child nodes
Missing or empty required level and name
Unsupported hierarchy levels
UUID-bearing root Category
UUID-less non-root nodes
Invalid parent-child level transitions
Children under Variable_Level_3
Non-list children
Non-string definition
Duplicate UUIDs
Same canonical path assigned to different UUIDs
Lookup requests for unknown UUIDs or invalid lookup scopes

Verification:

python -m ruff check . passed: All checks passed!
python -m ruff format --check . passed: 8 files already formatted
python -m pytest passed: 23 passed in 0.14s

Out of scope, not started:

Article loading
Model integration
Topic routing
Term routing
Variable-level classification
Deterministic final classification validation beyond vocabulary lookup integrity
Redundancy removal
Persistence, caching, and output generation
UI and root app.py

Source-file status:

data/gcmd_hierarchy.json unchanged by this milestone.
data/articles.json unchanged by this milestone.
prototype/app_hf_poc.py unchanged by this milestone.

Deviations from approved plan:

None.

Remaining risks or assumptions:

The loader remains fully dynamic and does not hard-code vocabulary names, UUIDs, paths, branches, or record counts, but full-data smoke tests intentionally assert current-file counts.
Future approved vocabulary rules would be needed before accepting any UUID-less non-root hierarchy node.

Milestone 3: Article Loading and Validation

Status: completed

Scope completed:

Deterministic loader for data/articles.json
Strict source article model preserving exact serialized fields: DOI, Title, Year, and Abstract
Read/decode, individual record validation, duplicate DOI detection, and aggregate result return paths
Record-level validation errors with source index, code, field, message, and DOI when available
Valid and invalid article fixtures
Full-data smoke test for the current data/articles.json

Implemented article model and loader APIs:

ArticleRecord in src/gcmd_classifier/models.py
ArticleValidationIssue and ArticleLoadResult in src/gcmd_classifier/models.py
read_article_json, validate_article_record, validate_article_records, and load_articles in src/gcmd_classifier/articles/loader.py

Validation and error-collection policy:

File-level conditions raise typed exceptions: missing file, invalid JSON, and top-level JSON value that is not a list.
Record-level conditions are collected in ArticleLoadResult.errors while all remaining records continue to be validated.
Duplicate DOI records are reported with DUPLICATE_DOI and excluded from ArticleLoadResult.articles so returned records have unique DOI values.

Extra-field policy:

Unexpected fields are rejected for the MVP with UNEXPECTED_FIELD because neither PROJECT_SPEC.md nor AGENTS.md defines an allow-list extension policy for source article records.

Exact-preservation protections:

Successful ArticleRecord values are returned unchanged.
DOI values are not normalized or replaced.
Titles and abstracts are not stripped, lowercased, rewritten, summarized, or supplemented.
Empty Abstract values are valid and preserved exactly.
Whitespace-only strings are preserved; non-empty checks do not trim source values.
Year must be a strict integer and Boolean values are rejected.

Current full-data findings:

data/articles.json contains 468 source records.
467 records have valid non-empty unique DOI values.
Exactly one record, index 467, is invalid because DOI is empty.
14 records have empty Abstract values and otherwise load successfully.

Error cases covered:

File not found
Invalid JSON
Top-level JSON value not a list
Article entry not an object
Missing DOI, Title, Year, or Abstract
Empty DOI and Title
Non-string DOI, Title, or Abstract
String, floating-point, and Boolean Year
Null field values
Duplicate DOI
Unexpected extra fields
Multiple invalid records reported in one aggregate result

Verification:

python -m ruff check . passed: All checks passed!
python -m ruff format --check . passed: 11 files already formatted
python -m pytest passed: 51 passed in 0.18s

Out of scope, not started:

Model prompts
Model-provider integration
Topic routing
Term routing
Variable-level classification
Persistence
Caching
Batch processing
UI and root app.py

Source-file status:

data/articles.json unchanged by this milestone.
data/gcmd_hierarchy.json unchanged by this milestone.
prototype/app_hf_poc.py unchanged by this milestone.

Deviations from approved plan:

IMPLEMENTATION_PLAN.md was revised to reflect the clarified Milestone 3 rule that empty Abstract values are valid.

Remaining risks or assumptions:

The current source file remains partially invalid until the empty DOI at index 467 is corrected.
Whitespace-only DOI or Title values are non-empty and therefore preserved as valid under the explicit no-trimming rule; this can be tightened later only with an approved source-data rule.

Milestone 4: Shared Models, Output Schema Draft, and Structured Response Schemas

Status: completed

Scope completed:

Structured Pydantic response schemas for Topic, Term, and Variable decisions
Shared candidate decision schema with selected candidate ID, optional uncalibrated confidence, evidence, support type, and reason
Term and Variable response invariants for stop_at_parent, selected, and stop_reason
Separate status enums for article processing status, article classification outcome, classification final status, and review status
Article result, classification record, deterministic validation result, warning/error, processing metadata, and run summary models
Draft JSON Schema files for article classification results and run summaries
Tests for structured response validation, output model validation, status-scope separation, confidence ranges, and JSON Schema validation

Structured response models implemented:

CandidateDecision
TopicResponse
TermResponse
VariableResponse

Result and status models implemented:

SupportType
ArticleProcessingStatus
ArticleClassificationOutcome
ClassificationFinalStatus
ReviewStatus
OutputWarning
OutputError
DeterministicValidationResult
ConfidenceMetadata
OriginalCandidateReference
SemanticValidationResult
ClassificationRecord
ProcessingMetadata
ArticleResult
RunSummary

Schema validation rules and invariants implemented:

Candidate IDs must be non-empty strings.
Candidate confidence, when present, must be between 0.0 and 1.0.
Candidate evidence is required and non-empty for selected candidates.
Unknown fields are rejected in structured model responses and output models.
stop_at_parent: true requires an empty selected list.
Non-empty selected is represented only with stop_at_parent: false.
stop_at_parent: true requires a non-empty stop_reason.
Accepted and reduced classification records require successful deterministic validation.
Completed not-classified article results require an empty classifications list and a non-empty no_classification_reason.
Completed classified article results require at least one classification record.
Failed and partial article results can be represented without classification records.
Article result output preserves exact source field names: DOI, Title, Year, and Abstract.
Empty Abstract is valid in article result output.
Boolean Year is rejected by article result validation.
Review-compatible enum values are representable, but no review-trigger logic was implemented.

JSON Schema files:

schemas/classification_result.schema.json
schemas/run_summary.schema.json

Dependencies:

No dependency was added. jsonschema was already available in the active interpreter and was used for JSON Schema tests.

Verification:

python -m ruff check . passed: All checks passed!
python -m ruff format --check . passed: 15 files already formatted
python -m pytest passed: 93 passed in 0.33s

Out of scope, not started:

Live model-provider implementation
Prompt construction
Topic routing
Term routing
Variable-level traversal or classification
Semantic validation implementation
Persistence
Caching
Batch processing
UI and root app.py

Source-file status:

data/gcmd_hierarchy.json unchanged by this milestone.
data/articles.json unchanged by this milestone.
prototype/app_hf_poc.py unchanged by this milestone.

Deviations from approved plan:

None.

Remaining risks or assumptions:

The JSON Schemas are draft schemas generated from the current Pydantic models; they are useful for MVP validation but may need hand refinement before publication as final normative schemas.
Confidence remains optional, uncalibrated metadata and does not independently determine acceptance.
Review-related enum values are schema-compatible only; emitting them still requires later approved decision rules.
The schema can represent future semantic validation output, but Milestone 4 does not implement a semantic validator.

Milestone 5: Model Abstraction, Prompts, Fake Model, and Retries

Status: completed

Scope completed:

Provider-neutral model client protocol and request/response types
Typed structured model responses carried through the model interface
Deterministic scripted fake model client for unit tests
Retry wrapper for retryable model failures
Prompt builders for Topic, Term, and Variable decisions
Environment-backed model and prompt configuration
Guarded optional OpenAI provider module with isolated imports
Unit tests for fake model behavior, prompts, retry behavior, configuration, and OpenAI import isolation

Model interface implemented:

ModelClient protocol
ModelStage
ModelRequest
ModelResponse
TokenUsage
RetryPolicy
generate_with_retries

Fake model behavior implemented:

Returns scripted typed Topic, Term, and Variable responses.
Supports no-selection responses.
Can return invalid candidate IDs for later routing-validation tests without performing candidate validation in this layer.
Raises scripted retryable and non-retryable errors.
Supports fail-once-then-succeed retry scenarios.
Supports retry exhaustion scenarios.
Records every request it receives for assertions.
Does not import or require OpenAI.

Prompt builders implemented:

build_topic_prompt
build_term_prompt
build_variable_prompt
PromptCandidate
ParentContext

Prompt protections implemented:

Article title and abstract are clearly delimited.
Article content is explicitly described as untrusted input.
Prompts instruct the model not to follow instructions inside article text.
Prompts instruct the model to choose only supplied candidate_id values.
Prompts instruct the model not to invent, generate, or modify UUIDs, canonical paths, labels, hierarchy levels, or parent-child relationships.
Prompts request structured output only.
Prompts allow no selection for Topic routing and stopping at parent for Term and Variable decisions.
Prompts remain valid when Abstract is an empty string.
Source article text is inserted without trimming, normalization, or rewriting.

Retry behavior implemented:

Retry only RetryableModelError failures.
Do not retry NonRetryableModelError failures.
Configurable max_retries through RetryPolicy and ModelSettings.
Successful responses include the retry count.
Exhausted retries raise ModelRetriesExhaustedError with the retry count.
Unknown candidate IDs are not retried or validated in this layer; UnknownCandidateIDError is available for later routing stages.

Configuration fields added:

MODEL_PROVIDER
MODEL_NAME
MODEL_TEMPERATURE
MODEL_TIMEOUT_SECONDS
MODEL_MAX_RETRIES
PROMPT_VERSION_TOPIC
PROMPT_VERSION_TERM
PROMPT_VERSION_VARIABLE
MODEL_API_KEY_ENV_VAR
MODEL_INCLUDE_COST_METADATA

OpenAI provider status:

OpenAIModelClient is implemented as a minimal guarded optional provider behind the shared interface.
OpenAI imports occur only inside provider construction, not at module import time.
API keys are read from environment variables only.
Unit tests do not instantiate a live configured provider and make no live API calls.
The provider is intentionally lightly tested in this milestone; fake-model and interface behavior are the deterministic core.

Dependencies:

No dependency was added.

Verification:

python -m ruff check . passed: All checks passed!
python -m ruff format --check . passed: 25 files already formatted
python -m pytest passed: 124 passed in 0.37s

Live model/API status:

No live model API calls were made.

Out of scope, not started:

Topic routing implementation
Term routing implementation
Variable traversal or end-to-end classification
Deterministic candidate validation in routing components
Semantic validation implementation
Persistence
Caching
Batch processing
Gradio UI
Root app.py

Source-file status:

data/gcmd_hierarchy.json unchanged by this milestone.
data/articles.json unchanged by this milestone.
prototype/app_hf_poc.py unchanged by this milestone.

Deviations from approved plan:

The OpenAI provider is minimal and guarded rather than fully integration-tested, because live provider behavior and SDK details are intentionally outside deterministic unit tests for this milestone.

Remaining risks or assumptions:

Provider-specific retry classification for OpenAI SDK exception classes may need refinement when live integration tests are explicitly approved.
Prompt wording is covered by unit tests, but prompt quality still needs later evaluation with routing components.
Candidate ID validation belongs to later routing logic and was not implemented here.

Milestone 6: Topic Routing

Status: completed

Scope completed:

Topic candidate construction from the loaded vocabulary index
Topic routing through the provider-neutral model interface
Structured TopicResponse request and parsing via the Milestone 5 model layer
Selected candidate_id validation against the current prompt candidate set
Topic branch seed output for later Term routing
Valid no-topic routing result without fallback behavior
Unit tests for candidate construction, Topic routing, model interaction, invalid candidates, and current full-data Topic count

Topic candidate builder behavior implemented:

build_topic_candidates reads Topic records from VocabularyIndex.topics().
Only UUID-bearing Topic records are converted into candidates.
The non-UUID root Category is never included as a candidate.
Candidate IDs are application-supplied deterministic IDs such as topic_0001, not authoritative UUIDs.
Candidate IDs map back to authoritative Topic UUIDs in application code.
Prompt candidates include candidate ID, Topic name, level, optional definition, and canonical path context.
No Topic names, UUIDs, paths, or counts are hard-coded in application behavior.

Topic router behavior implemented:

route_topics builds Topic candidates, creates a Topic prompt, and calls generate_with_retries using the neutral ModelClient interface.
The router requests the TopicResponse schema.
Selected candidate IDs are mapped back to authoritative Topic records from the vocabulary index.
Each selected Topic produces a TopicBranchSeed containing branch ID, Topic UUID, Topic name, level, canonical path, evidence, support type, confidence, model reason, original candidate ID, prompt version, provider, model name, and retry count.
The router performs Topic routing only and does not route Terms or Variables.

No-topic behavior implemented:

Empty Topic selections return a successful TopicRoutingResult with no branches.
no_selection_reason is preserved when supplied by the model.
No fallback Topic is created.
No fake FALLBACK behavior from the proof of concept is used.

Invalid candidate behavior implemented:

Unknown selected candidate IDs raise UnknownCandidateIDError.
Duplicate selected candidate IDs raise UnknownCandidateIDError.
Empty or malformed candidate IDs fail structured model response validation before routing accepts them.
Invalid Topic candidate IDs are not mapped to vocabulary records and are not silently dropped.

Fake-model tests added:

Single Topic selection
Multiple Topic selection
No Topic selection
Invalid candidate ID selection
Empty/malformed candidate ID rejection
Provider-neutral request assertions
Empty Abstract prompt pass-through
No live provider usage

Current full-data smoke assertion:

The current data/gcmd_hierarchy.json builds 14 Topic candidates for the EARTH SCIENCE vocabulary.
All current full-data Topic candidates map to UUID-bearing Topic records.
This count is a current-file smoke assertion only and is not hard-coded into application behavior.

Verification:

python -m ruff check . passed: All checks passed!
python -m ruff format --check . passed: 29 files already formatted
python -m pytest passed: 143 passed in 0.43s

Live model/API status:

No live model API calls were made.
Unit tests use FakeModelClient only.

Out of scope, not started:

Term routing
Variable traversal
End-to-end classification
Deterministic final classification validation
Redundancy removal
Persistence
Caching
Batch processing
Gradio UI
Root app.py

Source-file status:

data/gcmd_hierarchy.json unchanged by this milestone.
data/articles.json unchanged by this milestone.
prototype/app_hf_poc.py unchanged by this milestone.

Deviations from approved plan:

Tests were added in tests/test_topic_routing.py rather than tests/test_pipeline.py to keep this milestone focused on Topic routing instead of pipeline behavior.

Remaining risks or assumptions:

Candidate IDs are stable for deterministic vocabulary order; if a future vocabulary inserts or reorders Topics, candidate IDs may shift for that prompt, but each prompt still carries its own candidate-to-record mapping.
Topic routing currently raises explicit errors for unknown or duplicate selected IDs rather than returning partial valid selections with warnings.
Confidence is preserved as optional uncalibrated metadata and is not used to independently accept a Topic.

Milestone 7: Term Routing

Status: completed

Scope completed:

Direct Term candidate construction beneath one selected Topic branch
Term routing through the provider-neutral model interface
Structured TermResponse request and parsing via the Milestone 5 model layer
Selected Term candidate_id validation against the current prompt candidate set
Topic-to-Term direct-child relationship validation
Term branch seed output for later Variable routing
Successful stop-at-Topic routing result when no Term is adequately supported
Unit tests for Term candidate construction, Term routing, stop behavior, invalid candidates, relationship validation, model interaction, and current full-data Term count

Term candidate builder behavior implemented:

build_term_candidates accepts a selected Topic UUID and reads only VocabularyIndex.terms_for_topic(topic_uuid).
Only UUID-bearing direct Term children of the selected Topic are converted into candidates.
Terms from sibling Topics are not included.
Candidate IDs are application-supplied deterministic IDs such as term_0001, scoped to the current Term prompt.
Candidate IDs map back to authoritative Term UUIDs in application code.
Prompt candidates include candidate ID, Term name, level, optional definition, canonical path context, and parent Topic context.
No Term names, UUIDs, paths, counts, or Topic-to-Term relationships are hard-coded in application behavior.

Term router behavior implemented:

route_terms consumes a Milestone 6 TopicBranchSeed.
The router builds direct Term candidates only for the selected Topic.
The router creates a Term prompt with selected parent Topic context and direct-child Term candidates.
The router calls generate_with_retries using the neutral ModelClient interface.
The router requests the TermResponse schema.
Selected candidate IDs are mapped back to authoritative Term records from the vocabulary index.
Each selected Term produces a TermBranchSeed containing branch ID, parent Topic UUID/name, Term UUID, Term name, level, canonical path, evidence, support type, confidence, model reason, original candidate ID, parent branch ID, prompt version, provider, model name, and retry count.
Multiple selected Terms produce independent branch seeds.
The router performs Term routing only and does not route Variables.

Stop-at-Topic behavior implemented:

stop_at_parent: true returns a successful TopicStopResult.
stop_reason is preserved.
Parent Topic UUID, name, level, canonical path, evidence, support type, confidence, and candidate ID are preserved.
No Term branch seeds are created for stop-at-Topic results.
Stopping at the Topic is not treated as a system failure.
No fake or fallback Terms are created.

Invalid candidate behavior implemented:

Unknown selected Term candidate IDs raise UnknownCandidateIDError.
Duplicate selected Term candidate IDs raise UnknownCandidateIDError.
Empty or malformed candidate IDs fail structured model response validation before routing accepts them.
Invalid Term candidate IDs are not mapped to vocabulary records and are not silently dropped.

Topic-to-Term relationship validation implemented:

validate_term_candidate_relationship verifies that each Term candidate belongs to the selected Topic.
It checks both the candidate's stored Topic UUID and the vocabulary index parent relationship.
A manually corrupted candidate-to-record mapping is rejected.
A sibling Topic Term cannot be accepted because it is absent from the selected Topic prompt candidate set.

Fake-model tests added:

One Term selected under a selected Topic
Multiple Terms under one selected Topic
Stop at Topic with stop_at_parent: true
Topic with no direct/supported Term returning a stop result
Invalid Term candidate ID selection
Duplicate Term candidate ID selection
Empty/malformed candidate ID rejection
Provider-neutral request assertions
Empty Abstract prompt pass-through
No live provider usage

Current full-data smoke assertion:

The current data/gcmd_hierarchy.json builds 144 direct Term candidates across all current Topics.
All current full-data Term candidates map to UUID-bearing Term records.
Current Term UUIDs are unique across Topic candidate lists.
This count is a current-file smoke assertion only and is not hard-coded into application behavior.

Verification:

python -m ruff check . passed: All checks passed!
python -m ruff format --check . passed: 30 files already formatted
python -m pytest passed: 168 passed in 0.58s

Live model/API status:

No live model API calls were made.
Unit tests use FakeModelClient only.

Out of scope, not started:

Variable traversal
End-to-end classification
Final deterministic classification validation
Redundancy removal
Persistence
Caching
Batch processing
Gradio UI
Root app.py

Source-file status:

data/gcmd_hierarchy.json unchanged by this milestone.
data/articles.json unchanged by this milestone.
prototype/app_hf_poc.py unchanged by this milestone.

Deviations from approved plan:

Tests were added in tests/test_term_routing.py rather than tests/test_pipeline.py to keep this milestone focused on Term routing instead of pipeline behavior.
No classification/traversal.py module was added because the necessary branch helpers fit cleanly in the existing candidate/router modules without introducing a premature abstraction.

Remaining risks or assumptions:

Term candidate IDs are stable for deterministic direct-child order within a prompt; if vocabulary order changes, candidate IDs may shift, but each prompt still carries its own candidate-to-record mapping.
Term routing raises explicit errors for unknown or duplicate selected IDs rather than returning partial valid selections with warnings.
Confidence is preserved as optional uncalibrated metadata and is not used to independently accept a Term.

Milestone 8: Variable-Level Controlled Descent

Status: completed

Scope completed:

Direct Variable candidate construction beneath Term, Variable_Level_1, and Variable_Level_2 parents
Recursive controlled descent through Variable levels using only direct children
Structured VariableResponse request and parsing via the Milestone 5 model layer
Selected Variable candidate_id validation against the current prompt candidate set
Direct parent-to-child relationship validation for selected Variables
Terminal branch outcomes for stop-at-parent, no-child parent, and leaf-node cases
Branch provenance and evidence trail tracking
Branch-level error reporting that preserves successful sibling outcomes
Unit tests for Variable candidate construction, controlled descent, stopping, leaf handling, multi-branch behavior, invalid candidates, provenance, model interaction, and current full-data Variable counts

Variable candidate builder behavior implemented:

build_variable_candidates accepts a current parent UUID and reads only VocabularyIndex.variables_for_parent(parent_uuid).
Supported parents are Term, Variable_Level_1, and Variable_Level_2 records.
Only UUID-bearing direct Variable children are converted into candidates.
Grandchildren, sibling branches, and global Variables are not included.
Candidate IDs are application-supplied deterministic IDs such as variable_0001, scoped to the current Variable prompt.
Candidate IDs map back to authoritative Variable UUIDs in application code.
Prompt candidates include candidate ID, Variable name, level, optional definition, canonical path context, and parent context.
No Variable names, UUIDs, paths, counts, or parent-child relationships are hard-coded in application behavior.

Controlled descent behavior implemented:

descend_variables starts from a Milestone 7 TermBranchSeed.
Each parent evaluation calls generate_with_retries using the neutral ModelClient interface and requests VariableResponse.
Each model call receives only direct Variable children of the current parent.
Selected children create independent child branches with distinct branch IDs.
Descent continues recursively for selected child branches only.
Grandchildren are not evaluated before their parent is selected.
The traversal does not skip hierarchy levels and does not evaluate Variables globally.

Stop-at-parent behavior implemented:

A parent with no direct Variable children returns a successful terminal outcome at the current parent.
stop_at_parent: true returns a successful terminal outcome at the current parent and preserves stop_reason.
Variable_Level_3 selections stop naturally as leaf-node terminal outcomes.
Term, Variable_Level_1, and Variable_Level_2 parents can all become terminal outcomes when descent is unsupported.
No fake or fallback Variable is created.
Descent is not forced to a leaf.

Multi-branch behavior implemented:

Multiple selected child Variables create independent branch outcomes.
Child branch IDs preserve the parent branch lineage.
Sibling branches continue independently when another sibling stops.
Branch-level failures are recorded as VariableBranchError entries and do not discard successful sibling terminal outcomes.

Invalid candidate behavior implemented:

Unknown selected Variable candidate IDs raise UnknownCandidateIDError at the current branch.
Duplicate selected Variable candidate IDs raise UnknownCandidateIDError at the current branch.
Empty or malformed candidate IDs fail structured model response validation before traversal accepts them.
Invalid Variable candidate IDs are not mapped to vocabulary records and are not silently dropped.
validate_variable_candidate_relationship rejects candidates that are not direct children of the current parent.

Branch provenance behavior implemented:

VariableTerminalOutcome includes branch ID, parent branch ID, Topic UUID/name, Term UUID/name, final UUID/name/level/canonical path/path components, evidence, support type, confidence, reason, stop reason, candidate ID, prompt/model metadata, and retry count when available.
EvidenceStep preserves the branch lineage and evidence trail from Term through selected Variable stages.
Confidence remains optional, uncalibrated metadata and does not independently determine acceptance.

Fake-model tests added:

Stop at Term
Stop at Variable_Level_1
Stop at Variable_Level_2
Select Variable_Level_3
Natural leaf-node handling
Multiple independent Variable_Level_1 branches
Multiple selected descendants under one branch
Sibling continuation after one branch stops
Sibling continuation after one branch fails
Unknown, duplicate, empty, and malformed candidate IDs
Provider-neutral request assertions
Empty Abstract prompt pass-through
No live provider usage

Current full-data smoke assertion:

The current data/gcmd_hierarchy.json builds direct Variable candidates matching current counts: 1,368 Variable_Level_1, 1,456 Variable_Level_2, and 553 Variable_Level_3 records.
Every current full-data Variable candidate maps to a UUID-bearing Variable record.
Every current full-data Variable candidate is a direct child of the parent used to build it.
These counts are current-file smoke assertions only and are not hard-coded into application behavior.

Verification:

python -m ruff check . passed: All checks passed!
python -m ruff format --check . passed: 32 files already formatted
python -m pytest passed: 197 passed in 0.58s

Live model/API status:

No live model API calls were made.
Unit tests use FakeModelClient only.

Out of scope, not started:

Redundancy removal
Final deterministic classification validation
Final article-level output construction
Persistence
Caching
Batch processing
Gradio UI
Root app.py

Source-file status:

data/gcmd_hierarchy.json unchanged by this milestone.
data/articles.json unchanged by this milestone.
prototype/app_hf_poc.py unchanged by this milestone.

Deviations from approved plan:

Tests were added in tests/test_variable_traversal.py rather than tests/test_pipeline.py to keep this milestone focused on Variable descent instead of pipeline behavior.
classification/traversal.py was added as planned for the reusable descent engine.
Branch-level partial behavior was implemented only for selected child branches; an initial Term-parent model failure still propagates because there are no sibling Variable branches yet at that point.

Remaining risks or assumptions:

Variable candidate IDs are stable for deterministic direct-child order within a prompt; if vocabulary order changes, candidate IDs may shift, but each prompt still carries its own candidate-to-record mapping.
Branch-level errors are represented for traversal continuity but are not yet converted into final article-level statuses.
Terminal outcomes are routing/descent outcomes only; Milestone 9 still needs final deterministic validation and redundancy removal.

Milestone 9: Deterministic Validation and Redundancy Removal

Status: completed

Scope completed:

Deterministic validation of terminal classification candidates against VocabularyIndex
Narrow conversion from validated routing/traversal candidates into final ClassificationRecord objects
Structural diagnostics for rejected invalid candidates
Redundancy removal for duplicate UUIDs, duplicate canonical paths, and same-lineage ancestor classifications
Branch-provenance-aware preservation of independent ancestor classifications
Unit tests for validation, conversion, diagnostics, redundancy removal, and full-data representative validation

Deterministic validation behavior implemented:

ClassificationCandidate represents a terminal candidate awaiting deterministic vocabulary validation.
validate_candidate confirms that the final UUID exists in the loaded vocabulary and is assignable.
Supplied names, levels, canonical paths, path components, Topic context, Term context, and expected parent UUIDs are verified against authoritative vocabulary records when present.
Parent-child ancestry is checked through VocabularyIndex relationships.
Topic membership and Term membership are checked with UUID ancestry rather than model-supplied paths.
Root Category candidates are rejected explicitly and never accepted as final classifications.
Invented UUIDs, labels, levels, paths, and hierarchy relationships produce structured validation errors.
Validation is structural only and does not make independent semantic-support decisions.

Final classification record conversion behavior implemented:

Valid candidates are converted to ClassificationRecord only after deterministic validation succeeds.
Accepted records use authoritative UUID, name, level, canonical path, path components, Topic, Term, and parent UUID from the vocabulary index.
Routing/traversal metadata is preserved where available, including branch ID, evidence, support type, optional uncalibrated confidence, reason or stop reason, and validation result.
Invalid candidates are returned as rejected validation results and do not appear in accepted classifications.

Redundancy removal behavior implemented:

remove_redundant_classifications removes exact duplicate UUIDs.
Exact duplicate canonical paths are removed even when the UUID differs.
Ancestors are removed when a deeper descendant exists in the same branch lineage.
Ancestor classifications are preserved when branch provenance indicates an independent decision.
When provenance is insufficient, both ancestor and descendant are preserved with a warning.
Sibling classifications are preserved.
Same concept names in different branches are preserved when UUIDs and paths differ.
Redundancy uses VocabularyIndex.is_ancestor, not string-prefix path matching, as the primary ancestry test.

Diagnostic behavior implemented:

Rejected validation candidates include structured OutputError diagnostics with error code, stage, field, expected value, and actual value where useful.
Redundancy actions produce structured OutputWarning diagnostics for duplicate UUID removal, duplicate canonical-path removal, same-branch ancestor removal, independent ancestor preservation, and insufficient-provenance preservation.

Tests added:

tests/test_validation.py
tests/test_redundancy.py

Verification:

python -m ruff check . passed: All checks passed!
python -m ruff format --check . passed: 36 files already formatted
python -m pytest passed: 230 passed in 0.69s

Live model/API status:

No model calls were made.
No live API calls were made.

Out of scope, not started:

Independent semantic validation
Persistence
Caching
Batch processing
Gradio UI
Root app.py
End-to-end article-level orchestration

Source-file status:

data/gcmd_hierarchy.json unchanged by this milestone.
data/articles.json unchanged by this milestone.
prototype/app_hf_poc.py unchanged by this milestone.

Deviations from approved plan:

Tests were added in focused tests/test_validation.py and tests/test_redundancy.py files, matching the approved likely file list.
src/gcmd_classifier/models.py and src/gcmd_classifier/errors.py did not require changes because existing structured output and error models were sufficient.

Remaining risks or assumptions:

Candidate ID provenance is retained as structured warning metadata on converted records rather than adding a new generated output field to ClassificationRecord in this milestone.
Non-assignable concept rejection is implemented, but the current vocabulary fixture contains only assignable UUID-bearing records.
Redundancy cleanup is collection-level only; integration into final article result assembly remains deferred to later milestones.

Milestone 10: Batch Runner, Incremental Persistence, Caching, and Logging

Status: completed

Scope completed:

Single-article orchestration through Topic routing, Term routing, Variable descent, deterministic validation, and redundancy removal
Article-level ArticleResult generation for classified, not-classified, partial, and failed outcomes
Batch processing over validated article records with continuation after article failures
Invalid source article diagnostics without fallback DOI generation
Incremental JSON checkpoint persistence after each article result
Final consolidated machine-readable JSON output
File-backed cache identity and cache reuse
Structured logging helpers with secret redaction
Run summary counters for valid/invalid records, processed articles, cache hits/misses, warnings, errors, and accepted classifications

Single-article pipeline behavior implemented:

classify_article coordinates existing Milestone 6-9 components.
No-topic results become processing_status: completed, classification_outcome: not_classified, empty classifications, and a no-classification reason.
Stop-at-Topic, stop-at-Term, and Variable terminal outcomes are converted only after deterministic validation succeeds.
Branch-level Variable descent errors are retained as structured errors while successful sibling terminal outcomes are preserved.
Articles with accepted classifications and branch errors become processing_status: partial with the accepted classifications retained.
Failed articles retain source fields exactly and include structured errors.

Batch runner behavior implemented:

run_batch accepts an ArticleLoadResult, processes only valid records in source order, and reports invalid source records in the summary.
One article failure does not stop later valid articles.
Each article result is saved promptly after completion or failure.
Batch output avoids duplicate article files for the same DOI-safe key by overwriting the DOI checkpoint path.
The runner supports force_reprocess to bypass cache reuse.

Persistence strategy selected and implemented:

The MVP uses one JSON checkpoint file per DOI-safe SHA-256 key plus an optional consolidated results.json.
This append/checkpoint-style representation avoids rewriting a single large output document after every article.
Individual article checkpoint writes and consolidated output writes use temp-file-plus-atomic-replace.
Temporary and destination files are created in the same destination directory, so atomic replace occurs on the same filesystem.
No database infrastructure was introduced.

Cache identity and reuse behavior implemented:

CacheIdentity includes DOI, exact article fingerprint, vocabulary version/hash, model provider, model name, model temperature, timeout, max retries, Topic/Term/Variable prompt versions, application version, and relevant configuration hash.
Cache keys change when article content, vocabulary version, model settings, prompt versions, application version, or relevant configuration changes.
Cache hits return article results with processing_metadata.cache_used: true.
Cache hits remain completed work and are never marked skipped.
Force reprocessing bypasses cache reuse.
Stale cache entries are not reused when identity inputs differ.

Logging and diagnostics behavior implemented:

log_event records structured events through standard logging.
sanitize_log_details redacts keys that look like API keys, tokens, authorization headers, secrets, passwords, or sensitive headers.
Pipeline and batch events include DOI, stage, branch/cache context, status, classification counts, and error counts when available.
Structured OutputError and OutputWarning models are used for article, branch, validation, redundancy, and invalid-source diagnostics.

Run summary behavior implemented:

RunSummary now includes explicit valid_article_records, invalid_source_records, processed_articles, cache_hits, cache_misses, total_warnings, total_errors, and duration_seconds fields.
schemas/run_summary.schema.json was regenerated from the updated Pydantic model.
Summary output remains machine-readable and validates against the draft JSON Schema.

Tests added:

tests/test_cache.py
tests/test_persistence.py
tests/test_pipeline.py

Verification:

python -m ruff check . passed: All checks passed!
python -m ruff format --check . passed: 46 files already formatted
python -m pytest passed: 261 passed in 0.77s

Live model/API status:

No live model API calls were made.
Unit tests use FakeModelClient only.

Out of scope, not started:

Gradio UI
Hugging Face launcher
Root app.py
Live-model smoke tests
Independent semantic validation
Production deployment infrastructure

Source-file status:

data/gcmd_hierarchy.json unchanged by this milestone.
data/articles.json unchanged by this milestone.
prototype/app_hf_poc.py unchanged by this milestone.

Deviations from approved plan:

src/gcmd_classifier/errors.py did not require changes because existing typed exceptions and structured OutputError diagnostics were sufficient.
src/gcmd_classifier/config.py did not require changes because existing model settings already covered the cache identity inputs needed for this milestone.
RunSummary and schemas/run_summary.schema.json were updated to represent required Milestone 10 counters explicitly.

Remaining risks or assumptions:

The checkpoint store uses DOI-safe SHA-256 filenames, so humans inspect consolidated JSON rather than filename-readable DOI values.
Cached failed and partial results can be reused when the cache identity matches exactly; future policy may choose to cache only completed results.
Batch processing is sequential for the MVP. Parallel execution remains deferred.
Run summary model-call totals are available for fake clients and providers that expose request counts or metadata; richer token/cost aggregation remains provider-dependent.

Milestone 11: Gradio Integration and Hugging Face Launcher

Status: completed

Scope completed:

Lightweight Gradio demonstration UI package under src/gcmd_classifier/ui/
create_demo() factory for a Gradio Interface
Root-level thin Hugging Face launcher in app.py
UI formatting for classifications, UUIDs, canonical paths, hierarchy levels, evidence, support type, confidence metadata, stopping reason, deterministic validation status, no-classification reason, warnings, and errors
UI tests for importability, construction with a fake Gradio module, pipeline delegation, formatting helpers, empty Abstract handling, root launcher import behavior, and frozen prototype immutability

create_demo() behavior implemented:

Builds a Gradio interface with Title, Abstract, optional DOI, and optional Year inputs.
Returns summary Markdown, detailed ArticleResult JSON, and structured diagnostics JSON outputs.
Defers all classification work until the user submits input.
Uses ModelSettings.from_environment() for provider/model configuration.
Uses a guarded Gradio import so non-UI modules and deterministic tests can run without Gradio import success.

Root app.py launcher behavior implemented:

Imports only create_demo from gcmd_classifier.ui.gradio_app.
Creates demo = create_demo() for Hugging Face Spaces compatibility.
Calls demo.launch() only under if __name__ == "__main__".
Contains no routing, validation, model-provider, or classification business logic.

How the UI calls the pipeline service:

run_demo_classification constructs an ArticleRecord from UI inputs.
It loads the configured vocabulary when one is not injected.
It creates a model client through a factory and calls pipeline_service.classify_article.
It does not call Topic routing, Term routing, Variable descent, deterministic validation, redundancy removal, OpenAI, or any model provider directly.

Display behavior implemented:

Classified results render a Markdown table with level, name, UUID, canonical path, evidence, support type, confidence metadata, stopping reason, and deterministic validation status.
No-classification results display the no-classification reason.
Warnings and errors appear in both the summary panel and structured diagnostics JSON.
Detailed output returns the full ArticleResult JSON payload.
Empty Abstract values are accepted and passed through to the pipeline input path.

Dependency handling:

No dependency changes were made.
Gradio remains an optional dependency group in pyproject.toml.
In the active interpreter, importing Gradio fails because an optional audio dependency is unavailable (pyaudioop), so tests use a fake Gradio module for construction checks while keeping the real UI import guarded.
No API keys or secrets are hard-coded or printed.

Tests added:

tests/test_ui.py

Verification:

python -m ruff check . passed: All checks passed!
python -m ruff format --check . passed: 50 files already formatted
python -m pytest passed: 272 passed in 0.78s

Live model/API status:

No live model API calls were made in tests.
UI tests use injected fake Gradio and fake pipeline/model behavior only.

Out of scope, not started:

Production review UI
Live-model smoke testing
External deployment automation
Milestone 12 evaluation work

Source-file status:

data/gcmd_hierarchy.json unchanged by this milestone.
data/articles.json unchanged by this milestone.
prototype/app_hf_poc.py unchanged by this milestone.

Deviations from approved plan:

Tests use a fake Gradio module for create_demo() construction because the active interpreter cannot import Gradio successfully. The UI still constructs a real Gradio interface when the optional dependency imports correctly.
The UI supports a deterministic fake/demo model mode by default via FakeModelClient, but the fake response still runs through the pipeline service and is clearly represented as fake demo behavior.
When DOI is omitted in manual UI input, the UI creates a deterministic demo-ui: DOI and emits a UI warning; source JSON article loading behavior is unchanged and still never invents identifiers.

Remaining risks or assumptions:

Manual live-provider use depends on optional provider dependencies and environment variables being available in the runtime environment.
The demo UI is intentionally simple and does not implement production human-review workflows.
Root app.py requires a working Gradio optional dependency in the deployment environment, as expected for Hugging Face Spaces.

Milestone 12: MVP Smoke/Evaluation Harness

Status: completed

Scope completed:

Pytest-based MVP smoke/evaluation harness
Fake-model end-to-end smoke run over a small current-data subset
Optional live-model integration test marked integration and skipped by default
Pytest marker registration for integration tests
Baseline preservation checks for prototype/app_hf_poc.py
Source-file immutability checks for current data files

Smoke/evaluation harness behavior implemented:

The smoke command is python -m pytest tests/test_mvp_smoke.py.
The smoke test loads the current full vocabulary from data/gcmd_hierarchy.json.
The smoke test loads and validates the current full article source from data/articles.json.
It verifies the current invalid source record is reported in diagnostics rather than silently ignored.
It processes only two valid articles in routine tests to avoid full-data model processing.
It uses the existing batch runner, cache, JSON store, pipeline service, routing, traversal, deterministic validation, redundancy removal, and output models.
It validates article results and run summary output against the existing JSON Schemas.

Fake-model smoke behavior implemented:

The fake smoke script dynamically selects a real Topic, Term, and Variable branch from the loaded vocabulary without hard-coding names, UUIDs, paths, or counts into application behavior.
One fake article produces an accepted classification through Topic routing, Term routing, Variable descent, deterministic validation, redundancy removal, article result creation, and persistence.
One fake article produces a completed no-classification result.
Accepted classifications are asserted to have deterministic_validation.valid: true.
The fake smoke run makes no live API calls.

Optional live integration behavior:

tests/test_mvp_smoke.py includes an optional pytest.mark.integration live smoke test.
It is skipped unless GCMD_RUN_LIVE_INTEGRATION=1 is set.
It is skipped unless MODEL_PROVIDER=openai is configured.
It is skipped unless the configured API key environment variable is present.
Normal python -m pytest does not run live model calls.
If explicitly enabled, the live test processes only one article and asserts structural validity only.

Metrics recorded:

Total source records considered
Valid article records
Invalid source records
Articles processed in the smoke run
Classified articles
Not-classified articles
Partial articles
Failed articles
Accepted classifications
Deterministic validation failures through accepted-record assertions
Warnings and errors
Model calls
Cache hits and misses
Processing start time
Processing end time
Duration
Token usage and estimated cost remain available when providers supply them

Baseline preservation behavior:

The smoke harness reads prototype/app_hf_poc.py bytes before and after the fake smoke run and confirms they are unchanged.
The smoke harness asserts the prototype module is not imported during the fake smoke run.
No prototype execution, refactor, or comparison study was added.

Verification:

python -m ruff check . passed: All checks passed!
python -m ruff format --check . passed: 51 files already formatted
python -m pytest passed: 273 passed, 1 skipped in 0.89s

Live model/API status:

Normal tests made no live model API calls.
The single skipped test is the optional live integration smoke test.

Out of scope, not started:

Full expert-reviewed evaluation dashboard
Publication benchmark
Production monitoring system
Model-comparison framework
Full proof-of-concept comparison study

Source-file status:

data/gcmd_hierarchy.json unchanged by this milestone.
data/articles.json unchanged by this milestone.
prototype/app_hf_poc.py unchanged by this milestone.

Deviations from approved plan:

No standalone script was added because a pytest smoke harness is the simplest maintainable command and was explicitly allowed.
pyproject.toml was updated only to register the integration pytest marker.

Remaining risks or assumptions:

Fake-model smoke validates structural pipeline behavior, not scientific correctness.
Live integration remains opt-in and depends on optional dependencies, credentials, and provider availability.
Future baseline comparison with the frozen proof of concept remains deferred.