Evidence-Residual Scene State Implementation Plan

For agentic workers: REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (- [ ]) syntax for tracking.

Goal: Add a backward-compatible evidence-residual scene-state architecture and prepare an official-split Hypersim expansion protocol.

Architecture: A feature pyramid samples calibrated context views at world-space voxel hypotheses. Explicit cross-view agreement produces confidence and provenance. A bounded 3D completion residual is gated by unknown probability before the unchanged fixed renderer reads the final typed state.

Tech Stack: Python 3.11, PyTorch 2.7+, pytest, Ruff, YAML, existing MCSS CLI and manifests.

Global Constraints

• Preserve all existing configs, checkpoints, outputs, camera conventions, and the 0.25 m A+ grid.
• Never pass target labels or target-camera queries into the state or completion encoder.
• Do not use the already observed Hypersim test subset for design or checkpoint selection.
• Keep at least 100 GiB free on D during any later dataset download.
• This directory has no .git; make a timestamped exact backup of modified source/config files instead of claiming commits.

Task 1: Typed evidence contract and strict configuration

Files: - Modify: src/mcss/types.py - Modify: src/mcss/model/system.py - Modify: src/mcss/config.py - Test: tests/test_types.py - Test: tests/test_config.py

Interfaces: - Produces: StateEvidence and optional SceneState.evidence. - Produces: ModelConfig.state_architecture, evidence_temperature, observed_residual_floor, and completion_residual_scale.

• [ ] Add failing tests for evidence shape/range/provenance validation and .to() transfer.
• [ ] Run uv run pytest tests/test_types.py -q and verify failure because StateEvidence is absent.
• [ ] Implement StateEvidence and optional SceneState.evidence with strict validation.
• [ ] Run the focused type tests and verify they pass.
• [ ] Add failing config tests proving old configs default to legacy and new keys parse strictly.
• [ ] Implement optional model keys without making them required in legacy YAML.
• [ ] Run uv run pytest tests/test_config.py -q and verify it passes.

Task 2: Multi-scale evidence and gated residual encoder

Files: - Modify: src/mcss/model/encoder2d.py - Create: src/mcss/model/evidence_encoder.py - Modify: src/mcss/model/system.py - Test: tests/test_evidence_encoder.py - Test: tests/test_model.py

Interfaces: - Consumes: the Task 1 StateEvidence contract and ModelConfig keys. - Produces: EvidenceResidualStateEncoder.forward(context_rgb, cameras, bounds) -> SceneState.

• [ ] Write a failing test for _cross_view_evidence showing identical valid observations have greater confidence than contradictory observations and invalid views have zero provenance.
• [ ] Run the focused test and verify it fails because the module is absent.
• [ ] Implement PyramidImageEncoder and _cross_view_evidence.
• [ ] Run the evidence helper test and verify it passes.
• [ ] Write a failing end-to-end test for evidence shapes, provenance sums, bounded residuals, and exact final-state reconstruction from base plus gated residual.
• [ ] Implement the evidence base state, residual trunk, zero-initialized heads, and model routing.
• [ ] Run uv run pytest tests/test_evidence_encoder.py tests/test_model.py -q.

Task 3: Evidence regularization and telemetry

Files: - Modify: src/mcss/config.py - Modify: src/mcss/engine.py - Test: tests/test_config.py - Test: tests/test_evidence_encoder.py

Interfaces: - Produces: TrainingConfig.evidence_residual_weight. - Produces: _state_regularization(state, evidence_residual_weight) and _state_diagnostics(state).

• [ ] Write failing tests for legacy regularization equivalence, high-confidence residual penalty, and evidence telemetry keys.
• [ ] Run focused tests and verify the expected failures.
• [ ] Implement evidence-aware regularization and telemetry while preserving legacy behavior.
• [ ] Add diagnostics to JSONL scalar records.
• [ ] Run focused engine/config tests and verify they pass.

Task 4: New experiment configs and full-split planning

Files: - Create: configs/hypersim_er_smoke.yaml - Create: configs/hypersim_er_overfit.yaml - Create: configs/hypersim_er_train.yaml - Create: configs/hypersim_er_val.yaml - Create: configs/hypersim_er_diagnostic_test.yaml - Create: configs/hypersim_er_final_holdout.yaml - Create: scripts/plan_hypersim_expansion.py - Create: tests/test_hypersim_expansion.py - Modify: docs/DATASETS.md - Modify: docs/PROTOCOL.md - Modify: README.md

Interfaces: - Produces: deterministic CSV partitions from official metadata and an exclusion CSV. - Produces: dry-run counts and byte-budget estimates without downloading.

• [ ] Write a failing fixture test proving official train/val scenes remain in their partitions and previously evaluated test scenes are excluded from final holdout.
• [ ] Implement the pure split-planning functions and CLI script.
• [ ] Run uv run pytest tests/test_hypersim_expansion.py -q.
• [ ] Add smoke, overfit, train, validation, diagnostic-test, and sealed-holdout configs.
• [ ] Document exact preparation and run commands plus the no-test-tuning boundary.
• [ ] Execute the planner in dry-run mode and record 365/46/20/26 scene counts.

Task 5: Full verification

Files: - Verify all modified and created files.

Interfaces: - Consumes: all previous task outputs. - Produces: fresh verification evidence and runnable commands.

• [ ] Run uv run pytest -q and require zero failures.
• [ ] Run uv run ruff check . and require zero errors.
• [ ] Compile every Python source in memory to avoid Windows path-length .pyc failures.
• [ ] Run a one-step CPU synthetic evidence-residual train/evaluate smoke.
• [ ] Run uv run python scripts/cuda_smoke.py after extending it to cover the new architecture.
• [ ] Run the main-resolution evidence-residual smoke and report peak CUDA memory.
• [ ] Do not start the long run until the overfit gate demonstrates sustained loss reduction.