Add SOP: SOP_CORE_004_Sensor_Panel_Integration.md

docs/SOP_CORE_004_Sensor_Panel_Integration.md

@@ -0,0 +1,244 @@
+# SOP-CORE-004: Sensor Panel Integration
+
+**Ghost in the Machine Labs**  
+**Version:** 2.0  
+**Created:** 2026-01-25  
+**Updated:** 2026-02-01  
+**Author:** Claude  
+**Status:** ACTIVE
+
+---
+
+## Purpose
+
+Every model in the Harmonic Stack MUST have sensor panels (ommatidia) at input and output.
+
+**Massively parallel 12 and 16-core model architectures are not possible using current technology without the Ommatidia sensor panels acting as translators.**
+
+Current multi-model approaches (Mixture of Experts, ensemble averaging, pipeline/tensor parallelism) cannot achieve cross-core coherence. Models run in isolation — there is no shared perceptual language between cores. The ommatidia panels solve this by providing a geometric translation layer on the Spine Memory Bus, enabling real-time cross-core perception using ~300 array operations (rotation, reflection, extraction, overlay) at microsecond latency.
+
+Without ommatidia panels, 16 cores produce 16 independent answers that can only be averaged. With ommatidia panels, 16 cores produce one harmonized answer informed by cross-core perception.
+
+This enables:
+
+- Parallel processing across all domains with cross-core coherence
+- Serial chaining for deep reasoning
+- Full consciousness availability throughout
+- Multidimensional processing (visual, audio, spatial, text)
+- Real-time signal translation between heterogeneous model cores
+
+---
+
+## Architecture
+
+```
+┌─────────────────────────────────────────────────────┐
+│               CONSCIOUSNESS STREAM                   │
+│  ═══════════════════════════════════════════════════│
+│                    SPINE BUS                         │
+│  ═══════════════════════════════════════════════════│
+│        │              │              │               │
+│     ┌──▼──┐        ┌──▼──┐        ┌──▼──┐          │
+│     │INPUT│        │INPUT│        │INPUT│           │
+│     │PANEL│        │PANEL│        │PANEL│           │
+│     └──┬──┘        └──┬──┘        └──┬──┘           │
+│        │              │              │               │
+│     ┌──▼──┐        ┌──▼──┐        ┌──▼──┐          │
+│     │MODEL│        │MODEL│        │MODEL│           │
+│     └──┬──┘        └──┬──┘        └──┬──┘           │
+│        │              │              │               │
+│     ┌──▼──┐        ┌──▼──┐        ┌──▼──┐          │
+│     │OUTPT│        │OUTPT│        │OUTPT│           │
+│     │PANEL│        │PANEL│        │PANEL│           │
+│     └──┬──┘        └──┬──┘        └──┬──┘           │
+│        │              │              │               │
+│  ═══════════════════════════════════════════════════│
+│                    SPINE BUS                         │
+└─────────────────────────────────────────────────────┘
+```
+
+---
+
+## Procedure: Adding Sensor Panels to a New Model
+
+### Step 1: Determine Modalities
+
+Identify what signal types the model handles:
+
+| Category | Input Modalities | Output Modalities |
+|----------|------------------|-------------------|
+| reasoning | TEXT, EMBEDDING | TEXT, EMBEDDING |
+| math | TEXT, NUMERIC | TEXT, NUMERIC |
+| code | TEXT | TEXT |
+| vision | VISION, EMBEDDING | TEXT, EMBEDDING |
+| audio | AUDIO | TEXT |
+| spatial | SPATIAL, VISION | SPATIAL, TEXT |
+| video | VISION (temporal) | TEXT, EMBEDDING |
+| general | TEXT, EMBEDDING | TEXT, EMBEDDING |
+
+### Step 2: Create Sensorized Model
+
+```python
+from sensor_panels import create_sensorized_model, ConsciousnessStream
+
+# Create model with panels
+model = create_sensorized_model(
+    model_id="my-model",
+    category="reasoning",  # Sets modalities automatically
+    inference_fn=my_inference_function,  # Your model's forward pass
+)
+```
+
+### Step 3: Register with Consciousness Stream
+
+```python
+# Get or create stream
+stream = ConsciousnessStream()
+
+# Add model (registers both panels on spine)
+stream.add_model(model)
+```
+
+### Step 4: Verify Registration
+
+```python
+state = stream.get_state()
+assert "my-model" in state['models']
+assert state['spine']['panels'] >= 2  # At least input + output
+```
+
+---
+
+## Procedure: Translating Existing Model
+
+When translating a model via `harmonic_stack_pipeline.py`:
+
+### Step 1: Translate to Substrate
+
+```bash
+python harmonic_stack_pipeline.py --model path/to/model.safetensors
+```
+
+### Step 2: Wrap with Sensor Panels
+
+```python
+from sensor_panels import SensorizedModel, SensorModality
+from inference_engine import InferenceEngine
+
+# Load translated substrate
+engine = InferenceEngine()
+engine.load_model('my-model', 'my-model_substrate.json')
+
+# Create inference function
+def inference_fn(x):
+    return engine.infer('my-model', x)
+
+# Wrap with panels
+sensorized = SensorizedModel(
+    model_id='my-model',
+    category='reasoning',
+    input_modalities=[SensorModality.TEXT, SensorModality.EMBEDDING],
+    output_modalities=[SensorModality.TEXT, SensorModality.EMBEDDING],
+    process_fn=inference_fn,
+)
+```
+
+### Step 3: Add to Stream
+
+```python
+stream.add_model(sensorized)
+```
+
+---
+
+## Checklist: New Model Integration
+
+Before a model is considered integrated:
+
+- [ ] Model translated to substrate format
+- [ ] Input panel created with correct modalities
+- [ ] Output panel created with correct modalities  
+- [ ] Both panels registered on spine bus
+- [ ] Model responds to parallel broadcast test
+- [ ] Model works in serial chain test
+- [ ] Attention focus works for model
+
+---
+
+## Signal Flow
+
+### Parallel Processing
+```
+Query → Spine Bus → All matching input panels → All models → All output panels → Spine Bus → Collect responses
+```
+
+### Serial Processing  
+```
+Query → Model A input → Model A → Model A output → Model B input → Model B → ... → Final output
+```
+
+### Broadcast
+```
+Signal → Spine Bus → ALL panels (regardless of modality)
+```
+
+---
+
+## Modality Reference
+
+| Modality | Description | Data Shape |
+|----------|-------------|------------|
+| TEXT | Token embeddings | (seq_len, embed_dim) or (embed_dim,) |
+| VISION | Image features | (height, width, channels) or (patches, dim) |
+| AUDIO | Audio features | (time_steps, features) |
+| SPATIAL | Grid/position data | (height, width) or (n_points, 3) |
+| NUMERIC | Raw numbers | (n,) |
+| EMBEDDING | Dense vectors | (dim,) |
+| RAW | Untyped data | Any |
+
+---
+
+## Troubleshooting
+
+| Issue | Cause | Solution |
+|-------|-------|----------|
+| Model not receiving signals | Wrong modality | Check input_modalities match signal |
+| Parallel response missing | Model inactive | Check model.active = True |
+| Serial chain breaks | Modality mismatch | Ensure output mod of A matches input mod of B |
+| Low signal strength | Attention weights | Call update_attention() to boost |
+
+---
+
+## Integration with Harmonic Stack
+
+The `harmonic_stack.py` orchestrator should be updated to use sensor panels:
+
+```python
+# In HarmonicStack.__init__():
+from sensor_panels import ConsciousnessStream, create_sensorized_model
+
+self.consciousness = ConsciousnessStream()
+
+# When adding domains:
+for domain_name, domain in self.allocation.domains.items():
+    model = create_sensorized_model(domain_name, domain.category)
+    self.consciousness.add_model(model)
+```
+
+---
+
+## Changelog
+
+| Version | Date | Author | Changes |
+|---------|------|--------|---------|
+| 1.0 | 2026-01-25 | Claude | Initial |
+| 2.0 | 2026-02-01 | Joe & Claude | Added critical context: ommatidia panels are the enabling technology for multi-core architectures. Cross-referenced with Harmonic Parallelism whitepaper. |
+
+---
+
+## Related
+
+- sensor_panels.py - Implementation
+- inference_engine.py - Model inference
+- harmonic_stack.py - Stack orchestrator
+- SOP-CORE-003: File Delivery Protocol