fix: update traceability endpoint, remove retry logic, and fix report stale version

- Update GET traceability path from /api/v1/observability to /api/v1/traceability
- Remove 3x retry logic on 404: contract guarantees row is written before done event
- Update 404 error message to "Traceability not available for this answer"
- Remove unused wait helper
- Use message_id from API response (savedUi.messageId) instead of doneMessageId
from agentic stream, now that Golang backend returns message_id in POST/GET responses
- Fix ReportSidebar stale selectedVersion: reset versions and selectedVersion
synchronously on analysis change to prevent cross-analysis 404 report requests
- Update API_CONTRACT_BE_PYTHON.md to reflect endpoint and behavior changes

docs/API_CONTRACT_BE_PYTHON.md

@@ -1,6 +1,6 @@
 # Backend Agentic Service API Contract
 
-This document describes the Python agentic backend used by the frontend for AI chat, help/report tools, and observability data shown alongside chat answers.
+This document describes the Python agentic backend used by the frontend for AI chat, help/report tools, and traceability data shown alongside chat answers.
 
 Base path examples use relative URLs. Configure the frontend with the deployed Python service base URL.
 
@@ -11,14 +11,14 @@ The Python backend owns the generative AI interaction surface:
 1. Stream chat answers from the AI agent.
 2. Execute tool-style actions for help and report generation.
 3. Return report versions and report details.
-4. Return observability/provenance for a completed assistant answer.
+4. Return traceability for a completed assistant answer.
 
 The frontend uses this service during the analysis conversation flow:
 
 1. User sends a chat message.
 2. Frontend calls `POST /api/v2/chat/stream` and renders the streamed answer.
 3. When the stream emits `done`, frontend uses the returned `message_id` as the assistant answer correlation id.
-4. Frontend calls `GET /api/v1/observability` for planning, tool calls, and source provenance.
+4. Frontend calls `GET /api/v1/traceability` for planning, tool calls, and source provenance.
 5. Frontend calls `/api/v1/tools/help` for guided help and `/api/v1/tools/report` for report generation.
 
 ## Endpoint Summary
@@ -31,7 +31,7 @@ The frontend uses this service during the analysis conversation flow:
 | `POST` | `/api/v1/tools/report` | Generate and persist a new report version. |
 | `GET` | `/api/v1/tools/report/{analysis_id}` | List report versions for an analysis. |
 | `GET` | `/api/v1/tools/report/{analysis_id}/{version}` | Retrieve one report version. |
-| `GET` | `/api/v1/observability` | Retrieve provenance for one assistant answer. |
+| `GET` | `/api/v1/traceability` | Retrieve provenance for one assistant answer. |
 
 ## Common Concepts
 
@@ -39,7 +39,7 @@ The frontend uses this service during the analysis conversation flow:
 
 - `user_id`: user identifier passed by the frontend.
 - `analysis_id`: analysis conversation identifier.
-- `message_id`: assistant answer identifier generated by Python and returned in the stream `done` event; used to correlate chat streaming, Golang message persistence, and observability.
+- `message_id`: assistant answer identifier generated by Python and returned in the stream `done` event; used to correlate chat streaming, Golang message persistence, and traceability.
 
 ### Server-Sent Events
 
@@ -57,7 +57,7 @@ Common event types:
 | `done` | JSON object | Terminal success event. Includes `message_id`. |
 | `error` | text | Terminal error event. Stream stops after this. |
 
-The stream carries answer text only. Planning, tool call details, and full provenance are fetched from `GET /api/v1/observability` after the stream is done.
+The stream carries answer text only. Planning, tool call details, and full provenance are fetched from `GET /api/v1/traceability` after the stream is done.
 
 ## Chat
 
@@ -355,13 +355,13 @@ Response `404`:
 }
 ```
 
-## Observability
+## Traceability
 
-### `GET /api/v1/observability`
+### `GET /api/v1/traceability`
 
-Returns Responsible AI provenance for one assistant answer.
+Returns user-facing provenance for one assistant answer.
 
-The frontend should call this after the chat/help stream emits `done`, using the Python-generated `message_id` from the `done` event. If the row is not ready yet, the frontend may poll until `200` or stop on `404` according to product behavior.
+The frontend should call this after the chat/help stream emits `done`, using the `message_id` from the `done` event. The row is written **before** `done`, so an immediate GET returns `200` (no polling race). A `404` means the id is unknown or the turn errored before completing (error turns never produce a row).
 
 Query params:
 
@@ -373,15 +373,19 @@ Query params:
 Example:
 
 ```text
-GET /api/v1/observability?analysis_id=an_42&message_id=msg_88f1
+GET /api/v1/traceability?analysis_id=an_42&message_id=msg_88f1
 ```
 
+`intent` values the frontend may see: `chat` · `help` · `check` · `unstructured_flow` · `structured_flow` · `out_of_scope` · `blocked` (`blocked` = input-guard or Azure content-filter refusal; `chat` also covers the greeting fast-path and cache replays).
+
 Field rules:
 
-- `planning`: present only when the planner ran; otherwise `null`.
-- `thinking`: optional reasoning summary; `null` if unavailable.
-- `tool_calls`: every invoked tool with input, output, and status; empty for pure chat or greeting paths.
-- `sources`: required for retrieval flows; empty for chat/help paths that do not reference data.
+- `planning`: present only when the planner ran (`structured_flow`); otherwise `null`.
+- `thinking`: **always `null` in v1** — our agents are plain chat completions with no native reasoning output, and synthesizing it post-hoc would be unfaithful. The field stays in the payload so it can be populated later without a contract change.
+- `tool_calls`: every invoked tool with `input`, `output`, `status`, `task_id` (nullable), and `error` (nullable); empty for chat / help / greeting / refusal paths.
+- `sources`: required for retrieval flows; empty for chat / help / refusal paths and for `check`.
+- The payload also carries an internal `user_id` (ownership); the frontend may ignore it.
+- Truncation: `preview` ≤ 5 rows; any string inside `input`/`output`/`preview`/`snippet` ≤ 300 chars; rows beyond the preview are dropped (`row_count` is preserved).
 
 Response `200` for `structured_flow`:
 
@@ -389,49 +393,58 @@ Response `200` for `structured_flow`:
 {
   "analysis_id": "an_42",
   "message_id": "msg_88f1",
+  "user_id": "user_7",
   "intent": "structured_flow",
-  "generated_at": "2026-06-30T03:21:09.114Z",
+  "generated_at": "2026-07-06T03:21:09.114Z",
   "planning": {
     "goal_restated": "Find which regions drive revenue and why Q1 dipped.",
-    "assumptions": ["'last quarter' = Q1 2026"],
+    "assumptions": [],
     "steps": [
       {
         "step": 1,
         "stage": "data_understanding",
-        "objective": "Inventory the sales source"
+        "objective": "Inventory the sales source",
+        "status": "success",
+        "tools_used": ["check_data"]
       },
       {
         "step": 2,
         "stage": "modeling",
-        "objective": "Aggregate revenue by region"
+        "objective": "Aggregate revenue by region",
+        "status": "success",
+        "tools_used": ["retrieve_data", "analyze_aggregate"]
       }
     ]
   },
-  "thinking": "The question needs a per-region breakdown plus a cause, so I inventory the source, aggregate revenue by region, then compare quarters.",
+  "thinking": null,
   "tool_calls": [
     {
       "order": 1,
+      "task_id": null,
       "name": "check_data",
       "input": { "source_hint": "structured" },
-      "output": { "kind": "table", "summary": "1 source, 1 table, 48,213 rows" },
-      "status": "success"
+      "output": {
+        "kind": "table",
+        "columns": ["source_id", "name", "source_type", "table_count"],
+        "row_count": 1,
+        "preview": [["src_sales_db", "orders", "schema", 1]]
+      },
+      "status": "success",
+      "error": null
     },
     {
       "order": 2,
+      "task_id": null,
       "name": "retrieve_data",
-      "input": {
-        "source_id": "src_sales_db",
-        "table_id": "orders",
-        "select": ["region", "amount"],
-        "group_by": ["region"]
-      },
+      "input": { "ir": { "source_id": "src_sales_db", "table_id": "orders", "select": ["region", "amount"], "group_by": ["region"] } },
       "output": {
         "kind": "table",
         "columns": ["region", "total"],
         "row_count": 4,
         "preview": [["Central", 1210000], ["East", 740000]]
       },
-      "status": "success"
+      "status": "success",
+      "error": null
     }
   ],
   "sources": [
@@ -441,37 +454,36 @@ Response `200` for `structured_flow`:
       "name": "orders",
       "query": "SELECT region, SUM(amount) AS total FROM orders GROUP BY region",
       "detail": {
-        "tables": ["orders"],
-        "row_count": 48213
+        "table": "orders",
+        "row_count": 4
       }
     }
   ]
 }
 ```
 
+> Note: `retrieve_data`'s real `input` is the compiled query IR under an `ir` key (the planner builds an IR, never raw SQL). The executed SQL/rendered query appears on the corresponding `sources[].query`.
+
 Response `200` for `unstructured_flow`:
 
 ```json
 {
   "analysis_id": "an_42",
   "message_id": "msg_55",
+  "user_id": "user_7",
   "intent": "unstructured_flow",
-  "generated_at": "2026-06-30T03:40:02.001Z",
+  "generated_at": "2026-07-06T03:40:02.001Z",
   "planning": null,
   "thinking": null,
   "tool_calls": [
     {
       "order": 1,
+      "task_id": null,
       "name": "retrieve_knowledge",
-      "input": {
-        "query": "technology stack used in this project",
-        "top_k": 4
-      },
-      "output": {
-        "kind": "documents",
-        "row_count": 4
-      },
-      "status": "success"
+      "input": { "query": "technology stack used in this project" },
+      "output": { "kind": "documents", "row_count": 4 },
+      "status": "success",
+      "error": null
     }
   ],
   "sources": [
@@ -488,14 +500,15 @@ Response `200` for `unstructured_flow`:
 }
 ```
 
-Response `200` for simple chat or greeting:
+Response `200` for chat / greeting / help / refusals (`out_of_scope`, `blocked`):
 
 ```json
 {
   "analysis_id": "an_42",
   "message_id": "msg_12",
+  "user_id": "user_7",
   "intent": "chat",
-  "generated_at": "2026-06-30T03:05:00.000Z",
+  "generated_at": "2026-07-06T03:05:00.000Z",
   "planning": null,
   "thinking": null,
   "tool_calls": [],
@@ -507,13 +520,13 @@ Response `404`:
 
 ```json
 {
-  "detail": "No observability for message 'msg_88f1' yet."
+  "detail": "No traceability for message 'msg_88f1' yet."
 }
 ```
 
 Frontend rendering guidance:
 
-- Render observability separately from the streamed answer.
+- Render traceability separately from the streamed answer.
 - Default state can be collapsed.
 - Show planning, tool calls, and sources as separate sections.
-- Treat `planning: null`, `tool_calls: []`, and `sources: []` as valid states.
+- Treat `planning: null`, `tool_calls: []`, and `sources: []` as valid states.

src/app/components/analysis/AnalysisShell.tsx

@@ -39,7 +39,6 @@ function toUiMessage(message: AnalysisMessage): UiMessage {
   };
 }
 
-const wait = (ms: number) => new Promise((resolve) => window.setTimeout(resolve, ms));
 
 function normalizeAssistantContent(content: string) {
   return content
@@ -132,25 +131,17 @@ export function AnalysisShell() {
     if (!messageId) return;
     setMessages((prev) => prev.map((msg) => (msg.id === uiMessageId ? { ...msg, traceabilityLoading: true, traceabilityError: null } : msg)));
 
-    for (let attempt = 0; attempt < 3; attempt += 1) {
-      try {
-        const traceability = await getObservability(analysisId, messageId);
-        setMessages((prev) => prev.map((msg) => (msg.id === uiMessageId ? { ...msg, traceability, traceabilityLoading: false } : msg)));
-        return;
-      } catch (err) {
-        if (err instanceof AgenticApiError && err.status === 404 && attempt < 2) {
-          await wait(900);
-          continue;
-        }
-        const traceabilityError =
-          err instanceof AgenticApiError && err.status === 404
-            ? "Traceability is still being prepared. Try again after the next answer."
-            : err instanceof Error
-            ? err.message
-            : "Failed to load traceability";
-        setMessages((prev) => prev.map((msg) => (msg.id === uiMessageId ? { ...msg, traceabilityLoading: false, traceabilityError } : msg)));
-        return;
-      }
+    try {
+      const traceability = await getObservability(analysisId, messageId);
+      setMessages((prev) => prev.map((msg) => (msg.id === uiMessageId ? { ...msg, traceability, traceabilityLoading: false } : msg)));
+    } catch (err) {
+      const traceabilityError =
+        err instanceof AgenticApiError && err.status === 404
+          ? "Traceability not available for this answer."
+          : err instanceof Error
+          ? err.message
+          : "Failed to load traceability";
+      setMessages((prev) => prev.map((msg) => (msg.id === uiMessageId ? { ...msg, traceabilityLoading: false, traceabilityError } : msg)));
     }
   };
   useEffect(() => {

src/app/components/analysis/ReportSidebar.tsx

@@ -53,11 +53,9 @@ export function ReportSidebar({ analysis, userId, onCollapse }: ReportSidebarPro
   useEffect(() => {
     setDetail(null);
     setPrecondition(null);
+    setVersions([]);
+    setSelectedVersion(undefined);
     if (analysis?.id) loadVersions(analysis.id, { silent: true });
-    else {
-      setVersions([]);
-      setSelectedVersion(undefined);
-    }
   }, [analysis?.id]);
 
   useEffect(() => {

src/services/agenticApi.ts

@@ -205,7 +205,7 @@ export const listTools = (): Promise<{ count: number; tools: Tool[] }> => agenti
 
 export const getObservability = (analysisId: string, messageId: string): Promise<Observability> =>
   agenticJson(
-    `/api/v1/observability?analysis_id=${encodeURIComponent(analysisId)}&message_id=${encodeURIComponent(messageId)}`
+    `/api/v1/traceability?analysis_id=${encodeURIComponent(analysisId)}&message_id=${encodeURIComponent(messageId)}`
   );
 
 export const generateReport = (analysisId: string, userId: string): Promise<ReportDetail> =>