Rifqi Hafizuddin Claude Opus 4.8 commited on
Commit
c1e4608
Β·
1 Parent(s): 1570c5d

[NOTICKET] feat(orchestrator): flat 6-intent router + gate + check skill

Browse files

Rework the intent router from the 3-way source_hint classifier to a flat
6-intent handler router (chat, help, problem_statement, check,
unstructured_flow, structured_flow). Structured/unstructured data modality on
the slow path stays the Planner's job, not the router's.

- orchestration.py: RouterDecision { intent, rewritten_query, confidence };
kept GPT-4o + the classify() seam.
- intent_router.md: v2 prompt (labels, disambiguation, per-intent few-shots).
- chat_handler.py: dispatch on intent; chat/unstructured_flow/structured_flow/
check wired; problem_statement/help stubbed; back-compat source_hint on the
SSE intent event.
- gate.py: deterministic gate (structured_flow requires interview pass) +
AnalysisState stub contract (pending the lead's table).
- handlers/check.py: check_data/check_knowledge tool-pick + table render.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

src/agents/chat_handler.py CHANGED
@@ -2,12 +2,14 @@
2
 
3
  End-to-end flow per user message:
4
 
5
- 1. `IntentRouter.classify` β†’ `chat` / `unstructured` / `structured`.
6
- 2. Route:
7
- - `chat` β†’ no context. Pass straight to ChatbotAgent.
8
- - `structured` β†’ CatalogReader β†’ QueryService β†’ QueryResult.
9
- - `unstructured` β†’ DocumentRetriever (placeholder, raises until TAB
10
- ships) β†’ list[DocumentChunk].
 
 
11
  3. `ChatbotAgent.astream` β†’ yield text tokens.
12
  4. Wrap each step into an SSE-style event dict so the API endpoint can
13
  stream them as Server-Sent Events.
@@ -33,6 +35,7 @@ from src.middlewares.logging import get_logger
33
  from src.retrieval.base import RetrievalResult
34
 
35
  from .chatbot import ChatbotAgent, DocumentChunk
 
36
  from .orchestration import OrchestratorAgent
37
 
38
  if TYPE_CHECKING:
@@ -71,6 +74,7 @@ class ChatHandler:
71
  Callable[[str], SlowPathCoordinator] | None
72
  ) = None,
73
  analysis_store: AnalysisStore | None = None,
 
74
  enable_tracing: bool = False,
75
  ) -> None:
76
  self._intent_router = intent_router
@@ -88,6 +92,9 @@ class ChatHandler:
88
  self._enable_slow_path = enable_slow_path
89
  self._slow_path_factory = slow_path_coordinator_factory
90
  self._analysis_store = analysis_store
 
 
 
91
 
92
  # ------------------------------------------------------------------
93
  # Lazy default-dep builders
@@ -125,6 +132,14 @@ class ChatHandler:
125
  self._document_retriever = RetrievalRouter()
126
  return self._document_retriever
127
 
 
 
 
 
 
 
 
 
128
  # ------------------------------------------------------------------
129
  # Public entry
130
  # ------------------------------------------------------------------
@@ -147,7 +162,12 @@ class ChatHandler:
147
  yield {"event": "error", "data": f"Could not classify message: {e}"}
148
  return
149
 
150
- yield {"event": "intent", "data": decision.model_dump_json()}
 
 
 
 
 
151
 
152
  rewritten = decision.rewritten_query or message
153
  query_result = None
@@ -155,7 +175,7 @@ class ChatHandler:
155
  raw_chunks: Any = None
156
 
157
  # ---- 2. Route ------------------------------------------------
158
- if decision.source_hint == "structured":
159
  try:
160
  # One memoizing reader per request: the same catalog is otherwise
161
  # re-fetched from the catalog DB 4-5x across the slow-path run. This
@@ -182,7 +202,7 @@ class ChatHandler:
182
  )
183
  yield {"event": "error", "data": f"Structured query failed: {e}"}
184
  return
185
- elif decision.source_hint == "unstructured":
186
  try:
187
  raw_chunks = await self._get_document_retriever().retrieve(
188
  rewritten, user_id
@@ -201,13 +221,35 @@ class ChatHandler:
201
  )
202
  yield {"event": "error", "data": f"Document retrieval failed: {e}"}
203
  return
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
204
  # else: chat path β€” no context
205
 
206
  # ---- 2b. Emit sources ---------------------------------------
207
- sources = _build_sources(
208
- decision.source_hint, user_id, query_result, raw_chunks
 
 
 
 
209
  )
210
- logger.info("built sources", source_hint=decision.source_hint, sources_count=len(sources), raw_chunks_count=len(raw_chunks) if raw_chunks else 0)
211
  yield {"event": "sources", "data": json.dumps(sources)}
212
 
213
  # ---- 3. Stream answer ----------------------------------------
@@ -378,19 +420,33 @@ class ChatHandler:
378
  yield {"event": "done", "data": ""}
379
 
380
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
381
  def _build_sources(
382
- source_hint: str,
383
  user_id: str,
384
  query_result: Any,
385
  raw_chunks: Any,
386
  ) -> list[dict[str, Any]]:
387
  """Build the sources payload for the SSE `sources` event.
388
 
389
- - structured: one entry per executed table (table_name only).
390
- - unstructured: deduped by (document_id, page_label), Phase 1 shape.
391
  - chat or error: empty list.
392
  """
393
- if source_hint == "structured":
394
  if query_result is None or getattr(query_result, "error", None):
395
  return []
396
  table_name = getattr(query_result, "table_name", "") or ""
@@ -402,7 +458,7 @@ def _build_sources(
402
  "page_label": None,
403
  }]
404
 
405
- if source_hint == "unstructured" and raw_chunks:
406
  seen: set[tuple[Any, Any]] = set()
407
  sources: list[dict[str, Any]] = []
408
  for item in raw_chunks:
 
2
 
3
  End-to-end flow per user message:
4
 
5
+ 1. `OrchestratorAgent.classify` β†’ RouterDecision (one of six intents).
6
+ 2. Route by intent:
7
+ - `chat` β†’ no context. Pass straight to ChatbotAgent.
8
+ - `structured_flow` β†’ CatalogReader β†’ slow path / QueryService.
9
+ - `unstructured_flow` β†’ DocumentRetriever (placeholder, raises until TAB
10
+ ships) β†’ list[DocumentChunk].
11
+ - `check` β†’ check_data / check_knowledge tool β†’ rendered table.
12
+ - `problem_statement` / `help` β†’ placeholder (handlers pending).
13
  3. `ChatbotAgent.astream` β†’ yield text tokens.
14
  4. Wrap each step into an SSE-style event dict so the API endpoint can
15
  stream them as Server-Sent Events.
 
35
  from src.retrieval.base import RetrievalResult
36
 
37
  from .chatbot import ChatbotAgent, DocumentChunk
38
+ from .handlers.check import run_check
39
  from .orchestration import OrchestratorAgent
40
 
41
  if TYPE_CHECKING:
 
74
  Callable[[str], SlowPathCoordinator] | None
75
  ) = None,
76
  analysis_store: AnalysisStore | None = None,
77
+ check_invoker_factory: Callable[[str], Any] | None = None,
78
  enable_tracing: bool = False,
79
  ) -> None:
80
  self._intent_router = intent_router
 
92
  self._enable_slow_path = enable_slow_path
93
  self._slow_path_factory = slow_path_coordinator_factory
94
  self._analysis_store = analysis_store
95
+ # `check` skill: builds the data-access invoker (check_data/check_knowledge)
96
+ # per request with the authenticated user_id. Injectable for tests.
97
+ self._check_invoker_factory = check_invoker_factory
98
 
99
  # ------------------------------------------------------------------
100
  # Lazy default-dep builders
 
132
  self._document_retriever = RetrievalRouter()
133
  return self._document_retriever
134
 
135
+ def _get_check_invoker(self, user_id: str) -> Any:
136
+ """Build the per-request data-access invoker for the `check` skill."""
137
+ if self._check_invoker_factory is not None:
138
+ return self._check_invoker_factory(user_id)
139
+ from ..tools.data_access import DataAccessToolInvoker
140
+
141
+ return DataAccessToolInvoker(user_id, self._get_catalog_reader())
142
+
143
  # ------------------------------------------------------------------
144
  # Public entry
145
  # ------------------------------------------------------------------
 
162
  yield {"event": "error", "data": f"Could not classify message: {e}"}
163
  return
164
 
165
+ intent = decision.intent
166
+ # Back-compat: the frontend still reads `source_hint` off the intent event.
167
+ # Derive it from the new intent until the frontend migrates to `intent`.
168
+ event_data = decision.model_dump()
169
+ event_data["source_hint"] = _intent_to_source_hint(intent)
170
+ yield {"event": "intent", "data": json.dumps(event_data)}
171
 
172
  rewritten = decision.rewritten_query or message
173
  query_result = None
 
175
  raw_chunks: Any = None
176
 
177
  # ---- 2. Route ------------------------------------------------
178
+ if intent == "structured_flow":
179
  try:
180
  # One memoizing reader per request: the same catalog is otherwise
181
  # re-fetched from the catalog DB 4-5x across the slow-path run. This
 
202
  )
203
  yield {"event": "error", "data": f"Structured query failed: {e}"}
204
  return
205
+ elif intent == "unstructured_flow":
206
  try:
207
  raw_chunks = await self._get_document_retriever().retrieve(
208
  rewritten, user_id
 
221
  )
222
  yield {"event": "error", "data": f"Document retrieval failed: {e}"}
223
  return
224
+ elif intent == "check":
225
+ try:
226
+ invoker = self._get_check_invoker(user_id)
227
+ text = await run_check(rewritten, invoker)
228
+ except Exception as e:
229
+ logger.error("check route failed", user_id=user_id, error=str(e))
230
+ yield {"event": "error", "data": f"Lookup failed: {e}"}
231
+ return
232
+ yield {"event": "chunk", "data": text}
233
+ yield {"event": "done", "data": ""}
234
+ return
235
+ elif intent in ("problem_statement", "help"):
236
+ # The router emits these new intents, but their handlers are not wired
237
+ # yet (PS skill is the lead's; help skill deferred). Placeholder keeps
238
+ # dispatch exhaustive; real handlers land in a follow-up step.
239
+ logger.info("unwired intent stub", intent=intent)
240
+ yield {"event": "chunk", "data": _stub_message(intent)}
241
+ yield {"event": "done", "data": ""}
242
+ return
243
  # else: chat path β€” no context
244
 
245
  # ---- 2b. Emit sources ---------------------------------------
246
+ sources = _build_sources(intent, user_id, query_result, raw_chunks)
247
+ logger.info(
248
+ "built sources",
249
+ intent=intent,
250
+ sources_count=len(sources),
251
+ raw_chunks_count=len(raw_chunks) if raw_chunks else 0,
252
  )
 
253
  yield {"event": "sources", "data": json.dumps(sources)}
254
 
255
  # ---- 3. Stream answer ----------------------------------------
 
420
  yield {"event": "done", "data": ""}
421
 
422
 
423
+ def _intent_to_source_hint(intent: str) -> str:
424
+ """Map a router intent to the legacy `source_hint` value (frontend back-compat)."""
425
+ if intent == "structured_flow":
426
+ return "structured"
427
+ if intent == "unstructured_flow":
428
+ return "unstructured"
429
+ return "chat"
430
+
431
+
432
+ def _stub_message(intent: str) -> str:
433
+ """Placeholder reply for intents whose handlers are not wired yet."""
434
+ return f"The '{intent}' route is recognized but not wired up yet."
435
+
436
+
437
  def _build_sources(
438
+ intent: str,
439
  user_id: str,
440
  query_result: Any,
441
  raw_chunks: Any,
442
  ) -> list[dict[str, Any]]:
443
  """Build the sources payload for the SSE `sources` event.
444
 
445
+ - structured_flow: one entry per executed table (table_name only).
446
+ - unstructured_flow: deduped by (document_id, page_label), Phase 1 shape.
447
  - chat or error: empty list.
448
  """
449
+ if intent == "structured_flow":
450
  if query_result is None or getattr(query_result, "error", None):
451
  return []
452
  table_name = getattr(query_result, "table_name", "") or ""
 
458
  "page_label": None,
459
  }]
460
 
461
+ if intent == "unstructured_flow" and raw_chunks:
462
  seen: set[tuple[Any, Any]] = set()
463
  sources: list[dict[str, Any]] = []
464
  for item in raw_chunks:
src/agents/gate.py ADDED
@@ -0,0 +1,71 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """Deterministic routing gate β€” policy check over the router's intent.
2
+
3
+ After the LLM router picks an intent, the gate checks it against the per-analysis
4
+ Analysis State and returns the **effective** intent: allow as-is, or redirect. No
5
+ LLM, no I/O in `gate()` itself.
6
+
7
+ Only one rule has teeth in v1: an analytical request (`structured_flow`) requires a
8
+ passed interview (`interview_status == "pass"`); otherwise it is redirected to
9
+ `problem_statement` so the user defines the goal first. Everything else passes
10
+ through. `generate_report` is not a router intent (button/report API), so it is not
11
+ gated here.
12
+
13
+ `AnalysisState` + `get_analysis_state` are a **stub contract** today β€” the real
14
+ per-analysis row is owned by the lead (Postgres). Field names are pinned to match
15
+ the planned table so the swap is zero-change. See `ORCHESTRATOR_REWORK_PLAN.md` Β§4.
16
+ """
17
+
18
+ from __future__ import annotations
19
+
20
+ from typing import Literal
21
+
22
+ from pydantic import BaseModel, Field
23
+
24
+ from src.agents.orchestration import Intent
25
+ from src.middlewares.logging import get_logger
26
+
27
+ logger = get_logger("gate")
28
+
29
+ InterviewStatus = Literal["not_pass", "pass"]
30
+
31
+
32
+ class AnalysisState(BaseModel):
33
+ """Per-analysis state the gate reads every turn (stub contract).
34
+
35
+ Owned by the lead's Analysis State table; consumed here read-only.
36
+ `report_status` is `no_report` or a version tag (`V1`, `V2`, …) β€” unused by the
37
+ gate today, carried for the dispatch layer.
38
+ """
39
+
40
+ user_id: str
41
+ data_source_ids: list[str] = Field(default_factory=list)
42
+ interview_status: InterviewStatus = "not_pass"
43
+ report_status: str = "no_report"
44
+
45
+
46
+ def gate(intent: Intent, state: AnalysisState) -> Intent:
47
+ """Return the effective intent after applying the deterministic gate policy.
48
+
49
+ `structured_flow` requires `interview_status == "pass"`; otherwise redirect to
50
+ `problem_statement`. All other intents pass through unchanged.
51
+ """
52
+ if intent == "structured_flow" and state.interview_status != "pass":
53
+ logger.info(
54
+ "gate redirect",
55
+ requested=intent,
56
+ effective="problem_statement",
57
+ reason="interview_not_passed",
58
+ )
59
+ return "problem_statement"
60
+ return intent
61
+
62
+
63
+ async def get_analysis_state(analysis_id: str) -> AnalysisState:
64
+ """Load the Analysis State for an analysis (STUB).
65
+
66
+ Returns the table-default state until the lead's Postgres-backed reader lands.
67
+ Swap this body for the real query; the signature + return type stay frozen so
68
+ the dispatch layer never changes.
69
+ """
70
+ logger.debug("get_analysis_state stub", analysis_id=analysis_id)
71
+ return AnalysisState(user_id="", interview_status="not_pass")
src/agents/handlers/__init__.py ADDED
@@ -0,0 +1 @@
 
 
1
+ """Deterministic skill handlers dispatched by the orchestrator (non-LLM)."""
src/agents/handlers/check.py ADDED
@@ -0,0 +1,64 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """`check` skill handler β€” deterministic data/document inventory (no LLM).
2
+
3
+ The router emits a single `check` intent; this handler picks the concrete tool
4
+ (`check_data` for structured sources, `check_knowledge` for documents) and renders
5
+ the tool's `ToolOutput` table into a markdown reply. Tool selection is a cheap
6
+ keyword cue today; once Analysis State is wired it can prefer the bound-source
7
+ type. See `ORCHESTRATOR_REWORK_PLAN.md` Β§2.
8
+
9
+ The data-access invoker never throws (Β§8.4); `render_tool_output` handles the
10
+ `error` envelope defensively.
11
+ """
12
+
13
+ from __future__ import annotations
14
+
15
+ from typing import TYPE_CHECKING
16
+
17
+ from src.tools.contracts import ToolOutput
18
+
19
+ if TYPE_CHECKING:
20
+ from src.agents.slow_path.invoker import ToolInvoker
21
+
22
+ # Cues that point at documents rather than structured data. Anything else β†’ data.
23
+ _KNOWLEDGE_CUES = (
24
+ "document",
25
+ "docs",
26
+ "doc ",
27
+ "file",
28
+ "pdf",
29
+ "docx",
30
+ ".txt",
31
+ "uploaded",
32
+ "knowledge",
33
+ )
34
+
35
+
36
+ def pick_check_tool(message: str) -> str:
37
+ """Choose `check_knowledge` for document-flavoured asks, else `check_data`."""
38
+ lowered = message.lower()
39
+ if any(cue in lowered for cue in _KNOWLEDGE_CUES):
40
+ return "check_knowledge"
41
+ return "check_data"
42
+
43
+
44
+ def render_tool_output(out: ToolOutput) -> str:
45
+ """Render a `check_*` ToolOutput table into a markdown reply."""
46
+ if out.kind == "error":
47
+ return f"Sorry, I couldn't look that up: {out.error}"
48
+ columns = out.columns or []
49
+ rows = out.rows or []
50
+ if not rows:
51
+ return "Nothing registered yet β€” I don't see any matching sources."
52
+ header = "| " + " | ".join(columns) + " |"
53
+ separator = "| " + " | ".join("---" for _ in columns) + " |"
54
+ body = "\n".join(
55
+ "| " + " | ".join(str(cell) for cell in row) + " |" for row in rows
56
+ )
57
+ return f"{header}\n{separator}\n{body}"
58
+
59
+
60
+ async def run_check(message: str, invoker: ToolInvoker) -> str:
61
+ """Pick the check tool, invoke it (no args = inventory), render the result."""
62
+ tool = pick_check_tool(message)
63
+ out = await invoker.invoke(tool, {})
64
+ return render_tool_output(out)
src/agents/orchestration.py CHANGED
@@ -1,13 +1,17 @@
1
- """OrchestratorAgent β€” classifies a user message and emits source_hint.
2
 
3
- Output: needs_search (bool) + source_hint ∈ { chat, unstructured, structured }
4
- + rewritten_query (standalone form of the user's question, history-resolved).
5
 
6
- Phase 2 replaces the previous intent-classification body. The class name
7
- is preserved so existing import sites (`from src.agents.orchestration
8
- import OrchestratorAgent`) keep working. The default LLM chain is
9
- constructed lazily so the module is import-safe even without `.env`
10
- populated.
 
 
 
 
 
11
  """
12
 
13
  from __future__ import annotations
@@ -25,7 +29,14 @@ from src.middlewares.logging import get_logger
25
 
26
  logger = get_logger("orchestrator")
27
 
28
- SourceHint = Literal["chat", "unstructured", "structured"]
 
 
 
 
 
 
 
29
 
30
  _PROMPT_PATH = (
31
  Path(__file__).resolve().parent.parent
@@ -35,21 +46,29 @@ _PROMPT_PATH = (
35
  )
36
 
37
 
38
- class IntentRouterDecision(BaseModel):
39
  """LLM output. Pydantic so it can be used with `with_structured_output`."""
40
 
41
- needs_search: bool = Field(
42
- ..., description="True if we must look at the user's data to answer."
43
- )
44
- source_hint: SourceHint = Field(
45
  ...,
46
- description="Which downstream path: 'chat' (no lookup), "
47
- "'unstructured' (PDF/DOCX/TXT prose), 'structured' (DB / tabular file).",
 
 
 
 
 
48
  )
49
  rewritten_query: str | None = Field(
50
  None,
51
- description="Standalone version of the question, history-resolved. "
52
- "Null when needs_search=false.",
 
 
 
 
 
 
53
  )
54
 
55
 
@@ -74,11 +93,11 @@ def _build_default_chain() -> Runnable:
74
  ("human", "{message}"),
75
  ]
76
  )
77
- return prompt | llm.with_structured_output(IntentRouterDecision)
78
 
79
 
80
  class OrchestratorAgent:
81
- """Classifies a user message into chat / unstructured / structured.
82
 
83
  Inject `structured_chain` for tests; default builds the production
84
  Azure OpenAI chain on first use.
@@ -97,18 +116,14 @@ class OrchestratorAgent:
97
  message: str,
98
  history: list[BaseMessage] | None = None,
99
  callbacks: list | None = None,
100
- ) -> IntentRouterDecision:
101
  chain = self._ensure_chain()
102
  payload = {"message": message, "history": history or []}
103
  if callbacks:
104
- decision: IntentRouterDecision = await chain.ainvoke(
105
  payload, config={"callbacks": callbacks}
106
  )
107
  else:
108
  decision = await chain.ainvoke(payload)
109
- logger.info(
110
- "intent classified",
111
- source_hint=decision.source_hint,
112
- needs_search=decision.needs_search,
113
- )
114
  return decision
 
1
+ """OrchestratorAgent β€” classifies a user message into one of six intents.
2
 
3
+ Output: RouterDecision { intent, rewritten_query, confidence }.
 
4
 
5
+ The router is a **handler-level** intent classifier, not a data-modality
6
+ classifier: `structured_flow` routes to the slow Planner spine and
7
+ `unstructured_flow` to the fast RAG path; the structured/unstructured data mix on
8
+ the slow path is the Planner's job, not the router's. See
9
+ `ORCHESTRATOR_REWORK_PLAN.md`.
10
+
11
+ The class name `OrchestratorAgent` is preserved so existing import sites
12
+ (`from src.agents.orchestration import OrchestratorAgent`) keep working. The
13
+ default LLM chain is built lazily so the module is import-safe even without
14
+ `.env` populated.
15
  """
16
 
17
  from __future__ import annotations
 
29
 
30
  logger = get_logger("orchestrator")
31
 
32
+ Intent = Literal[
33
+ "chat",
34
+ "help",
35
+ "problem_statement",
36
+ "check",
37
+ "unstructured_flow",
38
+ "structured_flow",
39
+ ]
40
 
41
  _PROMPT_PATH = (
42
  Path(__file__).resolve().parent.parent
 
46
  )
47
 
48
 
49
+ class RouterDecision(BaseModel):
50
  """LLM output. Pydantic so it can be used with `with_structured_output`."""
51
 
52
+ intent: Intent = Field(
 
 
 
53
  ...,
54
+ description=(
55
+ "Handler route for this message: 'chat' (conversational, no data), "
56
+ "'help' (what-to-do-next guidance), 'problem_statement' (define or "
57
+ "refine the analysis goal), 'check' (inventory: what data/documents "
58
+ "exist), 'unstructured_flow' (answer from documents, fast RAG), or "
59
+ "'structured_flow' (analytical question over data, slow Planner path)."
60
+ ),
61
  )
62
  rewritten_query: str | None = Field(
63
  None,
64
+ description=(
65
+ "Standalone version of the question, history-resolved. Null for "
66
+ "'chat' and 'help' (no data lookup needed)."
67
+ ),
68
+ )
69
+ confidence: float | None = Field(
70
+ None,
71
+ description="Classifier confidence in [0, 1]. Optional.",
72
  )
73
 
74
 
 
93
  ("human", "{message}"),
94
  ]
95
  )
96
+ return prompt | llm.with_structured_output(RouterDecision)
97
 
98
 
99
  class OrchestratorAgent:
100
+ """Classifies a user message into one of the six router intents.
101
 
102
  Inject `structured_chain` for tests; default builds the production
103
  Azure OpenAI chain on first use.
 
116
  message: str,
117
  history: list[BaseMessage] | None = None,
118
  callbacks: list | None = None,
119
+ ) -> RouterDecision:
120
  chain = self._ensure_chain()
121
  payload = {"message": message, "history": history or []}
122
  if callbacks:
123
+ decision: RouterDecision = await chain.ainvoke(
124
  payload, config={"callbacks": callbacks}
125
  )
126
  else:
127
  decision = await chain.ainvoke(payload)
128
+ logger.info("intent classified", intent=decision.intent)
 
 
 
 
129
  return decision
src/config/prompts/intent_router.md CHANGED
@@ -1,82 +1,119 @@
1
- You are the intent router for an AI data assistant. Given a user's latest message (and optionally recent conversation history), decide which downstream path should handle it.
2
 
3
  ## Output
4
 
5
  Return three fields:
6
 
7
- - **`needs_search`** β€” `true` if we must look at the user's data to answer; `false` for greetings, farewells, off-topic chitchat, or meta questions about the assistant itself.
8
- - **`source_hint`** β€” one of:
9
- - `chat` β€” no data lookup needed (greetings, farewells, generic small talk).
10
- - `unstructured` β€” the user is asking about a topic, concept, feature, or factual knowledge that may exist in uploaded documents (PDF / DOCX / TXT). The user does not need to explicitly mention a document.
11
- - `structured` β€” the user is asking a **data question** answerable from a database or a tabular file (CSV / XLSX / Parquet). This includes counts, sums, top-N, filters, comparisons, trends, joins across registered structured sources.
12
- - **`rewritten_query`** β€” a **standalone** version of the user's question that incorporates necessary context from history. If the original message is already standalone, return it unchanged. If `needs_search` is `false`, leave this empty/null.
 
 
 
13
 
14
  ## Routing rules
15
 
16
- 1. If the message is ONLY a pure greeting / farewell / thanks / "how are you" / "what can you do" / compliment with no factual question β†’ `chat` + `needs_search=false`.
17
- 2. If the message asks a data question answerable from a database or tabular file (counts, sums, top-N, filters, comparisons, trends, sheet rows, table columns) β†’ `structured` + `needs_search=true`.
18
- 3. If the message asks about a topic, concept, feature, explanation, summary, or factual knowledge β€” even without explicitly mentioning a document β€” route to `unstructured` + `needs_search=true`. The user may have uploaded relevant documents covering that topic.
19
- 4. If ambiguous between structured and unstructured β†’ prefer `unstructured`. Only prefer `structured` if there are clear signals of tabular/numeric data questions.
20
- 5. Cross-source comparison ("compare DB sales to the customers.csv file") β†’ `structured`. The planner sees both source types in one prompt and can correlate.
 
 
 
 
 
 
 
 
21
 
22
  ## Rewriting follow-ups
23
 
24
- When history is present and the new message references prior context using pronouns or fragments ("tell me more", "what about last quarter?", "and by region?"), expand the rewritten_query into a fully standalone question. Example:
25
 
26
  History: "What was our top product last month?" β†’ "Pro Plan Annual at $487k"
27
  Message: "How does that compare to Q1?"
28
  rewritten_query: "How does Pro Plan Annual's revenue last month compare to Q1?"
29
 
30
- If the original is already standalone, copy it verbatim into rewritten_query.
31
 
32
  ## Few-shot examples
33
 
34
  ```
35
  User: "Hi"
36
- β†’ needs_search=false, source_hint="chat", rewritten_query=null
37
 
38
  User: "Bye, thanks"
39
- β†’ needs_search=false, source_hint="chat", rewritten_query=null
40
 
41
  User: "What can you do?"
42
- β†’ needs_search=false, source_hint="chat", rewritten_query=null
43
 
44
- User: "How many orders did we get last month?"
45
- β†’ needs_search=true, source_hint="structured",
46
- rewritten_query="How many orders did we get last month?"
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
47
 
48
  User: "What does the Q1 board memo say about churn?"
49
- β†’ needs_search=true, source_hint="unstructured",
50
- rewritten_query="What does the Q1 board memo say about churn?"
51
 
52
- User: "Top 5 customers by revenue this year"
53
- β†’ needs_search=true, source_hint="structured",
54
- rewritten_query="Top 5 customers by revenue this year"
55
 
56
  User: "apa key feature dari iot connectivity?"
57
- β†’ needs_search=true, source_hint="unstructured",
58
- rewritten_query="What are the key features of IoT connectivity?"
59
 
60
- User: "jelaskan tentang machine learning"
61
- β†’ needs_search=true, source_hint="unstructured",
62
- rewritten_query="Explain machine learning"
 
 
 
 
63
 
64
- User: "bagaimana cara kerja neural network?"
65
- β†’ needs_search=true, source_hint="unstructured",
66
- rewritten_query="How does a neural network work?"
67
 
68
- User: "what is the main purpose of this system?"
69
- β†’ needs_search=true, source_hint="unstructured",
70
- rewritten_query="What is the main purpose of this system?"
 
71
 
72
  History: assistant: "Pro Plan Annual led at $487,200 in April."
73
  User: "And in March?"
74
- β†’ needs_search=true, source_hint="structured",
75
- rewritten_query="What was Pro Plan Annual's revenue in March?"
76
  ```
77
 
78
  ## Constraints
79
 
80
- - Do not invent data. If the question is factual or knowledge-based (not clearly tabular), route to `unstructured` and let the retriever decide. Only route to `structured` if the question clearly involves counts, sums, filters, or trends from tabular sources.
 
81
  - Do not refuse β€” refusal happens later in guardrails. Just classify.
82
  - One JSON object as output; no prose, no markdown.
 
1
+ You are the intent router for an AI data assistant. Given a user's latest message (and optionally recent conversation history), decide which downstream **handler** should process it. You classify the route only β€” you do not answer the question.
2
 
3
  ## Output
4
 
5
  Return three fields:
6
 
7
+ - **`intent`** β€” exactly one of:
8
+ - `chat` β€” conversational, no data needed: greetings, farewells, thanks, "how are you", "what can you do", small talk.
9
+ - `help` β€” the user wants to know **what to do next** or how the process works ("what's the next step?", "how do I start?", "what should I do now?").
10
+ - `problem_statement` β€” the user wants to **define or refine the analysis goal**: the business problem, objectives, what to increase/decrease, targets/success metrics β€” or is answering questions about the goal.
11
+ - `check` β€” the user wants an **inventory** of what they have: "what data do I have?", "what columns are in this table?", "what documents did I upload?", "describe my dataset". This is metadata/listing, not analysis.
12
+ - `unstructured_flow` β€” the user asks about a **topic, concept, feature, explanation, or factual knowledge** that may live in uploaded documents (PDF/DOCX/TXT). Pure document Q&A. The user need not mention a document.
13
+ - `structured_flow` β€” the user asks an **analytical question over their data**: counts, sums, top-N, filters, comparisons, trends, correlations, segments, share-of-total, joins across structured sources. This routes to the slow analytical path.
14
+ - **`rewritten_query`** β€” a **standalone** version of the user's question, with context from history resolved. If the message is already standalone, copy it verbatim. Leave empty/null for `chat` and `help`.
15
+ - **`confidence`** β€” your confidence in the chosen intent, a number in [0, 1].
16
 
17
  ## Routing rules
18
 
19
+ 1. Pure greeting / farewell / thanks / "what can you do" / compliment with no task β†’ `chat`.
20
+ 2. "What do I do next / how do I proceed / where do I start" β†’ `help`.
21
+ 3. The user states or refines a goal, objective, target, or success metric, or answers a goal-defining question β†’ `problem_statement`.
22
+ 4. "What data / columns / tables / documents do I have", "describe my data", inventory or metadata requests β†’ `check`.
23
+ 5. A question answerable from document prose β€” a topic, concept, feature, explanation, summary, or factual knowledge, even without naming a document β†’ `unstructured_flow`.
24
+ 6. An analytical question answerable by computing over tabular/DB data (counts, sums, top-N, filters, comparisons, trends, correlations, segments) β†’ `structured_flow`.
25
+
26
+ ## Disambiguation (the boundaries that matter)
27
+
28
+ - **`check` vs `structured_flow`** β€” "what do I have / describe it" β†’ `check`; "analyze / compute / trend / correlate / compare it" β†’ `structured_flow`.
29
+ - **`unstructured_flow` vs `structured_flow`** β€” pure document/concept Q&A β†’ `unstructured_flow`; anything needing computation over tabular/DB data β†’ `structured_flow`. **When in doubt between "analytical AND also needs document context" β†’ `structured_flow`** (the analytical path can pull document context itself). Only choose `unstructured_flow` for *pure* document questions with no computation.
30
+ - **`help` vs `problem_statement`** β€” "what's next?" β†’ `help`; "here is my goal / let's define the objective" β†’ `problem_statement`.
31
+ - **`chat` vs everything else** β€” only use `chat` when there is no task and no data question at all.
32
 
33
  ## Rewriting follow-ups
34
 
35
+ When history is present and the new message references prior context with pronouns or fragments ("tell me more", "what about last quarter?", "and by region?"), expand `rewritten_query` into a fully standalone question. Example:
36
 
37
  History: "What was our top product last month?" β†’ "Pro Plan Annual at $487k"
38
  Message: "How does that compare to Q1?"
39
  rewritten_query: "How does Pro Plan Annual's revenue last month compare to Q1?"
40
 
41
+ If the original is already standalone, copy it verbatim into `rewritten_query`.
42
 
43
  ## Few-shot examples
44
 
45
  ```
46
  User: "Hi"
47
+ β†’ intent="chat", rewritten_query=null, confidence=0.99
48
 
49
  User: "Bye, thanks"
50
+ β†’ intent="chat", rewritten_query=null, confidence=0.99
51
 
52
  User: "What can you do?"
53
+ β†’ intent="chat", rewritten_query=null, confidence=0.95
54
 
55
+ User: "Okay I uploaded my data, what do I do next?"
56
+ β†’ intent="help", rewritten_query=null, confidence=0.93
57
+
58
+ User: "How does this work? Where should I start?"
59
+ β†’ intent="help", rewritten_query=null, confidence=0.9
60
+
61
+ User: "I want to reduce customer churn next quarter, target under 5%."
62
+ β†’ intent="problem_statement",
63
+ rewritten_query="Define the analysis goal: reduce customer churn next quarter to under 5%.",
64
+ confidence=0.9
65
+
66
+ User: "My goal is to grow revenue in the north region."
67
+ β†’ intent="problem_statement",
68
+ rewritten_query="Define the analysis goal: grow revenue in the north region.",
69
+ confidence=0.88
70
+
71
+ User: "What data do I have?"
72
+ β†’ intent="check", rewritten_query="What data sources do I have?", confidence=0.95
73
+
74
+ User: "What columns are in the orders table?"
75
+ β†’ intent="check", rewritten_query="What columns are in the orders table?", confidence=0.93
76
+
77
+ User: "What documents have I uploaded?"
78
+ β†’ intent="check", rewritten_query="What documents have I uploaded?", confidence=0.93
79
 
80
  User: "What does the Q1 board memo say about churn?"
81
+ β†’ intent="unstructured_flow",
82
+ rewritten_query="What does the Q1 board memo say about churn?", confidence=0.9
83
 
84
+ User: "jelaskan tentang machine learning"
85
+ β†’ intent="unstructured_flow", rewritten_query="Explain machine learning", confidence=0.85
 
86
 
87
  User: "apa key feature dari iot connectivity?"
88
+ β†’ intent="unstructured_flow",
89
+ rewritten_query="What are the key features of IoT connectivity?", confidence=0.85
90
 
91
+ User: "How many orders did we get last month?"
92
+ β†’ intent="structured_flow",
93
+ rewritten_query="How many orders did we get last month?", confidence=0.92
94
+
95
+ User: "Top 5 customers by revenue this year"
96
+ β†’ intent="structured_flow",
97
+ rewritten_query="Top 5 customers by revenue this year", confidence=0.93
98
 
99
+ User: "Is there a correlation between discount and units sold?"
100
+ β†’ intent="structured_flow",
101
+ rewritten_query="Is there a correlation between discount and units sold?", confidence=0.9
102
 
103
+ User: "How has monthly revenue trended by region, and what stands out?"
104
+ β†’ intent="structured_flow",
105
+ rewritten_query="How has monthly revenue trended by region this year, and what is unusual?",
106
+ confidence=0.88
107
 
108
  History: assistant: "Pro Plan Annual led at $487,200 in April."
109
  User: "And in March?"
110
+ β†’ intent="structured_flow",
111
+ rewritten_query="What was Pro Plan Annual's revenue in March?", confidence=0.9
112
  ```
113
 
114
  ## Constraints
115
 
116
+ - Pick exactly one `intent`. Do not invent values outside the six listed.
117
+ - Prefer `unstructured_flow` over `structured_flow` only for pure knowledge/document questions; prefer `structured_flow` whenever computation over data is involved.
118
  - Do not refuse β€” refusal happens later in guardrails. Just classify.
119
  - One JSON object as output; no prose, no markdown.