از پیام مشتری تا پاسخ هوشمند؛
ساخت یک تیم AI Agent

Read docs/PROJECT_SPEC.md. It defines a multi-agent CRM customer-success system.
Do not write application code in this stage.

PRINCIPLES (apply to every stage):
- Simple to complex: augmented LLM (LLM + tools + structured output) → fixed workflow → agent
  loop. Add complexity only when needed and document why.
- Per stage: Explore → Plan → wait for approval → Implement → Test → Commit (tag stage-XX).
- Numbers come from SQL/Python. The LLM only reasons and decides.

STACK:
- Backend: Python 3.11+, uv, SQLite, FastAPI, pydantic v2, pytest, typer, rich.
- LLM: AvalAI (OpenAI-compatible), called through the openai SDK with a custom base_url.
- Email: Resend. Frontend: minimal Next.js (TypeScript) app in /web, for approvals only.
- Data: the provided dataset in data/ (spec section 2). Never generate or modify it.

TASKS:
1. Summarize the spec in 10 bullets or fewer.
2. Create the structure:
   backend/src/crm_agents/{db, tools, analytics, llm, agents, orchestrator,
   execution, approval, evaluation, tracing, api}/
   backend/tests/, backend/scripts/, docs/, web/ (README only)
   data/ already contains crm.sqlite and ground_truth.json; leave both untouched.
3. Create AGENTS.md with these rules:
   - Readable code: clear names, docstrings, short functions, type hints.
   - Every agent input and output is a pydantic model.
   - Agents access data only through tools/. Only evaluation/ may read ground_truth.json.
   - data/crm.sqlite is read-only; the app reads and writes a working copy (DB_PATH).
   - Only llm/client.py imports openai. Only execution/email.py imports resend.

   - Every stage ships with tests. Never start the next stage without approval.
4. Create docs/ARCHITECTURE.md (agents, responsibilities, allowed tools, schemas, authority
   limits) and docs/STAGES.md (per stage: goal, deliverables, "why an agent, not a rule").
5. Create .env.example:
     AVALAI_API_KEY=
     AVALAI_BASE_URL=https://api.avalai.ir/v1
     LLM_MODEL=
     LLM_MODEL_CHEAP=
     RESEND_API_KEY=
     RESEND_FROM=
     EMAIL_MODE=dry_run        # dry_run | redirect | live
     EMAIL_REDIRECT_TO=        # redirect mode sends every email here
     AUTONOMY_LEVEL=1          # 1..4
     DB_SOURCE=data/crm.sqlite
     DB_PATH=data/crm_work.sqlite
     GROUND_TRUTH_PATH=data/ground_truth.json
     API_URL=http://localhost:8000
6. Create backend/pyproject.toml and llm/client.py: chat, tool calling, structured output
   parsed into pydantic (one repair retry on invalid JSON), network retries, and logging of
   model, tokens, and latency. Add a smoke test that is skipped without AVALAI_API_KEY.
7. Propose the plan for Stage 1.

Commit "stage-00: setup and design". Stop and wait.

Stage 1: load and understand the provided dataset. Read AGENTS.md first.
The dataset already exists. Do not generate or modify data.

INPUTS:
- data/crm.sqlite: 9 CRM tables (docs/PROJECT_SPEC.md, section 2).
- data/ground_truth.json: hidden story per customer. Evaluation only.

TASKS:
1. scripts/reset_db.py: copy DB_SOURCE to DB_PATH. Every later stage uses only DB_PATH.
2. db/connection.py: open DB_PATH with foreign keys on and rows returned as dicts.
3. db/models.py: pydantic models for the 9 tables, matching the actual columns.
4. docs/DATA.md, generated from the database itself: tables, columns, row counts, date range,
   allowed values of status/category/sentiment fields, and table relationships.
5. scripts/describe_data.py: per customer and month: revenue, orders, tickets by category,
   interactions, negative-sentiment share, overdue invoices. Print a compact table.
6. evaluation/ground_truth.py: the only loader of ground_truth.json.

EXPLORE (write in docs/STAGES.md): pick 3 customers with different patterns and describe what
the numbers suggest, without looking at the ground truth. Then compare with it.

TESTS: row counts match docs/DATA.md, foreign keys are valid, reset_db is repeatable,
no module outside evaluation/ references ground_truth.json.

Commit "stage-01: dataset". Stop and wait.

Stage 2: tool layer and deterministic analytics. Read AGENTS.md first.

Use the real column names from docs/DATA.md.

READ TOOLS (tools/read.py), each returning pydantic models with capped result sizes:
get_customer, get_contacts, get_orders(customer_id, start, end),
get_invoices(customer_id, status), get_interactions(customer_id, start, end),
get_support_tickets(customer_id, status), get_opportunities, get_tasks,
get_business_events(customer_id).

ANALYTICS (analytics/), pure SQL/Python:
monthly_revenue, revenue_trend, order_frequency, engagement, support_pressure (by category),
payment_risk (overdue count and amount), sentiment_trend, compare_periods.
Each returns numbers plus a one-line plain-language summary.

REGISTRY (tools/registry.py): every tool has a name, description, JSON schema generated from
its pydantic input, and a Python function. The registry is the only thing the LLM sees.

GUARD: no tool can read ground_truth.json. Add a test that proves it.

TESTS: every tool and metric on known customers from the dataset.

Commit "stage-02: tools and analytics". Stop and wait.

Stage 3: a single agent with tools (augmented LLM). Read AGENTS.md first.

Build agents/customer_analyst.py that answers "What is happening with customer X?"

- Loop: the LLM picks tools from the registry, tools run, results return, repeat until a
  final answer or 8 steps maximum.
- Output (pydantic): summary, key_metrics, evidence[], hypotheses[{cause, supporting,
  contradicting, confidence}], open_questions.
- System prompt in agents/prompts/customer_analyst.md: role, tool list, "fetch only what you
  need", "use only numbers returned by tools".
- CLI: crm ask <customer_id>
- Log every step (tool, arguments, tokens, latency) as JSONL in tracing/.

EXPERIMENT (write results in docs/STAGES.md): one prompt with the full customer dump vs the
tool-based agent, on 3 customers. Compare tokens and answer quality.

TESTS: tool loop with a mocked LLM; one live test skipped without an API key.

Commit "stage-03: single agent". Stop and wait.

Stage 4: event-driven trigger. Read AGENTS.md first.

Build agents/event_detector.py in two parts:
1. Deterministic scan (analytics only): flag candidate customers with a trigger reason,
   e.g. revenue drop + support pressure, overdue invoices, growing engagement and orders.
2. Cheap LLM triage (LLM_MODEL_CHEAP): choose event_type from CUSTOMER_HEALTH_CHANGED,
   PAYMENT_RISK_DETECTED, EXPANSION_SIGNAL, NO_ACTION, plus severity and trigger.
   It does not investigate.

Output a BusinessEvent model and store it in business_events with source 'scan_rule' or
'llm_triage'. Existing rows with source 'crm_alert' are the CRM's naive alerts; report where
the detector agrees and disagrees with them. CLI: crm scan

Document why detection is mostly rules and only triage uses an LLM.

TESTS: known archetypes produce the expected event types; healthy customers are mostly
NO_ACTION; seasonal_dip is not flagged as high severity.

Commit "stage-04: event detection". Stop and wait.

Stage 5: multi-agent pipeline with an orchestrator. Read AGENTS.md first.

AGENTS (each has its own prompt file, allowed tools, and pydantic input/output):
- ResearchAgent(BusinessEvent) → EvidencePack. Picks tools by event type
  (e.g. payment risk: invoices and orders first).
- AnalystAgent(EvidencePack) → Diagnosis: competing hypotheses, evidence for and against,
  primary cause, confidence. Analytics tools only.
- CustomerHealthAgent(Diagnosis + metrics) → HealthAssessment: state from HEALTHY, GROWING,
  EXPANSION_OPPORTUNITY, AT_RISK, PAYMENT_RISK, RECOVERING; temporary vs structural; rationale.
- NextBestActionAgent(HealthAssessment) → ActionProposal: action_type, priority, owner,
  task_summary, optional customer message draft, risk_level.

ORCHESTRATOR (orchestrator/pipeline.py):
- Fixed chain: event → research → analyst → health → next best action (prompt chaining).
- Routing: NO_ACTION stops early.
- Validate every output; retry once on invalid output, then fail the run with a clear error.
- Store each run with all intermediate outputs in a runs table, created in the working DB
  by db/migrations.py.
- CLI: crm run <customer_id> and crm run --all-events

TESTS: full pipeline with a mocked LLM, schema validation, early stop.

Commit "stage-05: multi-agent pipeline". Stop and wait.

Stage 6: execution, human approval, and autonomy levels. Read AGENTS.md first.

EXECUTION (execution/): create_task, update_task, create_note, update_customer_status,
prepare_customer_message, send_email. New tables (notes, messages, actions) go in
db/migrations.py and are created only in the working DB.
execution/email.py uses Resend according to EMAIL_MODE:
- dry_run: log only
- redirect: send every email to EMAIL_REDIRECT_TO
- live: send to the contact

POLICY (approval/policy.py), by AUTONOMY_LEVEL:
1 recommend only; 2 auto-create internal tasks and notes; 3 customer messages need approval;
4 low-risk actions run automatically, high-risk actions still need approval.
Every action has a status: proposed → approved/rejected → executed/failed, with who and when.

API (api/, FastAPI):
GET /runs, GET /runs/{id}, GET /approvals/pending,
POST /approvals/{id} {decision, edited_message?}, POST /scan, POST /run/{customer_id}

WEB (web/, Next.js + TypeScript, minimal, no auth):
- Pending approvals list.
- Detail view: event, diagnosis, health state, proposed action, editable email draft,
  Approve and Reject buttons.
- Run history.

TESTS: policy per level, email modes with Resend mocked, approval state transitions, API.

Commit "stage-06: execution and approval". Stop and wait.

Stage 7: evaluation against ground truth. Read AGENTS.md first.

evaluation/ is the only module that reads ground_truth.json (via evaluation/ground_truth.py).
It resets the working DB, then runs the pipeline on all customers.

METRICS:
- Event detection: precision and recall of flagged customers.
- Diagnosis: LLM-as-judge (separate prompt) scores whether the primary cause matches the true
  archetype in meaning, not exact wording; store the judge's rationale.
- Health state: accuracy and confusion matrix.
- Action: appropriateness against a rule table per archetype.
- Evidence: every cited number exists in the tool outputs of that run.
- Efficiency: tool calls, tokens, and latency per run.
- False positives on healthy and seasonal_dip customers.

OUTPUT: evaluation/report.md and a JSON file. CLI: crm eval
Support comparing two evaluation runs (e.g. prompt v1 vs v2).

TESTS: unit tests for every metric function.

Commit "stage-07: evaluation". Stop and wait.

Stage 8: tracing and end-to-end run. Read AGENTS.md first.

- tracing/: one trace per run with every agent step (input, tool calls, output, tokens,
  latency). Add a page in web/ that shows the chain as a timeline.
- scripts/end_to_end.py: reset the working DB → scan → pick one AT_RISK
  service_failure customer → run the pipeline → wait for approval in the web UI → execute with
  EMAIL_MODE=redirect → print the updated CRM records and the next event.
- README.md: setup, environment variables, commands per stage, architecture diagram
  (mermaid), known limitations.
- Final check: all tests pass; no forbidden imports; no ground_truth.json access outside
  evaluation/; data/crm.sqlite unchanged.

Commit "stage-08: tracing and end-to-end run". Stop and wait.