mirosociety/docs/plans/2026-03-17-analyst-report-design.md

# Analyst Report Design

## Problem

The current report shows raw metric floats (0.4, 0.5) that are meaningless to users doing market research or customer sentiment analysis. A user seeing "brand_sentiment: 0.72" cannot act on that. The report needs to tell users what to do, why, and back it with evidence — not just show numbers.

## Goals

1. **Decision support**: Lead with actionable recommendations (go/no-go, segment targeting, risk mitigation)
2. **Evidence gathering**: Back every claim with traceable data (agent quotes, action counts, timeline events)
3. **Emergent insights**: Surface things the user didn't ask about — unexpected patterns, coalition formation, cascade effects
4. **Background generation**: Report starts building immediately when simulation ends, ready when user clicks "View Report"

## Non-Goals

- Interactive post-report chat (future)
- Agent survey/interview (future)
- PDF/PowerPoint export (future, covered in enterprise strategy)
- Real-time streaming of report sections (future)

## Report Structure (Top to Bottom)

### 1. Executive Brief

Colored banner at the top. Green (go), amber (caution), red (rethink).

- **Headline**: One sentence verdict — "This product launch shows strong potential in 2 of 4 segments, with a critical churn risk in price-sensitive demographics"
- **Summary**: One paragraph expanding on the headline
- **Confidence**: High / Medium / Low — derived from simulation length, agent count, and metric volatility
- **Meta**: Agent count, total days, total events in a subtle row

### 2. Scorecard

Replaces the current raw metric bar charts. Grid of metric cards (2-3 columns).

Each card shows:
- **Metric name** (e.g., "Brand Sentiment")
- **Rating pill**: Critical / Weak / Moderate / Strong / Excellent (color-coded)
- **Value**: Scaled to 0-100 (not 0.0-1.0)
- **Trend**: Arrow + delta from start value (e.g., "↑ from 50")
- **Explanation**: One sentence from the LLM explaining *why* it's at this level

Rating thresholds (on 0-100 scale):
- 0-20: Critical (red)
- 21-40: Weak (orange)
- 41-60: Moderate (amber)
- 61-80: Strong (green)
- 81-100: Excellent (emerald)

For inverse metrics (conflict, churn_risk) the scale is reversed — high values = Critical, low = Excellent.

Market metrics (brand_sentiment, purchase_intent, word_of_mouth, churn_risk, adoption_rate) only shown for market simulations. Social metrics (stability, prosperity, trust, freedom, conflict) always shown.

### 3. Key Insights — "Things You Should Know"

3-5 emergent behaviors the user didn't anticipate. Each insight is a card with:
- **Type badge**: Opportunity (blue) / Risk (red) / Surprise (amber)
- **Title**: Short hook
- **Description**: 2-3 sentences explaining what happened and why it matters
- **Evidence tags**: Collapsible — lists agent names, days, action types that support the claim

Examples:
- "An informal boycott coalition formed on Day 12 when 3 price-sensitive agents independently started discouraging purchases"
- "Your product's strongest advocates are not the target demographic — they're from an adjacent segment you didn't design for"

### 4. Segment Deep-Dive

One card per segment (agents grouped by role/faction). Each card contains:
- **Segment name + adoption percentage**
- **Mini funnel**: Horizontal stacked bar showing Aware → Interested → Tried → Adopted → Churned with counts
- **Reaction summary**: 1-2 sentences on how this segment responded
- **Top objection**: Most common reason for rejection/abandonment (pulled from agent internal_thought and speech)
- **Champion profile**: Who in this segment advocated, and a short quote why
- **Representative quote**: A real agent quote from the simulation

### 5. Action Items

Numbered, prioritized list. Each item is a structured card:
- **Priority badge**: High (red) / Medium (amber) / Low (gray)
- **What to do**: Specific, concrete action
- **Why**: Evidence-backed reasoning
- **Expected impact**: What changes if this action is taken

### 6. Risks & Second-Order Effects

Split into two sub-sections:
- **Risks**: Each with severity badge (High/Medium/Low) and description
- **Second-order effects**: Bullet list of downstream consequences

### 7. Evidence Trail (Collapsible)

Initially collapsed behind a "Show Evidence Trail" toggle. Contains the existing report data restructured:
- **Narrative summary** (current "What Happened")
- **Key moments timeline** (current timeline with dot indicators)
- **Factions** (current faction cards)
- **Metric sparklines** (improved from current bar blocks — still simple div-based, but with better visual treatment)
- **The Surprise** (current surprise callout)

## Backend: Multi-Pass Analysis Pipeline

### Current State

`narrator.generate_report()` does:
1. Load world state, metrics history, narratives
2. Build 30-day epochs, summarize each with LLM (N calls)
3. One big LLM call with `REPORT_TRENDS_SYSTEM` prompt to produce everything
4. Attach metrics_history and meta

Problems: One massive LLM call tries to do too much. Raw metric floats passed to frontend. No structured data analysis before LLM.

### New Pipeline

#### Pass 0: Epoch Summarization (existing, unchanged)

Already happens. Epoch summaries are available when report generation starts.

#### Pass 1: Data Aggregation (no LLM — pure computation)

Runs before any LLM calls. Builds structured stats from raw simulation data.

**Per-metric stats:**
- start_value, end_value, min, max
- trend direction (up/down/flat — based on start vs end with > 5% threshold)
- volatility (standard deviation across the history)
- human_label (Critical/Weak/Moderate/Strong/Excellent based on thresholds)
- value_100 (scaled to 0-100)

**Per-agent stats:**
- Action counts by type (PURCHASE: 3, ABANDON: 1, RECOMMEND: 5, etc.)
- Speeches and internal thoughts (for quote extraction)
- Final emotional state, faction membership
- Personality traits (for segment correlation)

**Per-segment stats (agents grouped by role):**
- Aggregate action counts
- Adoption funnel: count of agents who did PURCHASE (adopted), ABANDON (churned), RECOMMEND (advocates), neither (neutral)
- Top speeches/thoughts from this segment

**Inflection points:**
- Rounds where any metric delta exceeded a threshold (e.g., > 0.05 change in a single round)
- Mapped to day numbers for the timeline

#### Pass 2: Metric Interpretation + Insights (1 LLM call)

**Input:** Structured stats from Pass 1 + epoch summaries

**Prompt asks for:**
- Per-metric: one-sentence explanation of why it reached its final value
- 3-5 emergent insights with evidence references (agent names, days, action types)
- Executive verdict: go/caution/rethink + confidence + headline + summary paragraph

**Output format:** Structured JSON matching the `executive_brief`, `scorecard[].explanation`, and `insights[]` fields.

#### Pass 3: Segment Analysis (1 LLM call)

**Input:** Per-segment agent data (actions, quotes, emotional states, personality traits) + structured stats

**Prompt asks for:**
- Per-segment: reaction summary, top objection, champion profile with quote, representative quote
- Cross-segment dynamics (which segments influenced which)

**Output format:** Structured JSON matching `segments[]` fields. Funnel numbers come from Pass 1 computation, not LLM.

#### Pass 4: Action Synthesis (1 LLM call)

**Input:** Everything from Passes 1-3

**Prompt asks for:**
- Prioritized action items: what to do, why, expected impact, priority
- Key risks with severity
- Second-order effects

**Output format:** Structured JSON matching `action_items[]`, `risks[]`, `second_order_effects[]`.

### LLM Cost

Current: N epoch calls + 1 big report call
New: N epoch calls (unchanged) + 3 focused calls
Net change: +2 LLM calls, but each is smaller and more focused → better quality per call.

## Report Data Structure

```python
{
  "executive_brief": {
    "verdict": "go" | "caution" | "rethink",
    "confidence": "high" | "medium" | "low",
    "headline": "str",
    "summary": "str"
  },
  "scorecard": [{
    "metric": "brand_sentiment",
    "label": "Brand Sentiment",
    "rating": "strong",
    "value": 72,
    "start_value": 50,
    "trend": "up" | "down" | "flat",
    "explanation": "str"
  }],
  "insights": [{
    "title": "str",
    "description": "str",
    "evidence": {
      "agents": ["name1", "name2"],
      "days": [8, 12],
      "actions": ["ABANDON", "RECOMMEND"]
    },
    "type": "opportunity" | "risk" | "surprise"
  }],
  "segments": [{
    "name": "str",
    "adoption_pct": 65,
    "funnel": {
      "aware": 20,
      "interested": 15,
      "tried": 12,
      "adopted": 8,
      "churned": 3
    },
    "reaction": "str",
    "top_objection": "str",
    "champion": {"name": "str", "why": "str"},
    "representative_quote": "str"
  }],
  "action_items": [{
    "action": "str",
    "reasoning": "str",
    "expected_impact": "str",
    "priority": "high" | "medium" | "low"
  }],
  "risks": [{"risk": "str", "severity": "high" | "medium" | "low"}],
  "second_order_effects": ["str"],
  "narrative": {
    "summary": "str",
    "key_moments": [{"day": "int", "title": "str", "description": "str"}],
    "surprise": "str",
    "factions": [{"name": "str", "description": "str", "peak_members": "int"}]
  },
  "metrics_history": [{"round": "int", "stability": "float", "...": "..."}],
  "meta": {
    "agent_count": "int",
    "total_days": "int",
    "total_actions": "int",
    "rules": ["str"],
    "world_name": "str"
  }
}
```

## Async Report Generation

### Trigger

When the simulation engine finishes its last round and status transitions to `completed`, immediately spawn a background task to generate the report.

### New Status Field

`report_status` on the simulation record:
- `pending` — simulation not done yet
- `generating` — report pipeline is running
- `ready` — report is stored and available
- `failed` — report generation errored

### Storage

Report JSON stored in the simulation's SQLite database in a new `report` table (single row, `report_json TEXT`).

### API

`GET /simulation/{sim_id}/report`:
- If `report_status == ready`: return the full report JSON
- If `report_status == generating`: return `{"status": "generating"}`
- If `report_status == failed`: return `{"status": "failed", "error": "..."}`
- If `report_status == pending`: return `{"status": "pending"}`

### Frontend Polling

1. User clicks "View Report" → navigates to `/report/:id`
2. Fetch report endpoint
3. If `ready` → render report
4. If `generating` → show skeleton loader with "Analyzing simulation results..." and poll every 3 seconds
5. On transition to `ready` → fade in the report

## Files Changed

### Backend
- `backend/app/services/narrator.py` — Replace `generate_report()` with multi-pass pipeline. Add `ReportAnalyzer` class with Pass 1 computation. Add new LLM prompt templates for Passes 2-4.
- `backend/app/services/engine.py` — After simulation completes, spawn background report generation task.
- `backend/app/db/store.py` — Add `report` table. Add `save_report()`, `get_report()`, `get_report_status()`, `set_report_status()` methods.
- `backend/app/api/routes.py` (or equivalent) — Update `/simulation/{sim_id}/report` endpoint to check status and return accordingly.

### Frontend
- `frontend/src/views/ReportView.vue` — Complete rewrite with new section structure, polling logic, skeleton loader.
- `frontend/src/api/client.js` (or equivalent) — Update `getReport()` to handle status responses and polling.