Use Case Foundry — User Guide
This guide is for practitioners running AI opportunity discovery: innovation leads, transformation teams, consultants, product leaders, and domain experts who need a repeatable, evidence-backed process — not another brainstorm.
For field-by-field builder instructions, see the Builder guide. For methodology and implementation notes, start at Resources.
How to use this guide
- If you are new, read in order: What it does → Getting started → Discovery program.
- If you are actively running sessions, jump to Building the evidence map,
Analyze vs Generate, and Troubleshooting.
- Keep the Builder guide open beside this guide for field-level help.
What Use Case Foundry does
Use Case Foundry is an AI Opportunity Discovery Engine. You build a structured model of a company — offerings, processes, data, skills, pains, constraints — and the engine systematically generates and ranks AI use cases that *that company in particular* is positioned to win.
The output is not a long generic list. It is a short, ranked portfolio with:
- Transparent scoring on Impact · Feasibility · Moat · Quality
- Explicit prerequisites (e.g. instrument data before automating)
- Per-candidate evidence strengthening guidance and interview questions
- A sequenced plan: quick wins plus one strategic bet
- Optional decision-ready dossiers, agent blueprints, pilot packages, and deployable scaffold exports
- A downloadable evidence & provenance map that stakeholders can open offline
Most AI ideation fails because it is generic. Use Case Foundry shifts discovery from open-ended brainstorming to an evidence-based system where every recommendation traces back to facts you captured — and gaps you still need to close.
What good looks like
By the time you are ready to Generate, you should usually see:
- High-severity gaps mostly closed in Improve the model
- At least one high-impact pain linked to each priority workflow
- Accessible data marked for key high-volume processes
- Clear prerequisites and sequencing (not a flat idea list)
- Only the candidates you actually want to take forward checked for Generate
Who this is for
| Role | What you get |
|---|---|
| Innovation / transformation lead | A defensible shortlist to fund, with prerequisites sequenced |
| Consultant or advisor | A structured discovery program you can run across client sessions |
| Functional leader (finance, ops, support, HR, …) | Department-scoped discovery without drowning in the whole company |
| Executive sponsor | Quick wins vs strategic bets, with business case framing when you Generate |
| Domain expert / SME | Contribute facts through interviews and evidence capture — no strategy doc required |
You do not need to understand the engine internals. You do need to treat discovery as a program over days, not a form you finish in one sitting.
Core concepts
Three layers
- Mapping — Turn what you know about a company into a structured evidence graph
(nodes: offerings, segments, accounts, processes, skills, assets, projects, pains, constraints; edges: serves, uses, reuses, bottlenecked_by).
- Generation — 52 operator patterns fire deterministically on graph facts to produce
candidate use cases (Automate, Augment, Physical Automation Adoption, Self-Serve Analytics, Asset Utilization Intelligence, Working Capital Optimization, Margin Defense, Simulation & Digital Twin, Scientific Discovery Acceleration, …).
- Prioritization — Candidates rank on Impact × Feasibility × Moat, with a
quality gate that demotes generic ideas. A Target Selector picks quick wins and one strategic bet, respecting prerequisites.
Hybrid engine
- Deterministic triggers decide *when* a use case should exist and compute base scores.
- LLM reasoning (only on Generate) writes *how* it should be executed for this company.
- Analyze uses no LLM — instant feedback as you build the model.
This separation matters: you can iterate on the evidence map all day without API cost, and you always know which scores came from rules vs model output.
The evidence graph is the product
Facts live in nodes; value lives on edges:
- Offering → serves → Segment — who buys what; unlocks Segment Expansion
- Process → uses → Data asset — grounds Automate/Augment feasibility
- Process → bottlenecked_by → Skill — where expert judgment gates throughput
- Offering → reuses → Project — proven capability you productize
- Pains attached to nodes — the strongest signal on the impact axis
An incomplete graph is expected at first. Dormant operators are a roadmap, not a failing grade.
Ranking axes
Every candidate carries four badges:
| Badge | Question it answers |
|---|---|
| I (Impact) | Does this matter? Driven by linked pains (severity × frequency, optional cost) |
| F (Feasibility) | Can we build and run it? Adjusted by constraints (budget, talent, risk, stack) |
| M (Moat) | Can competitors copy it? Raised by proprietary data, abstractions, scarce expertise |
| Q (Quality) | Is it specific and grounded? Generic candidates are demoted |
Candidates also show dependency lines: do first (prerequisites), unlocks (what they enable), overlaps with (related efforts on the same subgraph).
Tilt ranking with risk appetite (averse / balanced / aggressive) or the feasibility/moat slider — averse favors quick wins; aggressive favors defensible bets.
Getting started
Run the app
```bash pip install -r requirements.txt python -m engine.server --open # marketing site at /, app at /app ```
Hosted deployments serve the same UI. LLM keys are server-side environment variables only. Deployment setup is typically handled by your technical owner.
The three-step journey
The app organizes work into three focused steps. Only one step is visible at a time — use the top Source → Evidence → Review nav (or URLs `#source`, `#evidence`, `#review`, `#outside-in`) to move between them without losing context.
| Step | Name | What you do |
|---|---|---|
| 1 | Source | Load an example, auto-map from sources, or resume a saved model; optionally set Company context |
| 2 | Evidence | Add facts in the guided builder; any substantive model unlocks the Outside-in lane; after the first analysis, add capture or CX evidence only when useful |
| 3 | Review | Approve company-map proposals, run Analyze, read the board, close gaps, choose candidates, then Generate |
You do not need every field before getting value. On Evidence, debounced Analyze (no LLM) keeps the preview fresh: active operators, high-priority gaps, and the leading candidate. The full operator coverage panel and grouped candidates live on Review.
Outside-in is an optional parallel lane (`#outside-in`) rather than a numbered step. It opens after the model has substance and keeps scanning, external-evidence review, demand-hypothesis derivation, accepted ledgers, and account plays on one page. The Review page shows a pending-item link back to the lane while retaining company-map proposals and the opportunity board.
The lane adapts to the business model: scan named customers, channels, competitors, and suppliers; discover association, regulator, and trade-news sources when named accounts are unavailable; or use aggregate consumer-voice and platform-policy sources for B2C companies. Competitor discovery and claims remain review-gated. Accepted evidence can sharpen ranking, fire dedicated demand-gap operators, and support reviewable named-account plays, but it never silently becomes company truth or asserts private customer intent.
Step actions:
- Source → Evidence: Continue to Evidence (enabled once the model has a name or facts)
- Evidence → Review: Analyze & Review board re-runs analysis, then opens Review; Open Review board jumps to the last board without re-analyzing
- Evidence → Outside-in: Open Outside-in lane when the model has substance
- Back buttons return to the previous step; gap Open links on the preview jump back to the exact builder field
Your first session (30–60 minutes)
- Open `/app` and dismiss the welcome screen (or load an example from it).
- In Step 1 · Source, load Meridian Fab or Beacon Pay, then Continue to Evidence.
- In Step 2 · Evidence, use Build your evidence map (+ Offering, + Process, …) or change one field — watch the evidence preview react.
- Click Analyze & Review board (or Open Review board if you already analyzed) — read operator coverage, top candidates, and Improve the model.
- Do not Generate yet. Note the top 3 high-severity gaps; use Open on a preview gap or Open in builder on Review to jump back to Evidence.
That session teaches the loop. Real discovery continues across days and contributors.
The fastest path: load a bundled demo
If you want to see the *end* of the pipeline before building your own model, load the Atlas Ledger Production Demo (finance back office) or Meera Home & Kitchen example, or choose the Beacon Pay Production Demo for a regulated chargeback workflow. All three ship with a pre-built Generate output that already cleared every deterministic gate — pilot validation, mock execution, human pilot, and production handoff — so Analyze & Review board → Generate full plan with Production deployments ticked returns a finished package in seconds, no LLM call required for the bundled output. Open the production deployment card and click Download deployable scaffold (code) to see the real starter files described below before you invest time modeling your own company. Viewing and downloading these demos needs no API key; running an exported agent against real systems still requires your own LLM endpoint, credentials, and technical review.
The discovery program
Honest company models are multi-person, multi-day work. The product is designed for that.
Recommended cadence
| Phase | Actions |
|---|---|
| Day 0 | Auto-map + set archetype + one department lens → first draft. Run Analyze — expect gaps. |
| Each session | Pick one department focus. Close only the top 3 gaps in Improve the model. |
| Between sessions | Save to server (when signed in) or copy the YAML tab to a shared file. |
| Before Generate | Close high-severity gaps: linked pains, accessible data on high-volume processes, project abstractions where they matter. |
The interview loop
Interviews are first-class — they turn assumptions into grounded facts:
``` auto-map → generate interview agenda → conduct interviews → extract/update evidence graph → analyze → fix gaps → repeat ```
Every Analyze returns an Interview Sprint panel:
- Agenda — ask first: highest-leverage questions from current gaps
- Ask in the interview: candidate-specific questions on the opportunities you are validating
- Gap checklist: severity, why it matters, questions to ask
- Status per gap: Open → Captured → Done (tracks multi-day progress)
- Open in builder: jumps to the section that closes the gap
- Conduct interview: opens Map from evidence for that gap
Use Copy agenda as Markdown to take questions into a call, Slack, or email. After applying evidence proposals, the model re-Analyzes automatically.
Consultant interview guide
Consultants often need to extract critical evidence from partial memory and cross-functional context. Use the dedicated Consultant interview guide for:
- Leading/searching prompts by graph section
- Memory-retrieval question funnels
- Edge-completion checklists
- Session templates and anti-patterns
Continuity and saving
| Method | Survives refresh? | Cross-device? |
|---|---|---|
| Browser autosave (in-progress draft) | Yes, same browser | No |
| Save to server (when auth enabled) | Yes | Yes, when signed in |
| YAML tab export | Yes, if you save the file | Yes, via git/Drive/etc. |
Important: if auth is not enabled and you have not exported YAML, treat the browser as your only copy. When signed in, saved models are private to your account.
Context lenses
The company graph is the truth. Context lenses change how it is elicited, ranked, and framed — they never overwrite facts you captured.
Set them in the four-question Company context card on Source. Detailed SME, department, data-readiness, and peer controls stay optional.
| Question | Controls |
|---|---|
| Who is the company? | Archetype + resources |
| What matters this cycle? | Near-term focus + longer-term focus + budget |
| Who reads the output? | Audience |
| Use market context? | Peer lens; detailed lenses under More context lenses |
Archetype
Set under Source → Company context. Shapes section order and empty-state guidance:
| Archetype | Typical starting point |
|---|---|
| Startup | Segments and pains first, then unfair advantage (data, skill, method) |
| Scaleup | Offerings, segments served, data assets behind them |
| Agency | Delivered projects and reusable abstractions |
| Enterprise / SMB | High-volume processes, data produced, where time or money leaks |
Also sets default resource level (capacity to build and run AI).
Department lens
Set under More context lenses. Scopes a session to one function: finance, operations, customer support, HR, sales, marketing, healthcare ops, supply chain, compliance, software/engineering, …
Use this when different contributors own different parts of the company. Each person sees relevant section hints and pain prompts without the full org at once.
Data-maturity lens
Set under More context lenses. Adjusts expectations for data readiness: spreadsheet/manual ops through to ML-native. Influences gap questions and feasibility framing — does not invent data you do not have.
SME lens (e.g. India SME)
Set under More context lenses. Market context for operating realities: spreadsheet-first data, owner-led approval, lean IT, short ROI horizons. Supplies defaults and framing.
Peer lens
Set under Company context → Use market context? Reference company archetypes (not real clients) that suggest which gaps to close first while your graph is thin. Modes: off, auto-match, or manual pick. Never copies another company's graph — only nudges gap ordering and ranking early on.
Strategic focus lens
Set under Company context → What matters this cycle? Near-term and longer-term focus areas re-rank candidates: cost, productivity, innovation, risk, customer experience, … Plus budget direction (flat / slashed / increased — fund quick wins vs bets).
Affects Analyze ranking and Generate framing.
Audience lens
Set under Company context → Who will read the output? Shapes Generate output only (not Analyze scoring): founder, transformation lead, functional VP, agency partner, … Same candidates, different dossier emphasis.
Building the evidence map
On Step 2 · Evidence, the guided builder elicits facts that fire operators. Use Build your evidence map quick-add buttons (+ Offering, + Process, …) to seed sections; each click opens the matching section below.
While you edit, the evidence preview (top of the step) summarizes the latest Analyze run: active operators, high-priority gaps, and the leading candidate. The full operator coverage panel (N/52 firing, green vs dormant with unlock hints) appears on Review after you open the board.
Recommended fill order
Work top to bottom, prioritizing facts that unlock the most operators:
| Order | Section | Why |
|---|---|---|
| 1 | Company | Name + risk appetite set ranking tilt |
| 2 | Offerings | Most external plays start here; add components (`manual`, `rule_based`, `judgment`) |
| 3 | Segments | Link via serves — unlocks Segment Expansion and Customer Success |
| 4 | Assets | Accessible data is the main grounding signal; mark `accessible` only if usable today |
| 5 | Processes | Volume + repetitiveness → Automate; bottleneck skill → Augment; no data → Instrument First |
| 6 | Skills | Scarce skills power Augment, Training, Knowledge Capture |
| 7 | Projects | Abstraction field unlocks Abstraction Transfer (highest moat) |
| 8 | Pain & value interview → Pains | Attach pains to raise Impact |
| 9 | Constraints | Generate Compliance, Integration, Risk Guardrails, Knowledge Capture plays |
Section order may reorder based on archetype. Hover any ⓘ icon for field help.
Detailed section cheat sheet: Builder guide
Pain & value interview
Structured prompts above the raw pains form — e.g. "Where is time wasted?", "Which workflows delay revenue?" — seed graded pains attached to nodes you select. Better pain inputs raise impact and quality measurability.
YAML tab (power users)
The YAML tab remains for direct editing, git workflows, and round-tripping with the builder. Loading an example or auto-mapping populates the builder; edges the builder does not manage are preserved on round-trip.
Switching back from YAML without saving does not re-parse unsaved edits.
Auto-map: draft, not done
Auto-map drafts a starting model from:
- Website URLs (optional same-domain crawl)
- Pasted notes
- Uploaded files (`.txt`, `.md`, `.html`, `.pdf`, `.csv`, `.xlsx`, …), including annual and quarterly reports
It typically captures roughly half to two-thirds of a useful model from public material. It rarely includes linked pains, accessible data flags, bottleneck skills, or project abstractions.
For tabular uploads (`.csv`, `.xlsx`), Auto-map also computes uploaded-file insights before the LLM writes anything. When the file has useful operational signals, the app shows What your uploaded data shows with deterministic findings, a file-grounded AI use case, a week-one pilot shape, and buttons to Open live dashboard, Download dashboard (HTML), and Download one-pager. The first pilot is deliberately scoped to the uploaded file alone; every numeric claim is tied back to computed findings, and you can include that file-grounded use case in Generate when you want a deployment package based only on the upload.
When an uploaded report contains comparable financial periods, Auto-map can also extract a cited financial profile for review. Accepted asset-utilization, working- capital, and margin signals can fire three dedicated opportunity patterns; they remain tied to their report period and source note.
Do not rely on auto-map alone. After import:
- Run Analyze
- Close top gaps in Improve the model
- Add internal evidence auto-map missed
- Re-Analyze until high-severity gaps close
Use Merge into current model when extending an existing draft with new sources. Review review notes after import — inferred fields are flagged for human edit.
Identifiers in pasted notes and uploads are redacted server-side before any LLM call. See Trust and evidence boundary.
Workstation capture (evidence from how work actually happens)
Auto-map reads what a company says about itself. Workstation capture drafts model facts from what employees actually *do* — opt-in, local-first, and review-only. After the first analysis, open Add more evidence → Workstation capture; it turns recurring app/window/SaaS activity into `processes` (with volume and repetitiveness), touched `assets`, `uses` edges between them, and possible scarce `skills` — plus constraint hints from domain words like audit, approval, or HIPAA that a human must confirm.
One-click, local (Foundry and ActivityWatch on the same machine):
- Keep ActivityWatch running in the background.
- Run Analyze once, then open Add more evidence → Workstation capture.
- Click Import from ActivityWatch — the app fetches, summarizes, and opens
the review panel. No export or paste required.
Hosted Foundry (capture agent): run a small tray app on each workstation (`python -m collector.capture_tray`). It keeps ActivityWatch and summarization local and pushes only a redacted summary to your Foundry URL. Signing in shows a Workstation capture tab with two zones — Ready to review (workflow candidates that crossed the graduation gate) and Still watching (candidates still accumulating sessions/days before they earn review attention, which you can dismiss if noisy).
Either path returns patch proposals reviewed in the same panel Auto-map uses — nothing merges into your model without a human accepting it. Denylists for apps, domains, and folders apply before any event is summarized, and raw activity never leaves the employee's machine — only a redacted aggregate summary (or, for the CLI/API path, a fully local one-shot summarize-then-push) does.
See Trust and evidence boundary for the full privacy design.
Map from evidence (private discovery)
When facts live in internal systems (CRM, ERP, ticketing, spreadsheets), users may not want to upload raw records. Map from evidence closes gaps without full data sharing.
Privacy tiers
| Tier | What you share | Typical use |
|---|---|---|
| No-data guide | Local inspection + typed observations | First pass; nothing leaves browser until Extract |
| Metadata only | Field names, objects, stages, report titles | Reveal processes/assets without record contents |
| Aggregate only | Counts, rates, medians, distributions | Surface pains and bottlenecks safely |
| Redacted sample | Anonymized snippets | Ground abstractions and edge cases |
Raw evidence is not persisted. Only human-approved graph patches merge into the model.
Workflow
- Run Analyze → open Improve the model
- Pick a high-severity gap → Map from evidence
- Choose privacy tier; enter observations
- Review graph patch proposals → apply selected changes
- Re-Analyze and repeat
For stricter boundaries (VPC-only LLM), ask your technical owner to run the private-runtime deployment profile.
Analyze vs Generate
| Action | LLM | Output |
|---|---|---|
| Analyze | Not used | Prescores (I·F·M·Q), operator coverage, Improve the model panel, grouped candidates, Interview Sprint |
| Generate full plan | Used | LLM-written use cases and business artifacts for the candidates you selected, plus a sequenced plan (quick wins + strategic bet) |
| Critique & rewrite | Used (with Generate) | Extra pass flagging generic candidates before final ranking |
| Opportunity dossiers | Used (with Generate) | Decision-ready one-pagers: business case, pilot scope, risks, metrics, kill criteria |
| Agent blueprints | Used (with Generate) | Parameterized agent specs + deployment-readiness kits for the selected opportunities, gated by evidence confidence |
| Agent scaffolds | Used (with Generate) | Pilot-ready scaffolds on top of the blueprints: runtime contract, eval suite, readiness gap workflow, and pilot package (auto-enables Agent blueprints) |
| Pilot validations | Used (with Generate) | Mock-eval readiness report on top of the scaffolds: validation manifest, mock connector pack, eval run plan, and a pilot readiness verdict (auto-enables Agent blueprints + scaffolds) |
| Pilot executions | Used (with Generate) | Actually runs the mock eval suite from a ready validation against mock connector stubs: case results, unsafe-action check coverage, a threshold result, and a mock pilot verdict (auto-enables Agent blueprints + scaffolds + Pilot validations) |
| Human pilots | Used (with Generate) | Converts mock-eval-passed executions into a structured human pilot plan: protocol, evidence capture checklist, risk register, completion criteria, and a pilot readiness verdict (auto-enables all prior phases) |
| Production deployments | Used (with Generate) | Converts ready human pilots into a production deployment readiness package: connector inventory, deployment package, operating model, production eval plan, and a handoff readiness verdict (auto-enables all prior phases) |
When to Analyze: constantly — as you edit, after every evidence apply, between interview sessions. It is free and instant.
When to Generate: when high-severity gaps are closed and you need sponsor-ready wording. Generate usually takes a few minutes and requires LLM configuration.
After Analyze, every candidate card has an Include in Generate checkbox. The default is the top five non-generic candidates; use Top 5, All, or Clear in the selection bar to choose the set explicitly. Prerequisites for a selected opportunity are included automatically, and candidates you leave unchecked remain on the board as Analyze only context rather than consuming LLM time.
Use the How far should Generate go? slider to choose the artifact depth: roadmap only, agent blueprint, scaffold, validation, mock execution, human pilot, or production handoff. The advanced checkboxes mirror that depth for teams that want exact stage control.
Without LLM configured, Auto-map and Generate return ready-to-run prompts you can execute elsewhere.
Agent blueprints and deployment readiness
Because the engine already knows which operators fired, it can take the selected opportunities one step further: tick Agent blueprints before Generate to get a parameterized agent spec plus a deployment-readiness kit for each — mission, operating loop, inputs/outputs, tools and connectors, human approval points, then permissions, an eval plan, telemetry, rollout steps, and guardrails.
This is a specification, not an automated deployment. The hard parts of shipping an agent — connectors, permissions, data quality, approval boundaries, and evaluation — are surfaced honestly rather than hidden.
Crucially, graph evidence confidence drives whether an agent is justified, not just whether the opportunity is interesting. Each blueprint carries a readiness status, computed from your facts (not the LLM):
| Status | What it means |
|---|---|
| Ready | Strong evidence: linked pain, accessible data, specific grounding, no unmet prerequisite |
| Needs discovery | Worth building toward, but key evidence (data access, owners, scope) is still missing |
| Prototype only | A product/commercialization idea or thin evidence — explore as a prototype, not a deployed agent |
| Not recommended | The evidence does not justify creating an agent yet (no deployment kit is written) |
A use case can rank highly yet still be only "needs discovery" — high impact does not mean deployment-ready. Close the gaps the blockers list calls out (link a pain, mark data accessible, confirm an approval owner) and the status rises.
Agent scaffolds and the pilot loop
Tick Agent scaffolds to go one step further than the blueprint (this auto-enables Agent blueprints, since scaffolds build on them). For each agent you get four pilot-ready layers:
- Runtime contract — agent instructions, tool contracts, input/output schemas,
a least-privilege permission manifest, environment-variable names, mock connectors so the pilot runs offline, and a human-approval workflow.
- Eval suite — golden cases, synthetic edge cases, a pass/fail rubric,
unsafe-action checks, and the minimum threshold to clear before you trust it.
- Readiness gap workflow — each blocker turned into a concrete ask ("upload 10
redacted tickets", "confirm write vs draft-only access", "name the approval owner") so low readiness becomes an evidence loop, not a dead end.
- Pilot package — a scoped 2-week pilot: users, sample data, rollout steps,
monitoring, kill criteria, security review, sign-off checklist, and before/after metrics.
Each scaffold is classified by type — workflow agent, copilot agent, analytics agent, or prototype brief (for strategy/commercialization plays). These are pilot accelerators, not live deployments: scaffolding plus a validation loop to safely prove an agent before any production rollout. Testing expectation: run the eval suite against the mock connectors and clear the minimum threshold before the pilot, then use the gap workflow to close blockers.
Pilot validations and reading pass/fail readiness
Tick Pilot validations to go one step further than the scaffold (this auto-enables Agent blueprints and Agent scaffolds, since validation builds on both). Instead of just producing pilot material, this checks whether that material is actually complete and safe enough to run a mock pilot with — still never connecting to a real customer system:
- Validation manifest — what is testable right now vs. what is blocked by
missing evidence (e.g. no mock connector named for a tool, no pass/fail rubric for the golden cases).
- Mock connector pack — an offline stub for every declared tool: a
fixture-file name and the expected input/output shape, with write/action tools called out explicitly.
- Eval run plan — the golden cases, edge cases, unsafe-action checks, and
minimum threshold in one runnable plan, with placeholders and malformed entries already stripped out.
- Pilot readiness report — the verdict, in plain language:
| Readiness | What it means | |-----------|---------------| | Ready for mock eval | Tools, mock connectors, golden cases, a rubric, and (if needed) an approval workflow are all in place — run the mock eval suite | | Needs evidence | Something is missing or still open (evidence tasks, a rubric, connector coverage) — close it, then re-run | | Unsafe to pilot | A write/action tool has no approval boundary, or the underlying blueprint itself is not recommended — do not pilot yet |
A `prototype_brief` scaffold gets a prototype checklist instead of a mock-eval verdict — there is no runtime contract to validate, so pilot validation is skipped in favor of the smallest-prototype note.
This verdict is computed from the graph and the scaffold, not the LLM — the optional narrative (pilot summary, top risks, recommended next steps) can explain the verdict, but it cannot change it.
Pilot executions and reading mock-eval verdicts
Tick Pilot executions to go one step further than validation (this auto-enables Agent blueprints, Agent scaffolds, and Pilot validations, since execution builds on all three). Instead of just saying a pilot is ready to run, this actually runs the eval plan — offline, against mock connector stubs, never a real system:
- Execution manifest — whether this validation could actually be run, or
why it was skipped (not `ready_for_mock_eval`, a `prototype_brief`, no cases, or the mock connector pack isn't ready).
- Mock run inputs — the fixture-shaped input used for every case that ran.
- Case results — each golden/edge case scored pass (the mock stub
returned the fixture's declared expected output), fail (no expected output to grade against), or blocked (no connector coverage for that tool).
- Unsafe-action check results — whether every declared unsafe-action
check was actually exercised by at least one case. A check that is declared but never tested by any case shows untested — a real gap in the eval suite, not just a missing checklist item.
- Threshold result — cases passed / total against the declared minimum
pilot threshold (a missing or unreadable threshold conservatively assumes a 100% pass requirement rather than skipping the check).
- Mock pilot verdict, in plain language:
| Verdict | What it means | |---------|---------------| | Mock eval passed | Every executable case passed and the threshold was met | | Mock eval failed | The suite ran, but the pass rate fell short of the threshold | | Blocked | The validation wasn't ready to run, had no cases, or the mock connector pack isn't ready — see the execution manifest | | Unsafe to run | A write/action tool has an unsafe-action check that no case actually exercises, or the underlying validation itself was unsafe |
This phase is deliberately honest about what it can prove offline: a mock connector stub is defined to echo back the fixture-declared expected output, so a "pass" here confirms the scaffold's wiring — tool coverage, fixture completeness, safety-check coverage, threshold discipline — is sound. It does not, and cannot, grade genuine agent reasoning without a live LLM loop against real systems. The optional narrative (execution summary, failure analysis, recommended fixes) can explain the verdict, but it cannot change it.
Human pilots and reading pilot readiness
Tick Human pilots to go one step further than mock execution (auto-enables all prior phases). For each `mock_eval_passed` execution this generates a structured plan a customer technical owner can take into a real pilot:
- Pilot manifest — whether the human pilot is ready to run or what is
blocking it (missing pilot cohort, no sample data requirements, no kill criteria, missing security review checklist, no success metrics, or missing approval owner for write/action tools).
- Pilot protocol — duration, cohort, allowed workflows, human review
steps, escalation path, and success metrics.
- Evidence capture plan — what to collect during real pilot runs: agent
inputs/outputs, human decisions and overrides, unsafe-action attempts, latency, user feedback, and before/after metrics.
- Pilot risk register — residual risks, mitigations, named owners, and
stop conditions.
- Completion criteria — what "done" means for the pilot.
- Human pilot verdict, in plain language:
| Verdict | What it means | |---------|---------------| | Ready for human pilot | All gates cleared — the pilot can be scheduled | | Needs pilot setup | One or more required fields are missing — see the manifest | | Unsafe for human pilot | The underlying mock execution was `unsafe_to_run` — resolve safety issues first |
Production deployments and reading production readiness
Tick Production deployments to go one step further than human pilot planning (auto-enables all prior phases). For each `ready_for_human_pilot` plan this generates a production deployment readiness package:
- Readiness manifest — whether the production handoff is ready or what
is blocking it (incomplete human pilot, missing rollback plan, no monitoring, missing security review, no sign-off checklist, no eval plan, or missing approval owner for write/action tools).
- Deployment package — connector inventory with credential placeholders,
permission manifest, environment variables, rollout stages (shadow → supervised → production), rollback instructions, kill switch, and audit log requirements.
- Operating model — owner matrix for product, technical, security, support,
incident response, eval maintenance, and write-tool approval.
- Production eval plan — regression suite sourced from the mock eval cases,
shadow-mode checks, unsafe-action alert thresholds, and periodic review cadence.
- Production deployment verdict, in plain language:
| Verdict | What it means | |---------|---------------| | Ready for production handoff | All gates cleared — the deployment package can be handed to engineering | | Needs production setup | One or more required fields are missing — see the manifest | | Unsafe for production | The human pilot was `unsafe_for_human_pilot` — resolve safety issues first |
This phase produces a readiness package and gate, not an automated deployment. No credentials are created, no infrastructure is provisioned, and no external systems are called.
For concrete examples of this handoff path (ready, needs setup, unsafe), see [Evidence to deployment examples](../docs/EVIDENCE_TO_DEPLOYMENT_EXAMPLES.md) and `examples/evidence_to_deployment/`.
Each production deployment card also has a Download deployable scaffold (code) button — a zip of real starter files (`agent/runtime.py`, tool adapters, `.env.example`, `Dockerfile`/`docker-compose.yml`, an eval harness, `OPERATING_MODEL.md`, and a handoff README), generated deterministically from that candidate's runtime contract and connector inventory. What ends up inside depends on what each declared tool actually is — this is an honesty boundary, not a single mock skeleton:
| Export tier | What you get |
|---|---|
| Spreadsheet-native | A tool backed by a spreadsheet (e.g. a ticket queue or FAQ sheet) ships the company's real bundled `.xlsx` data plus a real reader/writer — no mock at all for that tool. |
| Typed connector | `rest_api`, `http_webhook`, `sql`, `local_file`, and `smtp` tools get a real, config-only adapter: set the environment variables in `.env.example` and it calls, queries, reads, or sends for real — no code changes. |
| Best-effort API | `bespoke_api` tools get a real HTTP call too, but the one `build_request()` mapping function is marked TODO, since a proprietary API's request/response contract can't be inferred from a company model alone. |
| Mock stub | Anything else falls back to an offline fixture-echoing stub — credentials are placeholders and nothing talks to a real system until you replace it. |
`agent/runtime.py` itself is real, not a placeholder — and its *structure* matches the agent's type, so packages differ in code, not just in the prompt:
- Workflow agents get a queue-processing runtime: batches are ranked and
deduped deterministically in code (bundled triage engine) before any LLM call, then processed per item in priority order.
- Copilot agents get a grounded, interactive runtime: every question is
grounded on the real files under `data/knowledge/` via bundled retrieval code, and citations come from retrieval, not the model.
- Analytics agents compute first and narrate second: bundled engines
produce the numbers in real code, and the LLM only explains them.
- The "AI drafting over spreadsheets" shape gets a real batch runtime that
reads the bundled `.xlsx` rows, calls an LLM to draft each reply, and writes drafts back for human review.
Each package also ships `agent/engines/` — real, stdlib-only implementations selected by the agent's operator (forecast baseline, schedule heuristic, significance testing, safe aggregation, record reconciliation, triage, retrieval) with selftests the bundled eval harness actually executes — and `agent/guards.py`, which enforces the operating model in code: a file-based kill switch, a per-run action budget, and an audit log of executed writes.
It's a scaffold, not a live deployment: a technical owner still supplies real credentials and an LLM key, and verifies any remaining mock or best-effort adapters before running this unattended in production. For the per-operator mapping of what ships and how confidently it delivers, see [Deployable code by operator](../docs/DEPLOYABLE_CODE_BY_OPERATOR.md).
Reading the results
Operator coverage
On Review, after Analyze. Shows N/52 firing. Green = operator matched graph facts. Grey = dormant, with unlock hint (e.g. "Add a project and write its reusable abstraction").
Aim for green operators that match the company's actual leverage — not every box ticked.
Candidate buckets
The review board groups candidates deterministically:
| Bucket | Meaning |
|---|---|
| Prerequisites | Do first — Instrument First, Risk Guardrails, Test Harness, Simulation & Digital Twin, Synthetic Data Generation, … |
| Quick wins | High feasibility + meaningful impact |
| Strategic bets | High moat, plausibly feasible |
| Needs more evidence | Medium quality — gaps still hurt ranking |
| Dropped as generic | Failed quality gate |
Each card also shows Evidence that would strengthen this. Use Add in builder to jump to the section that would improve the candidate, or use the candidate-specific interview question when the missing evidence needs a human answer. After Generate, cards are labeled Full plan generated or Analyze only so you can see which selected candidates received full artifacts.
Evidence & provenance map
From the builder or Review, choose Open evidence map or Download evidence map to create a self-contained HTML exhibit. It connects the reviewed company graph to source classes, cited financial metrics, open evidence gaps, and the ranked opportunities supported by those facts. The file works offline and does not require a Foundry login.
This is a proof-of-process export, not a full audit log: raw evidence, review history, outside-in signal ledgers, and complete candidate lineage are not embedded.
Generate output artifacts
Each generated use case may include:
- why_this_company — defensibility tied to graph facts
- first_experiment — smallest shippable pilot
- required_evidence — what to validate before scaling
- what_would_make_us_drop_this — kill criteria
With Opportunity dossiers enabled, selected quick wins and the strategic bet package into Markdown one-pagers you can copy to slides or memos.
Target Selector plan
The final plan surfaces:
- Quick wins — fund momentum (high impact + feasibility)
- One strategic bet — highest moat that is plausibly feasible
- Sequencing note — prerequisites honored; overlaps flagged
Quick wins and the strategic bet are never collapsed into one recommendation.
Roles in a multi-person program
| Contributor | Contribution | Tool surface |
|---|---|---|
| Program lead | Archetype, focus areas, when to Generate | Source + Evidence + Review |
| Domain SME (finance, ops, …) | Processes, pains, data access | Evidence (department lens + builder) |
| Technical lead | Assets, constraints, stack | Evidence → Assets + Constraints |
| Executive sponsor | Risk appetite, budget direction | Source (Company context) + Review → dossiers |
| Consultant | YAML export, cross-session continuity | Evidence YAML tab + Source saved models |
Assign one department per session. Diagnostics = the sprint backlog — do not ask contributors to complete the whole form before they see value.
More workflow detail: Consultant interview guide
Authentication and saved models
When the server has `AUTH_SECRET` set:
- Sign in / register to use Analyze, Generate, Auto-map, and saved models
- Saved models persist across devices, private to your account
- Marketing pages and the Builder guide remain public
When auth is disabled (typical local dev), all features work without login but models live in browser storage only.
Privacy and data handling
- Pasted notes, uploads, and evidence snippets are redacted before LLM calls
- Raw uploads are never persisted on the server
- Public website crawl content is treated as public (not redacted)
- Only human-approved graph facts enter the company model
- Workstation capture is opt-in; raw activity stays on the employee's
machine and only a redacted summary (or reviewable patch proposals) reaches Foundry
Full policy: Trust and evidence boundary
Troubleshooting
| Symptom | Likely cause | What to do |
|---|---|---|
| All candidates feel generic | Thin model — no linked pains, no accessible data | Close top gaps; add pains to high-volume processes |
| Many dormant operators | Missing edges or section facts | Read unlock hints; see fill order |
| Auto-map returned sparse model | Public sources lack operational detail | Merge internal docs; use Map from evidence |
| Generate fails or hangs | LLM not configured or timeout | Check `/api/status`; verify provider setup with your technical owner |
| Generate says nothing is selected | All candidate checkboxes are cleared | Pick specific cards, or use Top 5 / All in the selection bar |
| Workstation capture shows no ActivityWatch events | Permissions not granted, or app not running | Confirm ActivityWatch is running and (macOS) Accessibility/Screen Recording permissions are granted, then retry |
| Rankings shift when I edit | Expected — Analyze re-runs on every edit | Use risk appetite / slider to stabilize tilt |
| Lost work after refresh | Not saved | Save to server or export YAML |
| YAML edits not reflected | Switched tabs without applying | Return to YAML tab and ensure content is saved |
Example companies
The app ships contrasting examples you can load to learn the model:
| Example | Profile |
|---|---|
| Meridian Fab | Custom fabrication; physical ops, scarce estimator judgment |
| Lumen Consulting | Data/analytics consultancy; services, regulated clients |
| Beacon Pay | Regulated fintech; data-rich, risk-averse |
| Atlas Ledger | Finance back office; month-end close, reconciliation |
| Harbor Link 3PL | Logistics; supplier delays, stockouts |
| Cascade Support | B2B SaaS support; ticket triage, SLA recovery |
| Nova Growth | Marketing ops; campaign lifecycle, attribution |
| Atlas Ledger Production Demo | Bundled demo — AP invoice matching, pre-cleared through production handoff |
| Meera Home & Kitchen | Bundled demo — spreadsheet-native support triage with a real LLM-backed runtime |
| Beacon Pay Production Demo | Bundled demo — regulated chargeback evidence drafting with adjudicator approval gates |
Load one, change two fields, and watch the evidence preview react before modeling your own company. The three bundled deployment demos above are pre-graded: Generate returns a finished, gate-cleared output instantly instead of calling an LLM.
Further reading
| Document | Audience | Contents |
|---|---|---|
| Builder guide | Anyone using the builder | Field reference, fill order, interview sprint, Analyze vs Generate |
| Consultant interview guide | Consultants and interview leads | Leading/searching prompts to elicit graph-quality evidence |
| Methodology | Reviewers and sponsors | How ranking and evidence-based discovery work |
| Resources | Program leads and operators | Supporting material, examples, and implementation context |
| FAQ | New users | Common product and workflow questions |
Quick reference card
```
- Source → example | auto-map | optional Company context
- Evidence → add facts → operators green → pains linked | Outside-in
- Review → approve proposals → Analyze → close top 3 gaps → repeat
- Generate → when gaps closed → optional critique + dossiers
- More input → capture | CX (after first Analyze)
- Save → server or YAML between sessions
```
Loop: `add facts → analyze → interview → evidence → analyze → … → generate`
Remember: dormant operators = roadmap. Generic candidates = thin evidence. Defensible bets appear when the graph reflects how the business actually operates.