← Back to app
User guide

Run evidence-based AI discovery from first session to decision-ready plan.

The complete reference for practitioners: core concepts, the three-step workflow, context lenses, the interview loop, reading ranked results, and multi-day discovery programs.

3 workflow steps
52 operator patterns
4 ranking axes

Use Case Foundry — User Guide

This guide is for practitioners running AI opportunity discovery: innovation leads, transformation teams, consultants, product leaders, and domain experts who need a repeatable, evidence-backed process — not another brainstorm.

For field-by-field builder instructions, see the Builder guide. For methodology and implementation notes, start at Resources.

How to use this guide

  • If you are new, read in order: What it does → Getting started → Discovery program.
  • If you are actively running sessions, jump to Building the evidence map,

Analyze vs Generate, and Troubleshooting.

  • Keep the Builder guide open beside this guide for field-level help.

What Use Case Foundry does

Use Case Foundry is an AI Opportunity Discovery Engine. You build a structured model of a company — offerings, processes, data, skills, pains, constraints — and the engine systematically generates and ranks AI use cases that *that company in particular* is positioned to win.

The output is not a long generic list. It is a short, ranked portfolio with:

  • Transparent scoring on Impact · Feasibility · Moat · Quality
  • Explicit prerequisites (e.g. instrument data before automating)
  • Per-candidate evidence strengthening guidance and interview questions
  • A sequenced plan: quick wins plus one strategic bet
  • Optional decision-ready dossiers, agent blueprints, pilot packages, and deployable scaffold exports
  • A downloadable evidence & provenance map that stakeholders can open offline

Most AI ideation fails because it is generic. Use Case Foundry shifts discovery from open-ended brainstorming to an evidence-based system where every recommendation traces back to facts you captured — and gaps you still need to close.

What good looks like

By the time you are ready to Generate, you should usually see:

  • High-severity gaps mostly closed in Improve the model
  • At least one high-impact pain linked to each priority workflow
  • Accessible data marked for key high-volume processes
  • Clear prerequisites and sequencing (not a flat idea list)
  • Only the candidates you actually want to take forward checked for Generate

Who this is for

RoleWhat you get
Innovation / transformation leadA defensible shortlist to fund, with prerequisites sequenced
Consultant or advisorA structured discovery program you can run across client sessions
Functional leader (finance, ops, support, HR, …)Department-scoped discovery without drowning in the whole company
Executive sponsorQuick wins vs strategic bets, with business case framing when you Generate
Domain expert / SMEContribute facts through interviews and evidence capture — no strategy doc required

You do not need to understand the engine internals. You do need to treat discovery as a program over days, not a form you finish in one sitting.

Core concepts

Three layers

  1. Mapping — Turn what you know about a company into a structured evidence graph

(nodes: offerings, segments, accounts, processes, skills, assets, projects, pains, constraints; edges: serves, uses, reuses, bottlenecked_by).

  1. Generation52 operator patterns fire deterministically on graph facts to produce

candidate use cases (Automate, Augment, Physical Automation Adoption, Self-Serve Analytics, Asset Utilization Intelligence, Working Capital Optimization, Margin Defense, Simulation & Digital Twin, Scientific Discovery Acceleration, …).

  1. Prioritization — Candidates rank on Impact × Feasibility × Moat, with a

quality gate that demotes generic ideas. A Target Selector picks quick wins and one strategic bet, respecting prerequisites.

Hybrid engine

  • Deterministic triggers decide *when* a use case should exist and compute base scores.
  • LLM reasoning (only on Generate) writes *how* it should be executed for this company.
  • Analyze uses no LLM — instant feedback as you build the model.

This separation matters: you can iterate on the evidence map all day without API cost, and you always know which scores came from rules vs model output.

The evidence graph is the product

Facts live in nodes; value lives on edges:

  • Offering → serves → Segment — who buys what; unlocks Segment Expansion
  • Process → uses → Data asset — grounds Automate/Augment feasibility
  • Process → bottlenecked_by → Skill — where expert judgment gates throughput
  • Offering → reuses → Project — proven capability you productize
  • Pains attached to nodes — the strongest signal on the impact axis

An incomplete graph is expected at first. Dormant operators are a roadmap, not a failing grade.

Ranking axes

Every candidate carries four badges:

BadgeQuestion it answers
I (Impact)Does this matter? Driven by linked pains (severity × frequency, optional cost)
F (Feasibility)Can we build and run it? Adjusted by constraints (budget, talent, risk, stack)
M (Moat)Can competitors copy it? Raised by proprietary data, abstractions, scarce expertise
Q (Quality)Is it specific and grounded? Generic candidates are demoted

Candidates also show dependency lines: do first (prerequisites), unlocks (what they enable), overlaps with (related efforts on the same subgraph).

Tilt ranking with risk appetite (averse / balanced / aggressive) or the feasibility/moat slider — averse favors quick wins; aggressive favors defensible bets.

Getting started

Run the app

```bash pip install -r requirements.txt python -m engine.server --open # marketing site at /, app at /app ```

Hosted deployments serve the same UI. LLM keys are server-side environment variables only. Deployment setup is typically handled by your technical owner.

The three-step journey

The app organizes work into three focused steps. Only one step is visible at a time — use the top Source → Evidence → Review nav (or URLs `#source`, `#evidence`, `#review`, `#outside-in`) to move between them without losing context.

StepNameWhat you do
1SourceLoad an example, auto-map from sources, or resume a saved model; optionally set Company context
2EvidenceAdd facts in the guided builder; any substantive model unlocks the Outside-in lane; after the first analysis, add capture or CX evidence only when useful
3ReviewApprove company-map proposals, run Analyze, read the board, close gaps, choose candidates, then Generate

You do not need every field before getting value. On Evidence, debounced Analyze (no LLM) keeps the preview fresh: active operators, high-priority gaps, and the leading candidate. The full operator coverage panel and grouped candidates live on Review.

Outside-in is an optional parallel lane (`#outside-in`) rather than a numbered step. It opens after the model has substance and keeps scanning, external-evidence review, demand-hypothesis derivation, accepted ledgers, and account plays on one page. The Review page shows a pending-item link back to the lane while retaining company-map proposals and the opportunity board.

The lane adapts to the business model: scan named customers, channels, competitors, and suppliers; discover association, regulator, and trade-news sources when named accounts are unavailable; or use aggregate consumer-voice and platform-policy sources for B2C companies. Competitor discovery and claims remain review-gated. Accepted evidence can sharpen ranking, fire dedicated demand-gap operators, and support reviewable named-account plays, but it never silently becomes company truth or asserts private customer intent.

Step actions:

  • Source → Evidence: Continue to Evidence (enabled once the model has a name or facts)
  • Evidence → Review: Analyze & Review board re-runs analysis, then opens Review; Open Review board jumps to the last board without re-analyzing
  • Evidence → Outside-in: Open Outside-in lane when the model has substance
  • Back buttons return to the previous step; gap Open links on the preview jump back to the exact builder field

Your first session (30–60 minutes)

  1. Open `/app` and dismiss the welcome screen (or load an example from it).
  2. In Step 1 · Source, load Meridian Fab or Beacon Pay, then Continue to Evidence.
  3. In Step 2 · Evidence, use Build your evidence map (+ Offering, + Process, …) or change one field — watch the evidence preview react.
  4. Click Analyze & Review board (or Open Review board if you already analyzed) — read operator coverage, top candidates, and Improve the model.
  5. Do not Generate yet. Note the top 3 high-severity gaps; use Open on a preview gap or Open in builder on Review to jump back to Evidence.

That session teaches the loop. Real discovery continues across days and contributors.

The fastest path: load a bundled demo

If you want to see the *end* of the pipeline before building your own model, load the Atlas Ledger Production Demo (finance back office) or Meera Home & Kitchen example, or choose the Beacon Pay Production Demo for a regulated chargeback workflow. All three ship with a pre-built Generate output that already cleared every deterministic gate — pilot validation, mock execution, human pilot, and production handoff — so Analyze & Review board → Generate full plan with Production deployments ticked returns a finished package in seconds, no LLM call required for the bundled output. Open the production deployment card and click Download deployable scaffold (code) to see the real starter files described below before you invest time modeling your own company. Viewing and downloading these demos needs no API key; running an exported agent against real systems still requires your own LLM endpoint, credentials, and technical review.

The discovery program

Honest company models are multi-person, multi-day work. The product is designed for that.

PhaseActions
Day 0Auto-map + set archetype + one department lens → first draft. Run Analyze — expect gaps.
Each sessionPick one department focus. Close only the top 3 gaps in Improve the model.
Between sessionsSave to server (when signed in) or copy the YAML tab to a shared file.
Before GenerateClose high-severity gaps: linked pains, accessible data on high-volume processes, project abstractions where they matter.

The interview loop

Interviews are first-class — they turn assumptions into grounded facts:

``` auto-map → generate interview agenda → conduct interviews → extract/update evidence graph → analyze → fix gaps → repeat ```

Every Analyze returns an Interview Sprint panel:

  • Agenda — ask first: highest-leverage questions from current gaps
  • Ask in the interview: candidate-specific questions on the opportunities you are validating
  • Gap checklist: severity, why it matters, questions to ask
  • Status per gap: Open → Captured → Done (tracks multi-day progress)
  • Open in builder: jumps to the section that closes the gap
  • Conduct interview: opens Map from evidence for that gap

Use Copy agenda as Markdown to take questions into a call, Slack, or email. After applying evidence proposals, the model re-Analyzes automatically.

Consultant interview guide

Consultants often need to extract critical evidence from partial memory and cross-functional context. Use the dedicated Consultant interview guide for:

  • Leading/searching prompts by graph section
  • Memory-retrieval question funnels
  • Edge-completion checklists
  • Session templates and anti-patterns

Continuity and saving

MethodSurvives refresh?Cross-device?
Browser autosave (in-progress draft)Yes, same browserNo
Save to server (when auth enabled)YesYes, when signed in
YAML tab exportYes, if you save the fileYes, via git/Drive/etc.

Important: if auth is not enabled and you have not exported YAML, treat the browser as your only copy. When signed in, saved models are private to your account.

Context lenses

The company graph is the truth. Context lenses change how it is elicited, ranked, and framed — they never overwrite facts you captured.

Set them in the four-question Company context card on Source. Detailed SME, department, data-readiness, and peer controls stay optional.

QuestionControls
Who is the company?Archetype + resources
What matters this cycle?Near-term focus + longer-term focus + budget
Who reads the output?Audience
Use market context?Peer lens; detailed lenses under More context lenses

Archetype

Set under Source → Company context. Shapes section order and empty-state guidance:

ArchetypeTypical starting point
StartupSegments and pains first, then unfair advantage (data, skill, method)
ScaleupOfferings, segments served, data assets behind them
AgencyDelivered projects and reusable abstractions
Enterprise / SMBHigh-volume processes, data produced, where time or money leaks

Also sets default resource level (capacity to build and run AI).

Department lens

Set under More context lenses. Scopes a session to one function: finance, operations, customer support, HR, sales, marketing, healthcare ops, supply chain, compliance, software/engineering, …

Use this when different contributors own different parts of the company. Each person sees relevant section hints and pain prompts without the full org at once.

Data-maturity lens

Set under More context lenses. Adjusts expectations for data readiness: spreadsheet/manual ops through to ML-native. Influences gap questions and feasibility framing — does not invent data you do not have.

SME lens (e.g. India SME)

Set under More context lenses. Market context for operating realities: spreadsheet-first data, owner-led approval, lean IT, short ROI horizons. Supplies defaults and framing.

Peer lens

Set under Company context → Use market context? Reference company archetypes (not real clients) that suggest which gaps to close first while your graph is thin. Modes: off, auto-match, or manual pick. Never copies another company's graph — only nudges gap ordering and ranking early on.

Strategic focus lens

Set under Company context → What matters this cycle? Near-term and longer-term focus areas re-rank candidates: cost, productivity, innovation, risk, customer experience, … Plus budget direction (flat / slashed / increased — fund quick wins vs bets).

Affects Analyze ranking and Generate framing.

Audience lens

Set under Company context → Who will read the output? Shapes Generate output only (not Analyze scoring): founder, transformation lead, functional VP, agency partner, … Same candidates, different dossier emphasis.

Building the evidence map

On Step 2 · Evidence, the guided builder elicits facts that fire operators. Use Build your evidence map quick-add buttons (+ Offering, + Process, …) to seed sections; each click opens the matching section below.

While you edit, the evidence preview (top of the step) summarizes the latest Analyze run: active operators, high-priority gaps, and the leading candidate. The full operator coverage panel (N/52 firing, green vs dormant with unlock hints) appears on Review after you open the board.

Work top to bottom, prioritizing facts that unlock the most operators:

OrderSectionWhy
1CompanyName + risk appetite set ranking tilt
2OfferingsMost external plays start here; add components (`manual`, `rule_based`, `judgment`)
3SegmentsLink via serves — unlocks Segment Expansion and Customer Success
4AssetsAccessible data is the main grounding signal; mark `accessible` only if usable today
5ProcessesVolume + repetitiveness → Automate; bottleneck skill → Augment; no data → Instrument First
6SkillsScarce skills power Augment, Training, Knowledge Capture
7ProjectsAbstraction field unlocks Abstraction Transfer (highest moat)
8Pain & value interview → PainsAttach pains to raise Impact
9ConstraintsGenerate Compliance, Integration, Risk Guardrails, Knowledge Capture plays

Section order may reorder based on archetype. Hover any icon for field help.

Detailed section cheat sheet: Builder guide

Pain & value interview

Structured prompts above the raw pains form — e.g. "Where is time wasted?", "Which workflows delay revenue?" — seed graded pains attached to nodes you select. Better pain inputs raise impact and quality measurability.

YAML tab (power users)

The YAML tab remains for direct editing, git workflows, and round-tripping with the builder. Loading an example or auto-mapping populates the builder; edges the builder does not manage are preserved on round-trip.

Switching back from YAML without saving does not re-parse unsaved edits.

Auto-map: draft, not done

Auto-map drafts a starting model from:

  • Website URLs (optional same-domain crawl)
  • Pasted notes
  • Uploaded files (`.txt`, `.md`, `.html`, `.pdf`, `.csv`, `.xlsx`, …), including annual and quarterly reports

It typically captures roughly half to two-thirds of a useful model from public material. It rarely includes linked pains, accessible data flags, bottleneck skills, or project abstractions.

For tabular uploads (`.csv`, `.xlsx`), Auto-map also computes uploaded-file insights before the LLM writes anything. When the file has useful operational signals, the app shows What your uploaded data shows with deterministic findings, a file-grounded AI use case, a week-one pilot shape, and buttons to Open live dashboard, Download dashboard (HTML), and Download one-pager. The first pilot is deliberately scoped to the uploaded file alone; every numeric claim is tied back to computed findings, and you can include that file-grounded use case in Generate when you want a deployment package based only on the upload.

When an uploaded report contains comparable financial periods, Auto-map can also extract a cited financial profile for review. Accepted asset-utilization, working- capital, and margin signals can fire three dedicated opportunity patterns; they remain tied to their report period and source note.

Do not rely on auto-map alone. After import:

  1. Run Analyze
  2. Close top gaps in Improve the model
  3. Add internal evidence auto-map missed
  4. Re-Analyze until high-severity gaps close

Use Merge into current model when extending an existing draft with new sources. Review review notes after import — inferred fields are flagged for human edit.

Identifiers in pasted notes and uploads are redacted server-side before any LLM call. See Trust and evidence boundary.

Workstation capture (evidence from how work actually happens)

Auto-map reads what a company says about itself. Workstation capture drafts model facts from what employees actually *do* — opt-in, local-first, and review-only. After the first analysis, open Add more evidence → Workstation capture; it turns recurring app/window/SaaS activity into `processes` (with volume and repetitiveness), touched `assets`, `uses` edges between them, and possible scarce `skills` — plus constraint hints from domain words like audit, approval, or HIPAA that a human must confirm.

One-click, local (Foundry and ActivityWatch on the same machine):

  1. Keep ActivityWatch running in the background.
  2. Run Analyze once, then open Add more evidence → Workstation capture.
  3. Click Import from ActivityWatch — the app fetches, summarizes, and opens

the review panel. No export or paste required.

Hosted Foundry (capture agent): run a small tray app on each workstation (`python -m collector.capture_tray`). It keeps ActivityWatch and summarization local and pushes only a redacted summary to your Foundry URL. Signing in shows a Workstation capture tab with two zones — Ready to review (workflow candidates that crossed the graduation gate) and Still watching (candidates still accumulating sessions/days before they earn review attention, which you can dismiss if noisy).

Either path returns patch proposals reviewed in the same panel Auto-map uses — nothing merges into your model without a human accepting it. Denylists for apps, domains, and folders apply before any event is summarized, and raw activity never leaves the employee's machine — only a redacted aggregate summary (or, for the CLI/API path, a fully local one-shot summarize-then-push) does.

See Trust and evidence boundary for the full privacy design.

Map from evidence (private discovery)

When facts live in internal systems (CRM, ERP, ticketing, spreadsheets), users may not want to upload raw records. Map from evidence closes gaps without full data sharing.

Privacy tiers

TierWhat you shareTypical use
No-data guideLocal inspection + typed observationsFirst pass; nothing leaves browser until Extract
Metadata onlyField names, objects, stages, report titlesReveal processes/assets without record contents
Aggregate onlyCounts, rates, medians, distributionsSurface pains and bottlenecks safely
Redacted sampleAnonymized snippetsGround abstractions and edge cases

Raw evidence is not persisted. Only human-approved graph patches merge into the model.

Workflow

  1. Run Analyze → open Improve the model
  2. Pick a high-severity gap → Map from evidence
  3. Choose privacy tier; enter observations
  4. Review graph patch proposals → apply selected changes
  5. Re-Analyze and repeat

For stricter boundaries (VPC-only LLM), ask your technical owner to run the private-runtime deployment profile.

Analyze vs Generate

ActionLLMOutput
AnalyzeNot usedPrescores (I·F·M·Q), operator coverage, Improve the model panel, grouped candidates, Interview Sprint
Generate full planUsedLLM-written use cases and business artifacts for the candidates you selected, plus a sequenced plan (quick wins + strategic bet)
Critique & rewriteUsed (with Generate)Extra pass flagging generic candidates before final ranking
Opportunity dossiersUsed (with Generate)Decision-ready one-pagers: business case, pilot scope, risks, metrics, kill criteria
Agent blueprintsUsed (with Generate)Parameterized agent specs + deployment-readiness kits for the selected opportunities, gated by evidence confidence
Agent scaffoldsUsed (with Generate)Pilot-ready scaffolds on top of the blueprints: runtime contract, eval suite, readiness gap workflow, and pilot package (auto-enables Agent blueprints)
Pilot validationsUsed (with Generate)Mock-eval readiness report on top of the scaffolds: validation manifest, mock connector pack, eval run plan, and a pilot readiness verdict (auto-enables Agent blueprints + scaffolds)
Pilot executionsUsed (with Generate)Actually runs the mock eval suite from a ready validation against mock connector stubs: case results, unsafe-action check coverage, a threshold result, and a mock pilot verdict (auto-enables Agent blueprints + scaffolds + Pilot validations)
Human pilotsUsed (with Generate)Converts mock-eval-passed executions into a structured human pilot plan: protocol, evidence capture checklist, risk register, completion criteria, and a pilot readiness verdict (auto-enables all prior phases)
Production deploymentsUsed (with Generate)Converts ready human pilots into a production deployment readiness package: connector inventory, deployment package, operating model, production eval plan, and a handoff readiness verdict (auto-enables all prior phases)

When to Analyze: constantly — as you edit, after every evidence apply, between interview sessions. It is free and instant.

When to Generate: when high-severity gaps are closed and you need sponsor-ready wording. Generate usually takes a few minutes and requires LLM configuration.

After Analyze, every candidate card has an Include in Generate checkbox. The default is the top five non-generic candidates; use Top 5, All, or Clear in the selection bar to choose the set explicitly. Prerequisites for a selected opportunity are included automatically, and candidates you leave unchecked remain on the board as Analyze only context rather than consuming LLM time.

Use the How far should Generate go? slider to choose the artifact depth: roadmap only, agent blueprint, scaffold, validation, mock execution, human pilot, or production handoff. The advanced checkboxes mirror that depth for teams that want exact stage control.

Without LLM configured, Auto-map and Generate return ready-to-run prompts you can execute elsewhere.

Agent blueprints and deployment readiness

Because the engine already knows which operators fired, it can take the selected opportunities one step further: tick Agent blueprints before Generate to get a parameterized agent spec plus a deployment-readiness kit for each — mission, operating loop, inputs/outputs, tools and connectors, human approval points, then permissions, an eval plan, telemetry, rollout steps, and guardrails.

This is a specification, not an automated deployment. The hard parts of shipping an agent — connectors, permissions, data quality, approval boundaries, and evaluation — are surfaced honestly rather than hidden.

Crucially, graph evidence confidence drives whether an agent is justified, not just whether the opportunity is interesting. Each blueprint carries a readiness status, computed from your facts (not the LLM):

StatusWhat it means
ReadyStrong evidence: linked pain, accessible data, specific grounding, no unmet prerequisite
Needs discoveryWorth building toward, but key evidence (data access, owners, scope) is still missing
Prototype onlyA product/commercialization idea or thin evidence — explore as a prototype, not a deployed agent
Not recommendedThe evidence does not justify creating an agent yet (no deployment kit is written)

A use case can rank highly yet still be only "needs discovery" — high impact does not mean deployment-ready. Close the gaps the blockers list calls out (link a pain, mark data accessible, confirm an approval owner) and the status rises.

Agent scaffolds and the pilot loop

Tick Agent scaffolds to go one step further than the blueprint (this auto-enables Agent blueprints, since scaffolds build on them). For each agent you get four pilot-ready layers:

  • Runtime contract — agent instructions, tool contracts, input/output schemas,

a least-privilege permission manifest, environment-variable names, mock connectors so the pilot runs offline, and a human-approval workflow.

  • Eval suite — golden cases, synthetic edge cases, a pass/fail rubric,

unsafe-action checks, and the minimum threshold to clear before you trust it.

  • Readiness gap workflow — each blocker turned into a concrete ask ("upload 10

redacted tickets", "confirm write vs draft-only access", "name the approval owner") so low readiness becomes an evidence loop, not a dead end.

  • Pilot package — a scoped 2-week pilot: users, sample data, rollout steps,

monitoring, kill criteria, security review, sign-off checklist, and before/after metrics.

Each scaffold is classified by type — workflow agent, copilot agent, analytics agent, or prototype brief (for strategy/commercialization plays). These are pilot accelerators, not live deployments: scaffolding plus a validation loop to safely prove an agent before any production rollout. Testing expectation: run the eval suite against the mock connectors and clear the minimum threshold before the pilot, then use the gap workflow to close blockers.

Pilot validations and reading pass/fail readiness

Tick Pilot validations to go one step further than the scaffold (this auto-enables Agent blueprints and Agent scaffolds, since validation builds on both). Instead of just producing pilot material, this checks whether that material is actually complete and safe enough to run a mock pilot with — still never connecting to a real customer system:

  • Validation manifest — what is testable right now vs. what is blocked by

missing evidence (e.g. no mock connector named for a tool, no pass/fail rubric for the golden cases).

  • Mock connector pack — an offline stub for every declared tool: a

fixture-file name and the expected input/output shape, with write/action tools called out explicitly.

  • Eval run plan — the golden cases, edge cases, unsafe-action checks, and

minimum threshold in one runnable plan, with placeholders and malformed entries already stripped out.

  • Pilot readiness report — the verdict, in plain language:

| Readiness | What it means | |-----------|---------------| | Ready for mock eval | Tools, mock connectors, golden cases, a rubric, and (if needed) an approval workflow are all in place — run the mock eval suite | | Needs evidence | Something is missing or still open (evidence tasks, a rubric, connector coverage) — close it, then re-run | | Unsafe to pilot | A write/action tool has no approval boundary, or the underlying blueprint itself is not recommended — do not pilot yet |

A `prototype_brief` scaffold gets a prototype checklist instead of a mock-eval verdict — there is no runtime contract to validate, so pilot validation is skipped in favor of the smallest-prototype note.

This verdict is computed from the graph and the scaffold, not the LLM — the optional narrative (pilot summary, top risks, recommended next steps) can explain the verdict, but it cannot change it.

Pilot executions and reading mock-eval verdicts

Tick Pilot executions to go one step further than validation (this auto-enables Agent blueprints, Agent scaffolds, and Pilot validations, since execution builds on all three). Instead of just saying a pilot is ready to run, this actually runs the eval plan — offline, against mock connector stubs, never a real system:

  • Execution manifest — whether this validation could actually be run, or

why it was skipped (not `ready_for_mock_eval`, a `prototype_brief`, no cases, or the mock connector pack isn't ready).

  • Mock run inputs — the fixture-shaped input used for every case that ran.
  • Case results — each golden/edge case scored pass (the mock stub

returned the fixture's declared expected output), fail (no expected output to grade against), or blocked (no connector coverage for that tool).

  • Unsafe-action check results — whether every declared unsafe-action

check was actually exercised by at least one case. A check that is declared but never tested by any case shows untested — a real gap in the eval suite, not just a missing checklist item.

  • Threshold result — cases passed / total against the declared minimum

pilot threshold (a missing or unreadable threshold conservatively assumes a 100% pass requirement rather than skipping the check).

  • Mock pilot verdict, in plain language:

| Verdict | What it means | |---------|---------------| | Mock eval passed | Every executable case passed and the threshold was met | | Mock eval failed | The suite ran, but the pass rate fell short of the threshold | | Blocked | The validation wasn't ready to run, had no cases, or the mock connector pack isn't ready — see the execution manifest | | Unsafe to run | A write/action tool has an unsafe-action check that no case actually exercises, or the underlying validation itself was unsafe |

This phase is deliberately honest about what it can prove offline: a mock connector stub is defined to echo back the fixture-declared expected output, so a "pass" here confirms the scaffold's wiring — tool coverage, fixture completeness, safety-check coverage, threshold discipline — is sound. It does not, and cannot, grade genuine agent reasoning without a live LLM loop against real systems. The optional narrative (execution summary, failure analysis, recommended fixes) can explain the verdict, but it cannot change it.

Human pilots and reading pilot readiness

Tick Human pilots to go one step further than mock execution (auto-enables all prior phases). For each `mock_eval_passed` execution this generates a structured plan a customer technical owner can take into a real pilot:

  • Pilot manifest — whether the human pilot is ready to run or what is

blocking it (missing pilot cohort, no sample data requirements, no kill criteria, missing security review checklist, no success metrics, or missing approval owner for write/action tools).

  • Pilot protocol — duration, cohort, allowed workflows, human review

steps, escalation path, and success metrics.

  • Evidence capture plan — what to collect during real pilot runs: agent

inputs/outputs, human decisions and overrides, unsafe-action attempts, latency, user feedback, and before/after metrics.

  • Pilot risk register — residual risks, mitigations, named owners, and

stop conditions.

  • Completion criteria — what "done" means for the pilot.
  • Human pilot verdict, in plain language:

| Verdict | What it means | |---------|---------------| | Ready for human pilot | All gates cleared — the pilot can be scheduled | | Needs pilot setup | One or more required fields are missing — see the manifest | | Unsafe for human pilot | The underlying mock execution was `unsafe_to_run` — resolve safety issues first |

Production deployments and reading production readiness

Tick Production deployments to go one step further than human pilot planning (auto-enables all prior phases). For each `ready_for_human_pilot` plan this generates a production deployment readiness package:

  • Readiness manifest — whether the production handoff is ready or what

is blocking it (incomplete human pilot, missing rollback plan, no monitoring, missing security review, no sign-off checklist, no eval plan, or missing approval owner for write/action tools).

  • Deployment package — connector inventory with credential placeholders,

permission manifest, environment variables, rollout stages (shadow → supervised → production), rollback instructions, kill switch, and audit log requirements.

  • Operating model — owner matrix for product, technical, security, support,

incident response, eval maintenance, and write-tool approval.

  • Production eval plan — regression suite sourced from the mock eval cases,

shadow-mode checks, unsafe-action alert thresholds, and periodic review cadence.

  • Production deployment verdict, in plain language:

| Verdict | What it means | |---------|---------------| | Ready for production handoff | All gates cleared — the deployment package can be handed to engineering | | Needs production setup | One or more required fields are missing — see the manifest | | Unsafe for production | The human pilot was `unsafe_for_human_pilot` — resolve safety issues first |

This phase produces a readiness package and gate, not an automated deployment. No credentials are created, no infrastructure is provisioned, and no external systems are called.

For concrete examples of this handoff path (ready, needs setup, unsafe), see [Evidence to deployment examples](../docs/EVIDENCE_TO_DEPLOYMENT_EXAMPLES.md) and `examples/evidence_to_deployment/`.

Each production deployment card also has a Download deployable scaffold (code) button — a zip of real starter files (`agent/runtime.py`, tool adapters, `.env.example`, `Dockerfile`/`docker-compose.yml`, an eval harness, `OPERATING_MODEL.md`, and a handoff README), generated deterministically from that candidate's runtime contract and connector inventory. What ends up inside depends on what each declared tool actually is — this is an honesty boundary, not a single mock skeleton:

Export tierWhat you get
Spreadsheet-nativeA tool backed by a spreadsheet (e.g. a ticket queue or FAQ sheet) ships the company's real bundled `.xlsx` data plus a real reader/writer — no mock at all for that tool.
Typed connector`rest_api`, `http_webhook`, `sql`, `local_file`, and `smtp` tools get a real, config-only adapter: set the environment variables in `.env.example` and it calls, queries, reads, or sends for real — no code changes.
Best-effort API`bespoke_api` tools get a real HTTP call too, but the one `build_request()` mapping function is marked TODO, since a proprietary API's request/response contract can't be inferred from a company model alone.
Mock stubAnything else falls back to an offline fixture-echoing stub — credentials are placeholders and nothing talks to a real system until you replace it.

`agent/runtime.py` itself is real, not a placeholder — and its *structure* matches the agent's type, so packages differ in code, not just in the prompt:

  • Workflow agents get a queue-processing runtime: batches are ranked and

deduped deterministically in code (bundled triage engine) before any LLM call, then processed per item in priority order.

  • Copilot agents get a grounded, interactive runtime: every question is

grounded on the real files under `data/knowledge/` via bundled retrieval code, and citations come from retrieval, not the model.

  • Analytics agents compute first and narrate second: bundled engines

produce the numbers in real code, and the LLM only explains them.

  • The "AI drafting over spreadsheets" shape gets a real batch runtime that

reads the bundled `.xlsx` rows, calls an LLM to draft each reply, and writes drafts back for human review.

Each package also ships `agent/engines/` — real, stdlib-only implementations selected by the agent's operator (forecast baseline, schedule heuristic, significance testing, safe aggregation, record reconciliation, triage, retrieval) with selftests the bundled eval harness actually executes — and `agent/guards.py`, which enforces the operating model in code: a file-based kill switch, a per-run action budget, and an audit log of executed writes.

It's a scaffold, not a live deployment: a technical owner still supplies real credentials and an LLM key, and verifies any remaining mock or best-effort adapters before running this unattended in production. For the per-operator mapping of what ships and how confidently it delivers, see [Deployable code by operator](../docs/DEPLOYABLE_CODE_BY_OPERATOR.md).

Reading the results

Operator coverage

On Review, after Analyze. Shows N/52 firing. Green = operator matched graph facts. Grey = dormant, with unlock hint (e.g. "Add a project and write its reusable abstraction").

Aim for green operators that match the company's actual leverage — not every box ticked.

Candidate buckets

The review board groups candidates deterministically:

BucketMeaning
PrerequisitesDo first — Instrument First, Risk Guardrails, Test Harness, Simulation & Digital Twin, Synthetic Data Generation, …
Quick winsHigh feasibility + meaningful impact
Strategic betsHigh moat, plausibly feasible
Needs more evidenceMedium quality — gaps still hurt ranking
Dropped as genericFailed quality gate

Each card also shows Evidence that would strengthen this. Use Add in builder to jump to the section that would improve the candidate, or use the candidate-specific interview question when the missing evidence needs a human answer. After Generate, cards are labeled Full plan generated or Analyze only so you can see which selected candidates received full artifacts.

Evidence & provenance map

From the builder or Review, choose Open evidence map or Download evidence map to create a self-contained HTML exhibit. It connects the reviewed company graph to source classes, cited financial metrics, open evidence gaps, and the ranked opportunities supported by those facts. The file works offline and does not require a Foundry login.

This is a proof-of-process export, not a full audit log: raw evidence, review history, outside-in signal ledgers, and complete candidate lineage are not embedded.

Generate output artifacts

Each generated use case may include:

  • why_this_company — defensibility tied to graph facts
  • first_experiment — smallest shippable pilot
  • required_evidence — what to validate before scaling
  • what_would_make_us_drop_this — kill criteria

With Opportunity dossiers enabled, selected quick wins and the strategic bet package into Markdown one-pagers you can copy to slides or memos.

Target Selector plan

The final plan surfaces:

  • Quick wins — fund momentum (high impact + feasibility)
  • One strategic bet — highest moat that is plausibly feasible
  • Sequencing note — prerequisites honored; overlaps flagged

Quick wins and the strategic bet are never collapsed into one recommendation.

Roles in a multi-person program

ContributorContributionTool surface
Program leadArchetype, focus areas, when to GenerateSource + Evidence + Review
Domain SME (finance, ops, …)Processes, pains, data accessEvidence (department lens + builder)
Technical leadAssets, constraints, stackEvidence → Assets + Constraints
Executive sponsorRisk appetite, budget directionSource (Company context) + Review → dossiers
ConsultantYAML export, cross-session continuityEvidence YAML tab + Source saved models

Assign one department per session. Diagnostics = the sprint backlog — do not ask contributors to complete the whole form before they see value.

More workflow detail: Consultant interview guide

Authentication and saved models

When the server has `AUTH_SECRET` set:

  • Sign in / register to use Analyze, Generate, Auto-map, and saved models
  • Saved models persist across devices, private to your account
  • Marketing pages and the Builder guide remain public

When auth is disabled (typical local dev), all features work without login but models live in browser storage only.

Privacy and data handling

  • Pasted notes, uploads, and evidence snippets are redacted before LLM calls
  • Raw uploads are never persisted on the server
  • Public website crawl content is treated as public (not redacted)
  • Only human-approved graph facts enter the company model
  • Workstation capture is opt-in; raw activity stays on the employee's

machine and only a redacted summary (or reviewable patch proposals) reaches Foundry

Full policy: Trust and evidence boundary

Troubleshooting

SymptomLikely causeWhat to do
All candidates feel genericThin model — no linked pains, no accessible dataClose top gaps; add pains to high-volume processes
Many dormant operatorsMissing edges or section factsRead unlock hints; see fill order
Auto-map returned sparse modelPublic sources lack operational detailMerge internal docs; use Map from evidence
Generate fails or hangsLLM not configured or timeoutCheck `/api/status`; verify provider setup with your technical owner
Generate says nothing is selectedAll candidate checkboxes are clearedPick specific cards, or use Top 5 / All in the selection bar
Workstation capture shows no ActivityWatch eventsPermissions not granted, or app not runningConfirm ActivityWatch is running and (macOS) Accessibility/Screen Recording permissions are granted, then retry
Rankings shift when I editExpected — Analyze re-runs on every editUse risk appetite / slider to stabilize tilt
Lost work after refreshNot savedSave to server or export YAML
YAML edits not reflectedSwitched tabs without applyingReturn to YAML tab and ensure content is saved

Example companies

The app ships contrasting examples you can load to learn the model:

ExampleProfile
Meridian FabCustom fabrication; physical ops, scarce estimator judgment
Lumen ConsultingData/analytics consultancy; services, regulated clients
Beacon PayRegulated fintech; data-rich, risk-averse
Atlas LedgerFinance back office; month-end close, reconciliation
Harbor Link 3PLLogistics; supplier delays, stockouts
Cascade SupportB2B SaaS support; ticket triage, SLA recovery
Nova GrowthMarketing ops; campaign lifecycle, attribution
Atlas Ledger Production DemoBundled demo — AP invoice matching, pre-cleared through production handoff
Meera Home & KitchenBundled demo — spreadsheet-native support triage with a real LLM-backed runtime
Beacon Pay Production DemoBundled demo — regulated chargeback evidence drafting with adjudicator approval gates

Load one, change two fields, and watch the evidence preview react before modeling your own company. The three bundled deployment demos above are pre-graded: Generate returns a finished, gate-cleared output instantly instead of calling an LLM.

Further reading

DocumentAudienceContents
Builder guideAnyone using the builderField reference, fill order, interview sprint, Analyze vs Generate
Consultant interview guideConsultants and interview leadsLeading/searching prompts to elicit graph-quality evidence
MethodologyReviewers and sponsorsHow ranking and evidence-based discovery work
ResourcesProgram leads and operatorsSupporting material, examples, and implementation context
FAQNew usersCommon product and workflow questions

Quick reference card

```

  1. Source → example | auto-map | optional Company context
  2. Evidence → add facts → operators green → pains linked | Outside-in
  3. Review → approve proposals → Analyze → close top 3 gaps → repeat
  4. Generate → when gaps closed → optional critique + dossiers
  5. More input → capture | CX (after first Analyze)
  6. Save → server or YAML between sessions

```

Loop: `add facts → analyze → interview → evidence → analyze → … → generate`

Remember: dormant operators = roadmap. Generic candidates = thin evidence. Defensible bets appear when the graph reflects how the business actually operates.