Compass Ask — founder brief

1 Are we building a harness?

Compass Ask has 3 parts. Name each part separately.

Part 1 is the runtime loop PoC today. chat.py:run_turn() starts claude -p once per turn. Each call passes Opus 5, 12 read-only tools over MCP, and 1 session id. Each call also passes --strict-mcp-config, --allowedTools mcp__compass, --tools "", --setting-sources "", --max-turns 12, and a 240 s limit. The persona travels as a 6 KB argv string, not a path (chat.py:24). chat.py:26-27 strips every ANTHROPIC_* variable and re-injects the subscription token.

Part 2 is the eval harness PoC today. scenarios.py holds 12 scripted conversations. chat.py:grounding() checks that each provider name in the answer appears in the tool results. accept_offers.py accepts the bot's own closing offer, then checks delivery. Measured today: 33 turns, 33/33 grounded, 11/11 offers delivered, median 7.1 s, maximum 20.7 s, $2.265.

Part 3 is the expected-answer test. Part 3 does not exist yet. 4 facts limit the score above:

• 3 turns call no tool and still pass, because converse() relaxes the check to the whole session (chat.py:88-91).
• PROVIDER_NAMES is a fixed list of 10 names (chat.py:7), so an invented provider never fails.
• offer_fulfilled reduces to "the first line holds no negative phrase" (chat.py:78).
• scenarios.py stores questions, not expected answers. No check tests a correct answer.

production design Add expected-answer fixtures, a per-claim guard, and 1 negative control. Make the guard fail before you trust the guard. 33/33 is a pass rate on a test with no failing mode.

2 Tools

Compass Ask reads Compass through 12 read-only tools (mcp_server.py). PoC today

The tool layer enforces 1 founder rule end to end. DIRECTION_MEANING ships the client-side reading with every row (compass_tools.py:89). _recorded() returns 13 flags per provider and gates the offer rule (:43-47). _verified_note() adds the stale note (:55-59).

"No data is not no" lives in the prompt, not in the tools. Only 2 of 12 returns carry not_recorded_for. Move that rule into 1 shared serializer. production design

3 defects block a ship:

1. partial is unreachable, so an amber row reports full_match (:283). Compass holds the same dead branch (match_providers.rb:146-156), which hides gaps.
2. Zero account rows return red (:262). Compass returns grey (account_matching.rb:6-12). Moneycorp and Corpay then read as "will not serve".
3. match_transaction runs a prospect through the partner matcher. match_prospects.rb states that a prospect has no verdict.

3 Data sources

Compass holds 7 capability tables per provider, plus contacts, conversations and per-row provenance. The 12 tools read those tables and nothing else. lookup_requirements marks amount, pricing and cadence as not evaluated. PoC today

The data is thinner than the schema. Facts from data/compass.json, a copy of the local dev database:

• 6 live partners, 619 capability rows, 0 rows with a provenance source.
• 5 live partners carry last_verified_at of 3 or 4 March 2026, so 195 days. 4 of those 5 stamps land inside 2 seconds, so the stamp is a backfill.
• Corpay (UK) has a null last_verified_at and 0 rows in all 7 tables.
• 0 contacts and 0 conversations across all 6 live partners.
• Onboarding text runs 31 to 184 characters per live partner, and 0 for Corpay (UK).
• 191 account rows, 81 named rows, 8 recorded domiciles, and 171 of 191 rows in GB.

Every provenance line the bot has printed comes from data/prospects.json, which is synthetic. So prompt rule 2 is untested against real provenance. PoC today

text_search does not index conversations or contacts. So "who mentioned M-Pesa on a call" stays unanswerable after those tables fill. Index both tables. production design

4 Storage

PoC today 2 flat files hold the data: data/compass.json (157 KB) and data/prospects.json (23 KB). export.rb writes both through bin/rails runner, including contact emails, conversation summaries and recorder identity. Conversation state lives in the claude -p session file, addressed by a uuid. Transcripts land in runs/scenarios/*.json without the tool-result text, so nobody can re-audit a grounding verdict later.

production design Remove the export. Read live Postgres through a read-only role. Call Compass::MatchProviders and Compass::MatchProspects instead of a second matcher.

Own the transcript. Add compass_ask_sessions and compass_ask_turns. Store the question, the answer, the tool calls, the tool results, the model, the cost and the grounding verdict. Replay stored turns into each call. Keep the CLI session id as a debug value only.

3 storage constraints need work before the gap payoff is real:

1. Compass::Lookup validates an admin or an api_client (lookup.rb:240-244). Map each Slack user to an Account, or every source: 'ask' write fails.
2. The snapshot writer exists twice, in the admin concern and in api/v1/lookups_controller.rb:111-130. Extract 1 writer. UnservedAnalysis reads that exact shape.
3. INV-R1 keeps a prospect out of 5 surfaces (prd-rolodex.md:200-202). Every tool defaults include_prospects: bool = True, so the model decides scope today. Resolve scope from caller identity server-side.

5 Customisation

3 knobs earn a place. production design

1. system_prompt.md. The file holds the persona, the answer shape, the hard rules, the qualification order and the vocabulary. Change 1 file. No code deploy is necessary.
2. The channel and member allow-list. ASK-018 already specifies the Slack gates (plans/prd-compass-ask.md:318). Exclude every externally shared channel by structure.
3. 1 settings row. The row holds the line cap, the stale-day threshold and the text_search breadth. _verified_note() hard-codes 90 days today.

Everything else is code plus a spec. Keep these 5 items out of configuration:

• The tool set. A tool is a schema contract, not a setting.
• Prospect visibility. Derive prospect visibility from caller identity. Keep prospect visibility unreachable from the prompt.
• The model. Replace COMPASS_ASK_MODEL (chat.py:6) with a frozen Opus 5 id. Add a spec that rejects a Sonnet id. extract_from_transcript.rb:31 already sets claude-sonnet-5, so the drift is live.
• Match semantics. Change the wildcard domicile rule and the amber collapse in Ruby, with 1 RSpec case per status pair.
• The scenario suite. Move scenarios.py to RSpec with expected-answer fixtures.

system_prompt_v1.md stays for reference, not for a test. Both versions score 100% on the present guard, so no metric separates the 2 versions. PoC today

6 Persona

PoC today system_prompt.md is 1 file, passed on every turn. The file holds 6 sections: who you are, the answer shape, the hard limits, 6 hard rules, the qualification playbook and the vocabulary. The answer shape is a 4-part template: 1 count line, 1 line per provider, 1 caveat footer, at most 1 follow-up. The offer rule is item 4 of that shape. An offer must be a tool result, and a provider-specific offer needs the recorded flag to be true.

Version 1 failed because version 1 asked for a register, not a shape. Version 1 said "a sharp colleague" and allowed paragraphs. Paragraphs mixed 3 providers into prose, and "colleague" allowed opinions. Version 4 replaced the register with a shape plus a ban list. 3 of the 4 banned phrases come from real version 1 transcripts.

The prompt requests 8 lines. Nothing enforces 8 lines. 3 of 33 turns print 9 lines, and no reader asked for a full list. PoC today

production design Add a deterministic style guard, as Callum AI does. Check the line count, the ban list, the field codes and the offer position. Generate the vocabulary table from the enums and fail a spec on a missing phrase. Add a founder register that changes verbosity only. Never let a register suppress a hard rule. Move the working directory to an empty directory, because chat.py:29 runs inside a home tree that holds 2 CLAUDE.md files.

7 Skills

A skill is a named, versioned playbook: 1 trigger, the required inputs, a tool sequence, an output template and a guard. A skill is a fixed path through the 12 tools, and adds no data access.

Reject Claude Code skills on the server: a skill load needs setting sources, which re-open the CLAUDE.md leak. production design

Build each skill as a Ruby service in app/services/compass/ask/, then expose the service as 1 more tool. The tool description is the router, as the 12 tools already route. Reserve MCP prompts for a human slash command. Give each skill a scripted scenario and a style check. A skill without a passing scenario does not ship.

Hold 1 guardrail. A skill never writes a capability row or a provenance row. 2 writes are in scope: a Compass::Lookup with source ask, and the turn log. A skill that proposes a capability row emits a draft through ApplyTranscriptIngestion. The bot says "drafted 4 rows for Riva", never "recorded".

Ranked skills:

1. qualify-a-lead [PoC today, informal]. Promote the prompt block to a service first.
2. save-and-hand-off production design. Write the lookup, then reply with the number and link.
3. named-account-coverage production design. Serves lever 2, the top deal-killer.
4. verify-stale-provider production design. 5 of 6 live partners sit past 90 days.
5. compare-two-providers production design.
6. onboarding-pack production design. Onboarding text is 31 to 184 characters, so version 1 gives the contact.
7. prospect-shortlist-for-gap idea. Depends on skill 2.

8 What makes it 10x

The 10x version holds a view and brings that view to the reader. Compass already computes almost every signal a brief needs: ProviderRecency, UnservedAnalysis, GapCoverage and GapProspects.

2 facts block every proactive feature. First, the data is empty: 0 prospects, 0 conversations, 0 contacts and 0 provenance rows in the dev copy. Second, nothing schedules work: Compass runs Sidekiq but holds no cron gem. So a scheduled brief is a real story, not a switch. production design

Order the proactive features by the data each feature needs:

1. The questions the bot could not answer. This signal exists on day 1.
2. A capability row created or changed, with provenance attached.
3. A zero-match qualified lookup.
4. A stale live partner. Align the threshold first: Compass uses 30 and 90 days, the tool uses 90 only.
5. A new prospect, plus the Rolodex firm that covers each gap.

Build the capture loop first production design. Every tool already returns absence explicitly. Group that absence per live partner into a standing question list. Return the answers through ExtractFromTranscript, the draft store and per-row human acceptance. Change extract_from_transcript.rb:31 to Opus 5 first, because that path is the highest-consequence write in Compass.

"What changed in Compass this week" is already free. All 13 Compass models carry has_paper_trail, so that brief is a versions query. production design

Budget the scheduled work. 33 real turns cost $2.265. The binding limit is the 5-hour subscription window, not the money.

9 What makes it a habit

Post on a change, never on a clock. A bot that posts every day loses the reader inside a week. A bot whose triggers cannot fire ships as a feature that does nothing. Today the 90-day trigger fires for the whole panel at once, so order the triggers as section 8 lists them. production design

Make the bot ask 1 question back, not a survey. Example: "Corpay has no rows at all. Does Corpay do local GBP collection?" 1 tap writes 1 row with provenance. production design

Close the loop. "You asked about MZN on the 8th. Equals recorded MZN today." That line needs compass_ask_sessions and compass_ask_turns, because the PoC holds per-conversation memory only. production design

Change the guard shape before you ship a loop-closing post. The present guard checks provider names across the session, so the guard cannot catch the version 3 slip. Check each claim against the tool results of that turn. production design

State the latency honestly. Median 7.1 s, maximum 20.7 s, minimum 3.4 s. The 8 turns that call match_transaction run at a median of 11.4 s. Production tool time goes up, because production calls the real matcher against Postgres. Set the target as first line under 2 s and full answer under 10 s. production design

Protect capacity. Give the bot its own account. Cap concurrency. Queue on a rate limit and say "I will have this in N minutes". Never fall back to Sonnet.

10 How it can be more useful

Set 1 precondition before anyone beyond Stevan and Paul gets access. The per-claim guard runs and scores on the scenario set, and every answer carries a Compass link. Then release in this order: Stevan and Paul, Paul's team, the inbound desk, onboarding. production design

Per role:

• Paul Plewman, COO. A weekly gap list ranked by lookups blocked, with the Rolodex firm that covers each gap. A re-verify queue built from the rows that caveated real answers. Draft rows from his own call notes. No gap DM; the team channel only. production design
• Tom Radford and Sara, onboarding and inbound. Document questions per client type from the onboarding fields PoC today, with 1 of 6 live partners unanswerable by definition. Job 2 is capture: Corpay first.
• Saul Naeimi and the inbound desk. Pre-call qualification, plus an explicit recorded "no" so nobody promises on a null row. PoC today
• Mark. Win and loss against capability. Needs the Pipedrive mapping. idea
• Stevan. "What changed in Compass this week" from PaperTrail production design. Corridor concentration idea; no tool computes concentration today.

Resolve the hand-off with 1 rule. After a qualification the bot offers exactly 1 thing: "saved as Compass lookup number N", plus the link. The bot never contacts a client. production design

Log every question, tool call, answer, grounding verdict and token count in compass_ask_turns. Then the diligence answer about panel access is a table, not a Slack search.

11 Packaging for partners

Build nothing here before the internal bot ships. Start with version 0: the internal bot in a shared Slack channel. Set the version 1 trigger in the PRD: 25 partner questions per week for 1 month. production design

Host the partner bot at CT. Give the partner bot a tool set that returns no provider name, code, count or provenance. Add an egress check that fails the turn. Do not let the model compose a capability claim: CT is not FCA-regulated. Add Compass::PartnerStatement with a named approver. Build the catalogue offline. Emit a statement only when 3 live partners satisfy it. Today only 2 live partners hold a named row with a currency, so most detail drops. production design

Sample partner exchange:

> Partner: Spanish buyer, UK limited company, pays a developer in Madrid. Can you handle it? > Bot: Yes. CT handles a UK limited company that pays into Spain in euros. > 2 things decide the answer. Is the company the client? Is the developer's account in Spain? > Partner: Company. Yes, a Spanish bank. > Bot: Still a yes. Saved as qualification Q-4182. This is not a referral yet. Send the client's name and the CT desk takes over.

Never crosses the boundary:

• Provider name, code, id, logo, integration_type.
• The lookup.results snapshot, /api/v1/lookups/:id, /csv, /pdf.
• Provenance: source type, recorder, date, note, conversation id, transcript.
• Contact rows and prospect rows.
• Counts, rankings, partial-match reasons, the blocker behind a cannot_serve.
• Verification staleness, contact recency, onboarding text.
• Validation error strings from the requirements schema.
• Pricing, margin, commission.

12 What makes it incredible

The binding constraint is the data, not the interface. 3 ideas form 1 loop: capture what Compass does not know, log the demand, then convert a filled row back into revenue.

Idea (a) is the capture loop production design. Compass Ask is the only system that discovers what Compass is missing, because every tool returns absence explicitly. Log that absence, group the absence per live partner, then return the answers through the shipped ingestion path with per-row human acceptance. Idea (a) carries the other 2.

Idea (b) is 2-tier logging production design. Record every ask in compass_ask_turns with the parsed dimensions and the unknowns hit. Write a Compass::Lookup with source ask only when a qualification completes, because the model requires 7 present fields. A wide sweep is demand data, not a gap. Keep a prospect row out of the snapshot under INV-R1, and mark a real client need, or the gap curve measures curiosity.

Idea (c) is reopened lookups idea. A filled capability row re-runs the asks that row used to block. rerun_attributes and UnservedAnalysis#breakdown_for already hold the mechanism, and breakdown_for already flags a provider that failed on exactly 1 requirement. A lookup names no client, so add a nullable client reference at ask time. Route the result to a shared queue with an @-mention, never a DM.

Run 1 test first. Re-run 20 historic zero-match lookups against today's rows. Count how many name a reachable client. That rate decides whether idea (c) exists.

13 Valuation and revenue

Do not sell Compass. SaaS buyer research failed across brokers, family offices, treasurers and professional services. Compass Ask earns its place inside CT's funnel. production design

State the multiple honestly. An introducer earns 0.5 to 1.5 times revenue; a technology-enabled platform earns 2 to 5 times. The £15M to £25M target needs 3 to 3.5 times revenue at £4.5M to £6M. Expansion on today's £2.4M tops out near £12M, so no bot re-rates the company. Compass Ask serves 3 levers: named-account expansion, Nico top-5 retention, and key-man knowledge risk.

State the bar in pounds. FY26 forecast revenue is £2,403k at 62.1% gross profit. February booked £184k revenue and a £3k loss, against a £197k break-even. So Compass Ask needs about £5k per month of extra gross profit. Measure the baseline first.

Treat capacity as the cost: ClaudeSubscriptionClient strips every API key. The ceiling is the 5-hour window on 1 Opus seat. The asset is not ready either: 6 live partners, 1 empty, verified once 195 days ago.

The 5 day-one metrics:

1. Adoption: asks per week and weekly active askers. Stop below 30 asks per week from 4 askers by week 6.
2. Zero-match rate and the largest gap groups, split into unknown-blocked and recorded-blocked.
3. Latency for a single lookup and for a full qualification, with a p95.
4. Ask-attributed outcomes at day 90. Name the identifier that travels from ask to first trade.
5. Answer trust: the ungrounded-claim rate and the offer-fulfilment rate, weekly.