Server data from the Official MCP Registry
Read-only decision table over ACH return codes: 17 codes, 5 decisions, rule shown.
About
Read-only decision table over ACH return codes: 17 codes, 5 decisions, rule shown.
Security Report
Valid MCP server (2 strong, 1 medium validity signals). No known CVEs in dependencies. ⚠️ Package registry links to a different repository than scanned source. Imported from the Official MCP Registry. 1 finding(s) downgraded by scanner intelligence.
8 files analyzed · 1 issue found
Security scores are indicators to help you make informed decisions, not guarantees. Always review permissions before connecting any MCP server.
Permissions Required
This plugin requests these system permissions. Most are normal for its category.
What You'll Need
Set these up before or after installing:
Environment variable: PAYPILOT_UNRECORDED_PATH
Environment variable: PAYPILOT_TENANT
Environment variable: PAYPILOT_TENANCY
Environment variable: PAYPILOT_LEGACY_TENANT
Environment variable: PAYPILOT_SEND_EMAIL
Environment variable: PAYPILOT_APPROVAL_MODE
Environment variable: PAYPILOT_IMAP_SECURITY
Environment variable: PAYPILOT_APPROVER_EMAILS
Environment variable: PAYPILOT_PUBLIC_BASE_URL
Environment variable: PAYPILOT_APPROVAL_TTL_HOURS
Environment variable: PAYPILOT_GMAIL_GRACE_MINUTES
Environment variable: PAYPILOT_ALLOWED_RECIPIENTS
Environment variable: PAYPILOT_ALLOWED_LINK_HOSTS
Environment variable: PAYPILOT_HOLDOUT_PCT
Environment variable: PAYPILOT_HOLDOUT_SEED
Environment variable: PAYPILOT_LLM_BUDGET_USD
Environment variable: PAYPILOT_AI_DISCLOSURE
Environment variable: PAYPILOT_ALLOW_UNSIGNED_WEBHOOKS
Environment variable: PAYPILOT_FAKE_BANK
How to Install
Add this to your MCP configuration file:
{
"mcpServers": {
"io-github-ivansflowgit-paypilot-ach-returns-mcp": {
"args": [
"paypilot-ach-returns-mcp"
],
"command": "uvx"
}
}
}Documentation
View on GitHubFrom the project's GitHub README.
PayPilot
An AI dunning agent that recovers failed subscription payments.
Live demo -> paypilot.fly.dev - try it in the browser, no setup or API key required.

When a recurring charge fails, most of that revenue is recoverable - the customer
didn't decide to churn, their card just expired or a payment bounced. PayPilot
turns each invoice.payment_failed event into a grounded, on-brand recovery
action: it diagnoses why the payment failed, picks the right retry strategy for
that reason, and drafts a warm, one-click dunning email - all in a single API call.
It's built as a small, readable LangGraph agent with retrieval-augmented generation (RAG) over a dunning playbook, exposed through a FastAPI endpoint. The whole thing runs its test suite with no API key and no network.
How it works
A failed-payment event flows through a seven-node LangGraph StateGraph. Each node
enriches a shared, typed RecoveryState and hands it to the next:
flowchart LR
A[retrieve_context] --> R[assess_risk]
R --> B[diagnose_reason]
B --> C[choose_strategy]
C --> S[schedule_retry]
S --> D[draft_message]
D --> E[finalize]
E --> F([END])
| Node | What it does |
|---|---|
retrieve_context | Loads the customer record and pulls relevant snippets from the dunning playbook via the RAG retriever. |
assess_risk | Deterministic (no LLM): scores churn risk (low/medium/high) from the dunning attempt number and the customer's recent failure streak. |
diagnose_reason | Committed template by default, LLM only with PAYPILOT_LLM_DRAFT=1: a 1-2 sentence, playbook-grounded diagnosis of why the payment failed, reflecting the churn risk. |
choose_strategy | Deterministic (no LLM): maps the failure code to a fixed action + retry cadence, then tightens it when churn risk is high. Stable and unit-testable. |
schedule_retry | Deterministic (no LLM): turns the cadence into a concrete next_retry_at UTC time, ready to hand to a scheduler. |
draft_message | Committed template by default, LLM only with PAYPILOT_LLM_DRAFT=1: a short, warm dunning email with one clear call to action. |
finalize | Assembles the {diagnosis, risk, strategy, schedule, message, impact} response payload. |
Why RAG?
The recovery quality depends on dunning best-practice - retry timing, tone, when to
offer a grace period. Rather than bake that into prompts, PayPilot keeps it in an
editable knowledge source (data/playbook.md) that the
retriever feeds into the diagnosis and drafting nodes.
Which retriever depends on configuration, and it is worth being precise about
this. With OPENAI_API_KEY set, app/ingest.py builds a FAISS index over the
playbook using OpenAI embeddings (k=3). With no key - which is how the public
demo runs - it falls back to a lexical keyword retriever, so no FAISS index and
no embedding call is involved in anything a visitor sees.
And the playbook only changes the output on the LLM path. On the default
zero-inference path the committed templates are rendered as-is and retrieved
context is not consulted, so editing playbook.md changes nothing until
PAYPILOT_LLM_DRAFT=1 is set. Playbook edits are input to the build-time
generation step, not to every request.
Why a deterministic strategy node?
choose_strategy is intentionally not an LLM call. Retry cadence and the chosen
action come from a fixed rules table (app/decision.py) keyed on the failure code,
which comes from Stripe or Recharge:
| Failure code | Retry in | Action | Tone |
|---|---|---|---|
card_expired | ~1 day | request card update | Friendly, routine |
insufficient_funds | ~3 days | wait and retry | Soft, no pressure |
generic_decline | ~2 days | retry and verify | Calm, helpful |
issuer_do_not_retry | never | request new payment method | Plain, no blame |
card_details_invalid | never | request card update | Plain, no blame |
direct_debit_not_retried | never | request manual payment | Plain, no blame |
retries_exhausted | never | request new payment method | Plain, no blame |
The LLM writes the message; the policy stays predictable.
irreversible-reach maps which model-produced values in an agent can reach an action that cannot be undone. Its PayPilot manifest marks sending an email as PayPilot's only irreversible action, with deterministic guards in front of it; remove those guards and it fails. A manifest describes code, it does not test it.
Risk-aware escalation
assess_risk reads the dunning attempt number and the customer's recent
payment history (from data/customers.json) and buckets churn risk. When it's
high - a third attempt, or a run of recent failures - choose_strategy
tightens the retry cadence and marks the strategy escalated, the diagnosis
calls out the urgency, and impact discounts the recovery odds for a customer
who keeps bouncing. So the agent reasons about history, not just the single
event in front of it.
Quickstart
# 1. Install
python -m venv .venv && source .venv/bin/activate
pip install -r requirements.txt
# 2. Configure (only needed to call the live LLM; tests don't need it)
cp .env.example .env # then add your OPENAI_API_KEY
# 3. Run the API. PAYPILOT_TENANT=demo serves the bundled demo fixtures (data/customers.json).
PAYPILOT_TENANT=demo uvicorn app.api:app --reload
Call it
curl -s http://localhost:8000/payment-failed \
-H 'Content-Type: application/json' \
-d '{
"customer_id": "cust_001",
"amount": 1499.0,
"currency": "usd",
"failure_code": "card_expired",
"attempt": 1
}' | jq
{
"diagnosis": "The card on file for Acme Robotics has expired, so the Scale renewal couldn't be charged; ...",
"risk": { "attempt": 1, "prior_failures": 0, "churn_risk": "low", "escalate": false },
"strategy": { "action": "request_card_update", "retry_in_days": 1, "offer": "...", "escalated": false },
"schedule": { "retry_in_days": 1, "next_retry_at": "2026-07-02T09:00:00+00:00", "retry_on": "2026-07-02", "timezone": "UTC" },
"message": "Hello Acme Robotics, we tried to renew your Scale plan but the card we have on file has expired ...",
"impact": { "amount_at_risk": 1499.0, "currency": "USD", "recovery_likelihood": 0.7, "expected_recovered": 1049.3, "annual_value_at_risk": 17988.0, "churn_risk": "low" }
}
GET /health returns {"status": "ok"} for liveness checks. The full response
schema (typed with pydantic) is browsable at /docs.
The endpoint is rate limited per client IP, and the response payload is a typed
RecoveryResponse (diagnosis, risk, strategy, schedule, message,
impact), so the contract shows up precisely in the OpenAPI docs.
POST /payment-failed/batch runs a whole billing run (up to 50 events) in one
call and adds a portfolio aggregate - total at risk, total expected recovered,
and how many accounts are high churn risk. GET /portfolio-impact rolls that up
across the demo customers and powers the recoverable-revenue headline on the
landing page.
Speaks Stripe
Status: no client traffic reaches this deployment, so no merchant's live Stripe
delivery has either. The handler is exercised offline by tests/test_stripe.py.
POST /webhooks/stripe accepts a real Stripe invoice.payment_failed event. It
verifies the Stripe-Signature header (HMAC-SHA256) when STRIPE_WEBHOOK_SECRET
is set, acknowledges other event types with a 200 so Stripe won't retry, maps
Stripe decline codes (expired_card, insufficient_funds, ...) to PayPilot's
failure codes, and runs the recovery graph. Point a webhook (or
stripe trigger invoice.payment_failed) at it; add
metadata.paypilot_customer_id to resolve a demo customer.
Speaks Recharge
Status: this deployment carries no client traffic, so no merchant's live Recharge
delivery has reached it. The handler is exercised offline by tests/test_recharge.py.
POST /webhooks/recharge accepts Recharge charge/failed and
charge/max_retries_reached deliveries for Shopify subscription stores. It checks
X-Recharge-Hmac-Sha256 against RECHARGE_CLIENT_SECRET the way Recharge
documents it, and maps every one of Recharge's 109 published error types to a
PayPilot failure code (the full table). Recharge runs its own retries, so when it will retry, its
retry_date is reported as the schedule with source="recharge" and no customer
email is drafted (message_suppressed: "recharge_retry_scheduled"). Errors that are
not a payment problem the customer can fix (inventory, shipping, tax, test mode)
and error types PayPilot does not recognise are acknowledged and not run, so no
customer gets an email about them.
A merchant comes on in three steps. They create an API token and copy the API
Client Secret from their Recharge admin; the secret goes on the deployment as
RECHARGE_CLIENT_SECRET; then
RECHARGE_API_TOKEN=... python scripts/recharge_register_webhooks.py --apply
registers both webhooks against this deployment (dry run without --apply, and
safe to re-run).
The webhook is idempotent on the Stripe event id, so a retried delivery
replays the stored result instead of re-running the graph. POST /payment-failed
and /batch accept an optional Idempotency-Key header for the same guarantee.
Every response carries X-Process-Time and X-Request-ID headers, emits a
structured JSON access log, and 429s include Retry-After. GET /metrics (admin token required)
returns a JSON snapshot (request counts by status, average latency, recoveries
run, total expected recovered). Batches roll up per currency, so a mixed
USD/EUR/GBP billing run stays correct (aggregate.by_currency).
For discovery, the app also serves /robots.txt, /sitemap.xml, and an
/llms.txt summary for AI answer engines,
and the landing page ships SoftwareApplication + FAQPage JSON-LD.
The closed recovery loop
PayPilot does not stop at drafting. It records what failed, what it sent, and what happened next, so "we recovered X" is a figure you can audit rather than a claim.
invoice.payment_failed -> record -> strategy -> portal link -> email
|
invoice.paid / payment_succeeded -> recovered <---------------+
customer.subscription.deleted -> churned
Per-invoice state machine: failed -> messaged -> clicked -> recovered | churned | exhausted. Illegal moves raise rather than silently overwrite, so the
dashboard can never contradict revenue it already reported. Amounts are stored
in integer minor units with a currency code. Floats appear only at the
presentation edge, never in storage or arithmetic.
State only advances to messaged on a real send. A dry run, a suppressed
recipient or a provider failure leaves the invoice at failed, because claiming
we contacted someone we did not is exactly what this ledger exists to prevent.
An invoice we never saw fail is never counted as a recovery. Most invoices in a Stripe account are paid without ever failing, and counting those would inflate the one number the product is judged on.
Honest attribution
Some failed invoices recover on their own: Stripe retries them, and customers
fix their cards unprompted. So /report splits invoices into three arms -
treated (actually messaged), holdout (deliberately withheld) and untouched
(a dry run or suppressed send) - and reports each separately. A lift figure is
withheld until both arms reach 30 invoices, and says so rather than printing a
confident percentage from four data points.
Holdout assignment is a deterministic hash of the invoice id, stable across restarts and reproducible from invoice ids alone. It defaults to 0 percent: withholding dunning from paying customers is a decision, not a default.
A public sample of the dashboard, on a fixed cohort and labelled as sample data, is at /report/sample. The real one is admin-gated: it holds revenue data and reveals which invoices were withheld from dunning.
Prove it end to end
PAYPILOT_DEMO_EMAIL=you@example.com make demo-loop
Drives Stripe test mode through the whole cycle with a test clock: a subscription is created and paid, its card goes bad, a month passes, the renewal genuinely fails, the recovery runs, the card is fixed, the invoice is paid, and the dashboard is printed before and after. It refuses to run against a live key.
Zero-token architecture
The default path performs no inference at all. A dunning email for a given
failure code is the same class of output every time, so the copy is generated
once, reviewed by a human, committed as data/templates/dunning.json, and
filled deterministically at runtime. That makes what a customer reads reviewable
the way code is reviewable: it diffs, and changing it is a pull request.
An OPENAI_API_KEY alone does not enable chat inference; live drafting also
requires PAYPILOT_LLM_DRAFT=1.
One honest caveat: a key does still enable embeddings. With OPENAI_API_KEY
set, the retriever builds a FAISS index over the playbook once per process (a
lazy singleton in app/ingest.py) and then embeds the query on each request.
So the index cost is paid once, but per-request query embedding is a real, small
token cost on a path otherwise described as zero-inference.
Three CI gates keep the rest honest: a full recovery must construct no chat
model, a ChatOpenAI( call site without a written BUILD-TIME / CACHEABLE /
TRUE-RUNTIME classification fails the build, and the committed copy must cover
every failure code and contain no URL. The call-site gate matches on that
literal string, so it would not catch a different SDK or a call outside app/.
Multilingual copy, on the same seam
A locale is a dimension on the committed catalogue rather than a second system: the generator writes it, a human reviews it, the runtime fills it, and no model is called on the path. Two decisions are worth naming.
A locale is all or nothing. A half-translated catalogue is not merged key by
key with English, because that mails one customer a Spanish subject over an
English body, which reads as broken software rather than as a missing
translation. A catalogue that does not cover every kind and every failure code is
refused as a locale and English is served whole, with the reason readable per
locale (incomplete_locales()). Completeness is measured against the English
catalogue rather than a fixed list, so adding a failure code to English makes
every translation incomplete until it is translated, which is the direction that
fails safe.
The fallback is reported, not silent. render_with_locale() returns the
locale actually used, because a silent fallback is indistinguishable from a
translation that is working, and a climbing fallback count is the measurement
that says which language to translate next. A missing catalogue falls back rather
than failing closed: an English dunning email recovers money where no email
recovers none. Missing copy altogether is still fatal.
A language is never derived from the currency, and a test greps the module's own source to keep it that way. EUR spans twenty countries, USD is legal tender in Ecuador and Panama, and GBP is not a language.
State today, measured rather than implied: available_locales() returns
['en']. The machinery is complete and runnable and there is one catalogue in
it. scripts/generate_templates.py --locale translates the live English rather
than regenerating per language, so the languages cannot drift apart in meaning,
and it refuses a draft that renamed a {name}, {plan} or {business}
placeholder, since str.format would then leave a customer reading a literal
brace. Producing a reviewed Spanish catalogue is one command and a speaker's
review, not a build.
An MCP server over the decision table
An agent asks, the table decides, and the answer carries which rule fired. The
model scores, suggests and drafts; governed logic decides. app/mcp_server.py is
that sentence made callable: an MCP server whose three tools read a fixed
decision table, so the answer cannot be argued with by anything the agent read on
the way in.
Seventeen ACH return codes, five decisions. There are five answers to the one question that decides what happens next: what has to change before this debit can exist again. Nothing, the account details, their instruction, our own entry, or nothing ever will. A table with one row per code is a reference document and the registrar publishes that for free. A table with one row per decision is a rule set, and that is the part a team cannot download.
| Row | Goes again | What has to change first | A person acts first | Codes |
|---|---|---|---|---|
| 1 | yes | nothing | no | R01, R09 |
| 2 | not on this entry | account details | no | R02, R03, R04, R13, R20, R28 |
| 3 | not until they ask | their instruction | no | R08 |
| 4 | corrected only | our own entry | no | R11 |
| 5 | no | nothing will | yes | R05, R07, R10, R14, R15, R16, R29 |
It is published at paypilot.fly.dev/ach-returns
and readable in the repository at
docs/ach-return-decisions.md. Both are generated
from app/ach_return_map.py by scripts/render_ach_page.py, so neither holds the
table and neither can disagree with it.
R11 is the row worth the table. It arrives in the unauthorized family and is
still correctable, so it is the one code where the family and the decision
disagree, and the one thing here that is hard to know.
The default is visible as a default. Every code from R00 to R99 that the
table does not list lands on row 5, where nothing automatic happens, and the
answer comes back with matched set to false and a reason saying it fell through
rather than matched. It is never null and it is never row 1. A caller that cannot
tell a match from a fallback has been handed a guess dressed as an answer.
The three tools
| Tool | Answers |
|---|---|
lookup_return_code | one code: its row, whether the entry goes again, what has to change first, the wording for the customer, and whether a named person acts first. R01 and R09 also carry the reinitiation limit (at most two times, within 180 days of the original settlement date) with its source; no other code states one |
list_decisions | the five rows and the codes on each, so the table is legible to an agent rather than only queryable |
explain | which row fired and why, plus what the table does and does not state about the code's family |
Nothing writes, nothing calls a model, and nothing reaches the network. That is
declared per tool as read_only_hint and open_world_hint: false rather than
promised in prose, so a client can read it off the tool list and
tests/test_mcp_server.py asserts it. The same suite runs every tool again with
socket.socket replaced by a function that raises, which is the claim measured
rather than intended.
What it refuses to do
- It does not report a return code family unless the source table states one.
The published rulebook is sold rather than free, so a family asserted from
memory is an unverified claim.
R11is the one code whose family the source states. For every other codeexplainreturns no family and says why. - It does not invent whether a person is needed. That column exists in the source table and is empty on every row, so the value is derived from the decision row instead, by a rule named in the module. The import asserts the source column is still empty, so if anyone fills it in the server refuses to start rather than silently disagreeing with its own source.
- It does not guess at an unknown code. See the default row above.
Install it
Published on 28 October 2026 to PyPI and the official MCP Registry as io.github.IvanSFlowGit/paypilot-ach-returns-mcp. It ships as its own package, built from the same three source files, so a client needs no clone:
uvx paypilot-ach-returns-mcp # serves over stdio
From a clone of this repository instead:
pip install "mcp>=2,<3" # the extra is declared in pyproject.toml
python -m app.mcp_server # serves over stdio
This repository runs from source rather than being pip-installed, so
pip install -e ".[mcp]" does not build: setuptools cannot auto-discover
packages in a flat layout with several top-level directories. The
[project.optional-dependencies] entry declares the dependency and its bound;
install it directly as above.
A client stanza ships at .mcp.json, pointing at .venv/bin/python
so it works against the virtualenv the Quickstart creates. Point command at
whichever interpreter has the extra installed.
Proving a client can load it
A green unit suite is not evidence that a client can load the server: those are
different failures with the same tick. scripts/mcp_witness.py spawns the server
as a subprocess, completes the initialize handshake as a real MCP client, and
carries its controls in the run rather than firing them once:
python scripts/mcp_witness.py
It checks the tool list is exactly the three tools and that each is declared read
only and closed world, then runs a positive control (R11 must come back on
row 4 in the unauthorized family) and a negative one (an unlisted code must
come back on row 5 with matched false). A server that answered nothing would
pass neither; a server that matched everything would pass the positive alone. It
exits non-zero naming the control that failed, so it gates a release rather than
being read by eye, and CI runs it on every push.
Verified on 2 October 2026 against mcp 2.2.0, protocol 2025-11-25.
Deployment model
One deployment per client is the default and the strongest separation. One deployment can also serve several clients, but that has not carried production traffic for more than one client yet (see Tenancy: what is built and what is not). There is no tenant-facing control plane. The client creates a restricted Stripe key scoped to invoices (read), customers (read) and billing portal sessions (write) - nothing else - and registers their own webhook endpoint with its own signing secret. Those keys live in their deployment's environment, never in ours, never in code.
PayPilot never collects card details. The only payment surface is a
Stripe-hosted page: a billing portal session, or the invoice's
hosted_invoice_url.
Two settings matter more than the rest:
PAYPILOT_DB_PATHmust be on a persistent volume. It holds the recovery ledger. On ephemeral storage a redeploy erases the history every number is computed from.- Webhook signature verification is mandatory by default. With no
STRIPE_WEBHOOK_SECRETset, every event is rejected with a 400. The only way to accept unsigned events isPAYPILOT_ALLOW_UNSIGNED_WEBHOOKS=1, which exists for the credential-free public demo and belongs nowhere near real customer data.
Full client setup: docs/onboarding.md, about 30 minutes.
Tenancy: what is built and what is not
One deployment can serve several clients, or each client can have a deployment of
their own. Design and milestones: docs/multi-tenancy.md.
Per-client configuration: docs/tenant-config.md. What a
client's IT lead is told it touches: docs/what-paypilot-touches.md.
| Built and tested (deployed in the live app; only single-tenant mode is in use) | Not built |
|---|---|
Every ledger row carries a tenant_id; keys, joins, deletes and purges are scoped to it (M2) | Production use of per-tenant database logins (M9), and any run with more than one client. The default shared role stops a missing predicate but not injected SQL that sets the tenant itself |
| Per-tenant configuration and Fernet-encrypted, write-only secrets (M3) | A tenant-facing UI |
Bearer tokens (hash stored), five roles, one authorize() that every route calls; a wrong-tenant caller gets 404 (M4) | Per-tenant quotas and per-tenant LLM budget accounting |
/admin routes and scripts/tenants.py: create, suspend, resume, offboard, tokens, secrets, GDPR export and erase (M4, M6) | OIDC sign-in for humans (tokens only) |
POST /t/{tenant}/webhooks/{stripe,recharge,chargebee}: one failure body, tenant-keyed idempotency, unsigned only for demo (M5) | Any production run with more than one client |
| Suspension: dunning events dropped, closing events still applied, approvals expired, no mail leaves (M5) | |
A migration of an existing ledger (python -m app.migrate_tenancy), run by scripts/start.sh before the server starts | |
Tests that fail if a query loses its tenant predicate, if a route skips authorize(), or if a new outbound call appears (tests/test_egress_inventory.py) | |
An optional Postgres backend with forced row-level security (M7) and a verified one-off copy of an existing SQLite ledger into it (M8). Off by default. Checked by scripts/pg_rls_check.py against a real Postgres 15 on a throwaway cluster (2026-10-28), where every check passed |
Postgres backend (optional, off by default). PAYPILOT_DB_BACKEND=postgres with
PAYPILOT_TENANCY=multi keeps the ledger in Postgres instead of SQLite. Each tenant gets its own
connection as a role with no access to platform tables; every transaction sets the tenant first;
app/pg_schema.sql forces row-level security on the six tenant tables, so a query with no tenant
set sees nothing. Connection settings are the libpq variables (PGHOST, PGDATABASE, PGPORT,
PGSSLROOTCERT, verified TLS with no fallback) plus PAYPILOT_PG_TENANT_USER/PASSWORD and
PAYPILOT_PG_PLATFORM_USER/PASSWORD. To move an existing ledger across,
python scripts/pg_copy_ledger.py LEDGER.db is a dry run, and --apply copies inside one
transaction and commits only if every table is identical by row count and content hash. The
claim is limited: the SQL translation and the isolation were exercised on a throwaway Postgres 15,
not on a production database. By default every tenant shares one database role, which cannot stop
injected SQL. Setting PAYPILOT_PG_TENANT_ROLES=1 gives each tenant its own database login
(pp_t_<tenant>, password derived from PAYPILOT_PG_ROLE_KEY, created by the operator with
scripts/pg_roles.py), so the database reads the tenant from who is connected; that mode (M9) is
written, independently reviewed, and its attack checks (escaping to another tenant by SET ROLE, by
the setting trick, by shadowing a function, and by changing its own password) pass on a real
Postgres 15 on a throwaway cluster; it has not carried production traffic. In this mode the app refuses to start if the database admin
or shared-role credentials are in its environment, and scripts/pg_roles.py sync disables the shared
login. It needs PostgreSQL 15 or later, or REVOKE CREATE ON SCHEMA public FROM PUBLIC, and the
provisioning step refuses otherwise. Single-tenant
Fly deployments on SQLite are unchanged.
Single mode (PAYPILOT_TENANCY unset) behaves as it always did: one tenant, ADMIN_TOKEN
for the admin routes, the original webhook URLs. Multi mode (PAYPILOT_TENANCY=multi)
has no static admin token: ADMIN_TOKEN becomes a one-shot bootstrap value that mints the
first platform token (POST /admin/bootstrap, or scripts/tenants.py bootstrap) and
then opens nothing. The public demo runs as the single tenant demo (PAYPILOT_TENANT
and PAYPILOT_LEGACY_TENANT in fly.toml).
Deploys fail loudly on an unmigrated or foreign ledger. scripts/start.sh
runs the migration before uvicorn and stops the container if it refuses, and
/health opens the ledger and answers 503 (with the error class only) when it
cannot. The migration is not the Fly release_command, because a release machine
is not known to mount the volume.
A normal deploy restarts the one machine. Measured on 2026-10-28: about five seconds from the old
process stopping to the health check passing, with the migration reporting already current. Fly logs
the health check as failing during that window. There is one machine and one volume, so there is no
old version serving while the new one starts.
A refused migration takes the whole app down. A machine with a volume is
updated in place, so when the start script stops the container there is no older
version left serving, and the landing page goes with it. Before the first deploy
of this change, copy the live ledger and run the migration against the copy
(PAYPILOT_DB_PATH=copy.db PAYPILOT_TENANT=demo PAYPILOT_LEGACY_TENANT=demo python -m app.migrate_tenancy), and require exit 0 and a 200 from /health.
Rolling back a migration
A successful migration cannot be undone by deploying the previous image: the old
code writes without a tenant and fails on the new NOT NULL column. To go back,
stop the machine, copy /data/paypilot.db.pre-tenancy-<timestamp>.bak over
/data/paypilot.db, delete paypilot.db-wal and paypilot.db-shm, then deploy
the old image. The backup is written before any change and verified row by row
against the locked file.
The decision slice, on Fly and on AWS
The part of PayPilot that decides money also runs on its own, with nothing else
attached: one request in, the rules table consulted, a decision out naming the
rule that fired, and an audit row written before the decision is returned. It
is standard library only (app/decision.py), so the same code serves it in two
places:
| Fly.io | AWS (eu-west-2) | |
|---|---|---|
| Compute | the FastAPI app | Lambda (arm64, Python 3.12) |
| Edge | Fly proxy | API Gateway HTTP API, throttled |
| Audit store | SQLite on the Fly volume | RDS Postgres, private subnets, TLS verified |
| Infrastructure | fly.toml | Terraform in infra/aws/ |
Five routes, identical status codes and bodies on both:
GET /health no auth, no database, carries the build commit
POST /decide bearer token, decision + audit row
GET /decisions/{invoice_id} bearer token, the recorded decisions, newest first
POST /decisions/{invoice_id}/outcome bearer token, records whether the invoice was paid
GET /decisions/outcomes/summary bearer token, the outcome counts for this client
/health carries the commit the image was built from, because "is the deploy
live" was otherwise answerable only by counting the paths in the published
OpenAPI document against a local count, which is a proxy. Unset reads
unknown rather than being omitted: an absent key reads as an older build of
the same image, which is the reassuring reading and the wrong one.
tests/test_decision_contract.py asserts
the body is identical on both deployments and caught the field being added to
FastAPI and not to the Lambda handler, which would have left the AWS path
unable to report what it was built from.
Two required fields, and that is the whole integration. POST /decide accepts
four fields and requires two of them: invoice_id and failure_code raise when
absent, while attempt defaults to 1 and prior_failures to 0
(app/decision.py). Any other field is refused by name rather
than ignored. So a caller that keeps no retry history sends an invoice id and a
failure code and is integrated, and the Stripe, Recharge and Chargebee webhooks below
feed the same rules table rather than being the way in. They are not adapters
onto this route and the distinction is checkable: the webhook path enters at
handle_event or run_recovery while /decide enters at
handle_decision_request, and what they share is STRATEGY_RULES, defined
once in app/decision.py and imported by
app/nodes.py. Both routes are in
the OpenAPI document at /docs: a contract
nobody can find is not published.
One client cannot read another's decisions. The audit row carries a
client_id resolved from the bearer token and from nowhere else. DECISION_API_TOKEN
is either one opaque token, which resolves to the client id default exactly as
a single-tenant deployment already behaves, or a JSON object mapping token to
client id. Both inserts stamp it and both selects filter on it, and a lookup with
the wrong client returns 404 rather than an empty 200, so it cannot confirm
somebody else's invoice exists. A client_id in the request body is refused: a
caller declaring its own identity is a label, not authentication.
One contract suite, two base URLs. tests/test_decision_contract.py
runs against the FastAPI app and the Lambda handler in process on every CI run,
and against any deployed URL when PAYPILOT_CONTRACT_BASE_URL and
PAYPILOT_CONTRACT_TOKEN are set. A separate AWS suite would prove the suite,
not the deployment. On 15 September 2026 the live run passed against Fly and
against AWS, and again against AWS after the whole stack was destroyed and
applied again from clean.
Built as a working system rather than a client engagement. It is deployed and running, and it does not carry client traffic.
What the AWS side does, and why:
- Fails closed. No configured token is a 503, never an allow. A decision whose audit row cannot be written is refused, not returned unrecorded.
- Append-only by database grant, not only by code. The public function logs
in as
paypilot_app, which maySELECTandINSERTon the audit table and nothing else. A separate bootstrap function (app/decision_bootstrap.py), reachable from no route and invoked by Terraform, holds the master password, applies the schema, and then logs in as the app role and provesUPDATE,DELETE,TRUNCATEandCREATEare each refused with Postgres error 42501. Any other outcome fails the apply. - No NAT gateway, no internet gateway. The function needs no outbound internet: secrets are read from SSM at plan time, so the most expensive idle resource on a small AWS account is simply absent.
- Observed, not only logged. API access logs (seven days, no auth header or body), an alarm on rejected tokens, an alarm on Lambda throttling, and a monthly budget, all notifying one address.
- Pinned supply chain. The Lambda's five packages are pinned by sha256 and
installed with
--require-hashes; a changed wheel fails the build. CI actions are pinned to commit SHAs. - One token per platform. Fly and AWS each hold their own bearer token, so a leaked one opens one deployment, not both.
- A locked-down network. The VPC's default security group carries no rules, and rejected traffic is recorded in VPC flow logs.
- A deploy identity that cannot promote itself. Terraform runs as an IAM user
holding only
infra/aws/iam/policies, built from the API calls CloudTrail recorded it making. It may create roles only under the stack's name and only with a permissions boundary that caps them at logging and VPC networking, and it is denied creating users, keys or policy versions, stopping CloudTrail, and changing its own permissions. Changes to those policies are a root action, by design. - Account guardrails in their own Terraform root
(
infra/aws/account/), so the slice's destroy cycle can never remove them: a multi-region CloudTrail with log file validation, an account-wide S3 public access block, EBS encryption by default and an IAM Access Analyzer.
Not measured yet, so not claimed: what the AWS slice costs to leave running. That number is read off the bill after 24 hours idle and added here, not estimated.
Reproduce it (your own AWS account, with three SSM SecureString parameters
created first: the master database password, the app role's password and the
API token; names in infra/aws/variables.tf):
infra/aws/build_lambda.sh
cd infra/aws && terraform init && terraform apply
PAYPILOT_CONTRACT_BASE_URL="$(terraform output -raw api_base_url)" \
PAYPILOT_CONTRACT_TOKEN=... ../../.venv/bin/python -m pytest ../../tests/test_decision_contract.py -k live
Terraform state holds the SSM values in plaintext. It is local and gitignored; treat it as a secret.
The outcome loop, and why it is a separate table
The decision API used to record what it decided and never learn whether it
worked. The stateful webhook path marks a failure row recovered on
invoice.paid (app/loop.py); the stateless path a client
actually integrates had no equivalent, so a recovery-rate claim was true of the
demo and unmeasurable where the product is sold.
POST /decisions/{invoice_id}/outcome closes it, and the outcome lands in its
own decision_outcomes table rather than in a column on decision_audit
(app/decision_audit.py). An outcome column would
need an UPDATE, and "there is no update or delete method on either backend" is
a guarantee rather than an observation, so the outcome is appended beside the
decision instead of mutating it. Both tables stay append-only and a disputed
number can still be traced to the rows that produced it.
A caller may report exactly two outcomes, paid and not_paid. unknown is
deliberately not reportable: it is what outcome_for returns when no row
exists, so nobody can assert it and nobody has to. Unknown means nobody has
told us, which is not the same as not paid, and collapsing those two is how a
recovery rate becomes fiction. Reports are idempotent on a unique index over
(client_id, report_key), so a retry cannot double-count.
Bank rails, and the one that ships a refusal
app/rails.py is one registry, so a caller asks the registry rather than
importing a rail. Three are registered: ach, sepa and bacs.
ACH and SEPA each ship a real table read from the scheme's own published text,
with the scheme's own suggested creditor action per code rather than an action
inferred here (app/ach_return_map.py,
app/sepa_return_map.py).
Documentation truncated — see the full README on GitHub.
Reviews
No reviews yet
Be the first to review this server!
More Developer Tools MCP Servers
Rea
Freeby Morluto · Developer Tools
Reverse engineer anything from your terminal or agent with one CLI and MCP server.
Paperclip
Freeby Paperclipai · Developer Tools
Trending hip-hop artist momentum scores across four cultural dimensions.
Toleno
Freeby Toleno · Developer Tools
Toleno Network MCP Server — Manage your Toleno mining account with Claude AI using natural language.
MCP Marketplace
Freeby mcp-marketplace · Developer Tools
Search and install MCP servers from inside your AI client.
mcp-creator-python
Freeby mcp-marketplace · Developer Tools
Create, build, and publish Python MCP servers to PyPI — conversationally.
by Lharries · Communication
Read, search, and send WhatsApp messages through your AI assistant
