Marc Manley

SnapSchaught: The phase 1 build plan

With the stack chosen (Next.js + Postgres via Neon + Vercel Blob + Claude vision extraction), this is the phase 1 build plan: repo setup, Vercel/Neon/Blob provisioning, schema draft, and the Claude Code task breakdown.

The companion post SnapSchaught: Choosing the stack covers the technology audit—framework, database, extraction engine, charting, redaction, and retention decisions. This post is what actually gets built.

Repo setup

Suggested repo name: snapschaught (matches the app name, matches the naming convention of every other property—marc-site, middle-ground-dev-site, plain lowercase-hyphenated names).

Confirmed available (checked via gh repo view marcmanley/snapschaught—no existing repo).

What I’ll do (once approved):

  • gh repo create marcmanley/snapschaught --private --description "Document-ingestion financial budgeting webapp"—private repo, matching the pattern of marc-site, middleground, senior-moments (all private)
  • Scaffold with create-next-app (App Router, TypeScript, Tailwind, ESLint) as the initial commit
  • Add CLAUDE.md at repo root (per the standard pattern) documenting: schema conventions, the extraction-interface abstraction (swappable backend requirement), redaction pipeline location, standard build/verify workflow
  • Add a LICENSE file—recommend MIT for the app’s own code, with a note in the README calling out the two non-MIT dependencies (PyMuPDF’s AGPL option if used; Vercel/Anthropic as proprietary hosted services)

Vercel project under MGMCUpland

What I can do myself:

  • Once the GitHub repo exists, connect it to Vercel via the Vercel GitHub App installation already authorized on MGMCUpland (the team all other properties now live under). Standard “Import Project” flow, no new GitHub App re-authorization needed.
  • Configure the build (Next.js auto-detected), set production domain to snap.middleground.dev
  • Wire up Neon and Vercel Blob integrations via one-click Vercel Marketplace flows (these write env vars directly into the Vercel project)

What you need to do (dashboard/account-level actions I can’t perform):

  • Confirm Vercel Blob is enabled/available on MGMCUpland’s plan (Pro-tier should include it, but Blob has usage-based billing that needs acknowledgment the first time)
  • Confirm the Neon integration is authorized for MGMCUpland—first-time marketplace integrations sometimes require an explicit “Add Integration” click and account linking
  • DNS: likely no new action needed—snap.middleground.dev is a new subdomain under middleground.dev, which is already pointed at Vercel. Vercel will show the exact CNAME target once the domain is added; if OnlyDomains’ DNS panel needs a new record added, I’ll report back the exact record
  • Anthropic API key: if you want a dedicated API key (separate from the Hermes fleet’s key) for cost tracking/isolation, generate that in the Anthropic console and hand me the key to add as a Vercel env var

Neon (database) provisioning

  1. From Vercel project’s Storage tab, add Neon integration (Marketplace)
  2. Accept free tier initially (100 compute-hours/month, 0.5GB storage)—ample for phase 1
  3. Region: match to Vercel deployment region (likely iad1/us-east)
  4. Neon auto-injects DATABASE_URL into Vercel env vars
  5. Run initial schema migration via Drizzle against the fresh Neon database

Vercel Blob provisioning

  1. From Vercel project’s Storage tab, add Blob storage (native Vercel product, simpler than Neon)
  2. Vercel auto-injects BLOB_READ_WRITE_TOKEN
  3. No bucket/region config needed—single global namespace per project

Schema draft (phase 1 scope)

Designed for single-user now, multi-user-friendly later: every table that would eventually need per-user scoping carries a user_id column from day one, defaulting to a single hardcoded “owner” row—so adding real auth later is a matter of wiring up an auth provider and enforcing the existing column, not a schema migration across every table.

-- Single row for now; becomes a real users table when multi-user auth is added.
CREATE TABLE users (
  id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
  email TEXT,                          -- nullable in phase 1
  created_at TIMESTAMPTZ DEFAULT now()
);

CREATE TABLE documents (
  id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
  user_id UUID NOT NULL REFERENCES users(id),
  original_filename TEXT NOT NULL,
  mime_type TEXT NOT NULL,             -- application/pdf, image/jpeg, image/png, text/markdown, text/plain
  blob_url TEXT,                        -- NULL once discarded per retention policy
  redacted BOOLEAN NOT NULL DEFAULT false,
  retained BOOLEAN NOT NULL DEFAULT false,  -- false = discard after extraction
  extraction_status TEXT NOT NULL DEFAULT 'pending', -- pending | processing | success | failed
  extraction_error TEXT,
  raw_extraction_json JSONB,           -- structured output from extraction engine
  uploaded_at TIMESTAMPTZ NOT NULL DEFAULT now(),
  processed_at TIMESTAMPTZ
);

CREATE TABLE stores (
  id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
  user_id UUID NOT NULL REFERENCES users(id),
  canonical_name TEXT NOT NULL,        -- "Vons" (normalized)
  raw_names TEXT[] NOT NULL DEFAULT '{}', -- ["Vons #2210", "VONS 2210 SUNSET"]
  created_at TIMESTAMPTZ DEFAULT now(),
  UNIQUE (user_id, canonical_name)
);

CREATE TABLE categories (
  id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
  user_id UUID NOT NULL REFERENCES users(id),
  name TEXT NOT NULL,                  -- "Groceries", "Utilities — Electric"
  parent_category_id UUID REFERENCES categories(id), -- allows nesting later
  UNIQUE (user_id, name)
);

CREATE TABLE transactions (
  id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
  user_id UUID NOT NULL REFERENCES users(id),
  document_id UUID NOT NULL REFERENCES documents(id),
  store_id UUID REFERENCES stores(id),          -- nullable: utility bill has no "store"
  category_id UUID REFERENCES categories(id),
  transaction_date DATE NOT NULL,
  total_amount NUMERIC(12,2) NOT NULL,
  document_type TEXT NOT NULL,          -- 'credit_card_statement' | 'utility_bill' | 'receipt' | 'other'
  metadata JSONB,                       -- type-specific extras: utility usage (kWh, gallons), statement period
  created_at TIMESTAMPTZ DEFAULT now()
);

CREATE TABLE line_items (
  id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
  transaction_id UUID NOT NULL REFERENCES transactions(id) ON DELETE CASCADE,
  raw_item_name TEXT NOT NULL,          -- "ORG CARROTS 2LB" — exactly as extracted
  canonical_item_name TEXT,             -- "Carrots" — nullable until phase 2 normalization
  quantity NUMERIC(10,3),
  unit TEXT,                            -- "lb", "ea", "gal", "kWh"
  unit_price NUMERIC(12,4),
  total_price NUMERIC(12,2) NOT NULL,
  created_at TIMESTAMPTZ DEFAULT now()
);

-- Indexes for actual query patterns
CREATE INDEX idx_transactions_user_date ON transactions(user_id, transaction_date);
CREATE INDEX idx_transactions_category ON transactions(category_id);
CREATE INDEX idx_transactions_store ON transactions(store_id);
CREATE INDEX idx_line_items_canonical_name ON line_items(canonical_item_name);
CREATE INDEX idx_line_items_transaction ON line_items(transaction_id);

Phase 1 note on normalization: the audit flagged normalization as “the unglamorous but critical design problem,” properly scoped to phase 2. Phase 1 writes raw_item_name faithfully and leaves canonical_item_name null—the column exists now so phase 2 doesn’t need a schema migration, just a backfill job. Phase 1’s dashboard only needs transaction-level and category-level rollups, which don’t depend on item normalization.

Task breakdown for Claude Code (phase 1 scope)

Sequenced so each task is independently buildable/testable, matching the “smallest possible change, verify before moving on” discipline. I’ll run these via claude -p (print mode) for mechanical scaffolding and interactive mode (tmux) for anything needing iterative back-and-forth (extraction prompt tuning, dashboard layout).

  1. Scaffold—create-next-app (App Router, TS, Tailwind), add shadcn/ui, add CLAUDE.md/LICENSE/README.md, PWA manifest + Serwist service worker, empty app/api/ structure. Verify: npm run build succeeds, deploys to Vercel preview, PWA is installable (Lighthouse check).

  2. Database layer—Drizzle ORM setup, schema from above as Drizzle schema files, migration run against fresh Neon database, thin data-access module (lib/db/*.ts) with typed query functions. Verify: migration applies cleanly, seed script inserts one fake document/transaction/line-item and reads it back.

  3. Upload UI—drag-and-drop + file-picker component (mobile + desktop), client-side file-type validation (PDF/JPEG/PNG/MD/TXT only), direct-to-Blob signed-URL upload flow (browser uploads straight to Blob, not through a function first). Verify: manually drop a test PDF and image on both phone and desktop, confirm files land in Blob storage.

  4. Redaction module—swappable redact(file, type) interface; PyMuPDF-based implementation for PDF, Presidio-based for JPEG/PNG, wired to run when user toggles “redact” at upload time. This runs as its own Python-based Vercel serverless function since PyMuPDF/Presidio are Python libraries. Verify: upload test PDF containing a fake account number, toggle redact on, confirm number is gone from both visual render and text-extraction check (not just visually covered).

  5. Extraction module—swappable extract(file, type) → StructuredDocument interface; phase-1 implementation calls Anthropic API with vision prompt tuned per document type (credit card statement / utility bill / receipt), returns normalized JSON (vendor, date, total, line items, category guess). Includes PDF text-layer-first check before routing to vision. Verify: run against 2-3 real documents of each type, manually check extracted totals/line items against source for accuracy.

  6. Ingest pipeline wiring—connects upload → optional redaction → extraction → DB write, with extraction_status transitions (pending → processing → success/failed) and error surfacing. Verify: full end-to-end flow from file drop to populated transactions/line_items row, including deliberately-broken file to confirm failure handling.

  7. Real-time UI update—lightweight mechanism (SSE endpoint or polling) so dashboard reflects newly processed document without manual refresh. Verify: drop file, watch “processing…” state resolve into visible data with no reload.

  8. Basic dashboard—document list view (status, date, vendor, total), category-rollup summary (spend by category, simple date-range filter), using Recharts for initial charts. Verify: dashboard reflects real ingested data accurately, responsive check on mobile and desktop viewports (confirm desktop layout actually uses extra space).

  9. Retention enforcement—implement discard-after-extraction default: after successful extraction, if retained = false, delete Blob object and null out blob_url. Verify: upload without “keep” checked, confirm Blob object is actually gone after processing completes.

  10. Deploy + health check—push to main, confirm Vercel deploy succeeds on MGMCUpland, run standard rendered-page screenshot verification on both mobile and desktop viewports, confirm snap.middleground.dev resolves and serves correctly.

Not in phase 1 scope (deferred to phase 2/3): item-name normalization, item-level price-history charts, store-name normalization, store comparison views, TradingView Lightweight Charts integration (phase 1 uses Recharts only).

Open items before I start building

  1. Repo visibility—private (matching the usual pattern) or public (leaning further into “as open-source as possible”)? Not decided yet.
  2. Dedicated Anthropic API key for cost isolation, or reuse the Hermes fleet’s key? Affects one setup step.
  3. Your go-ahead on this plan—repo name, schema shape, 10-task sequence—before any gh repo create or Vercel project is made.

Once approved, I’ll start at Task 1 and report back after each verified milestone rather than batching all 10 into one silent run—easier to catch a wrong turn early.