takescake

MTG Card Scanner: Hybrid OCR + Art Matching + Live Pricing

2025-10-04

Scanning physical Magic cards should feel instant and accurate—no typing names, no guessing which printing you're holding. We just shipped a major upgrade to our card scanner that combines three different matching systems: OCR text recognition, visual art similarity, and live market pricing. Here's how it works and why it matters for deck building and collection management.

The Problem: Single-Method Scanning Fails

Most card scanners rely on just one approach:

  • Text-only OCR misses short names ("Snap" → "Instant Ramen" because "Instant" appears on the card)
  • Image-only matching fails with glare, rotation, or damaged cards
  • Manual search wastes time and breaks flow when you're sorting bulk

The solution: combine all three methods and let them vote.

How Hybrid Search Works

1. OCR Text Extraction (Primary)

When you scan or upload a card photo:

  • Tesseract.js extracts all visible text
  • Smart cleaning removes OCR artifacts ("Snap NO" → "Snap")
  • Card type words ("Instant", "Sorcery") are filtered out
  • First 5 lines get priority weighting (card names appear at top)
  • Short names (≤6 chars) get lenient matching thresholds

Result: 100% text match for "Snap" across 10+ printings.

2. Art Similarity Analysis (Tiebreaker)

For cards with identical text scores (all "Snap" prints = 100%):

  • Extract art region from scan (8.5% offset, 83%×48% dimensions)
  • Downsample to 96×96 pixels for fast comparison
  • Load cached local images for candidate matches
  • Compute pixel-by-pixel RGB similarity
  • Dynamic weight adjustment: 65%/35% text/art default, shifts based on confidence

Result: 98% art match identifies exact printing, even when 8 variants exist.

3. Ranking Algorithm (Confidence)

Final sort prioritizes:

  1. Cards with confirmed art matches over text-only
  2. Higher combined scores (weighted text + art)
  3. Cards with local image cache over remote lookups

Edge case handled: Cards that fail to load images (CORS, missing cache) rank below art-confirmed matches to ensure the correct print wins.

Instant TCGplayer Pricing

Every scanned card result shows:

  • Market price (large, prominent)
  • Low/Mid prices (smaller, below market)
  • "Buy on TCGplayer" button (affiliate link, opens new tab)

Data Flow

Scan → OCR + Art Match → Enrich with Local Price Cache → Display
  • Source: Nightly cache (tcgcsv-prices.json, updated 21:10 UTC)
  • Coverage: ~110k cards with TCGplayer product IDs
  • Response time: ~5ms (no network request during scan)
  • Fallback: No price? No price box (graceful degradation)

Why Local Cache Matters

Network requests during scanning kill responsiveness. We pre-fetch all TCGplayer pricing once per day, so every scan is instant. The price box only appears if data exists—no loading spinners, no API delays.

Technical Implementation

Name Extraction Improvements

// Before: "Snap NO" matched against "Instant Ramen" (77%)
// After: cleanOCRName() strips trailing junk
cleanOCRName("Snap NO") → "Snap" → exact match (100%)

Cleaning logic:

  • Remove trailing "NO", "NA", single chars/numbers
  • Preserve apostrophes, commas, hyphens (for "Jace, the Mind Sculptor")
  • Extract first word separately with extra weight

Art Matching Pipeline

1. extractArtRegion(scanImage) → 96×96 RGB samples
2. loadCachedCardImages(candidateIds) → local files or remote URLs
3. computeArtSimilarity(scan, candidate) → percentage
4. computeCombinedScore(textScore, artScore) → weighted blend
5. sortByArtConfidence() → prioritize confirmed matches

CORS workaround: Local image cache (public/cards/*.jpg) avoids cross-origin canvas restrictions. Remote Scryfall images fail silently and fall back to text-only scoring.

Pricing Integration

// API route enriches matches with prices
const cardsWithPrices = await applyLocalPrices(results.map(r => r.card));

// Client displays price box
{match.tcgcsv?.market && (
  <PriceBox>
    ${match.tcgcsv.market.toFixed(2)}
    <BuyButton href={buildTcgProductUrl(match.tcgplayer_id)} />
    Low: ${match.tcgcsv.low} | Mid: ${match.tcgcsv.mid}
  </PriceBox>
)}

Use Cases: Why This Matters

Deck Building

  • Scan alternate art versions and compare prices before buying
  • Check if you already own a specific printing (foil vs. non-foil)
  • Identify reprints instantly (Original Exodus Snap vs. List reprint)

Collection Management

  • Bulk sorting: scan piles quickly, group by value
  • Trade evaluation: instant market prices during negotiations
  • Inventory tracking: match exact printings for insurance/resale

Budget Optimization

  • Price checking: see if that bulk rare spiked
  • Version comparison: original set vs. reprint pricing
  • Buy decisions: one-click affiliate link to TCGplayer

Debug Mode: See the Thinking

Toggle "Show Debug Info" to watch the system work:

=== OCR TEXT EXTRACTION ===
Raw OCR: "Snap NO\n= A Sr ED 5\nInstant ()\nReturn target creature..."
Extracted: ["Snap", "Snap", "Snap", "A Sr ED", "Instant ()"]

=== TEXT MATCHING ===
  1. Snap (Exodus) - 100%
  2. Snap (The List) - 100%
  3. Snap (Duel Decks) - 100%

=== ART SIMILARITY ===
  [1] Snap (Exodus): Text=100% Art=98% → Combined=99%
  [2] Snap (The List): Text=100% Art=76% → Combined=89%
  [3] Snap (Duel Decks): Text=100% Art=94% → Combined=97%

=== FINAL RANKING ===
  1. Snap (Exodus) - 99% (Text: 100% | Art: 98%) ← YOUR CARD
  2. Snap (Duel Decks) - 97% (Text: 100% | Art: 94%)
  3. Snap (The List) - 89% (Text: 100% | Art: 76%)

This transparency helps verify accuracy and understand why the system chose a specific print.

Performance Characteristics

Speed

  • OCR extraction: ~2-4 seconds (Tesseract.js client-side)
  • Text matching: ~100ms (fuzzy search across 110k cards)
  • Art comparison: ~60-200ms (6 candidates × 96×96 pixels)
  • Price enrichment: ~5ms (local cache lookup)
  • Total: ~3-5 seconds from scan to results

Accuracy (Tested with "Snap" Card)

  • Text-only: 77% confidence on wrong card ("Instant Ramen")
  • Hybrid (text + art): 99% confidence on correct print (Exodus)
  • Improvement: 22 percentage points, correct card moves from #9 to #1

Storage Requirements

  • Card images: ~11GB for 110k cards (~100KB avg per image)
  • Price cache: ~15MB (tcgcsv-prices.json)
  • Trade-off: Storage for speed and accuracy (no runtime network requests)

Common Edge Cases Handled

Short Card Names

  • Problem: "Snap" only 4 characters, many false positives
  • Solution: Lower threshold for short names, boost first-word matches, filter card types

OCR Artifacts

  • Problem: "Snap NO" extracted instead of "Snap"
  • Solution: Strip trailing junk (NO, NA, single chars/numbers)

Missing Local Images

  • Problem: 40k+ cards don't have cached images yet
  • Solution: Fall back to text-only scores, rank art-confirmed matches higher

Identical Text Matches

  • Problem: 8 "Snap" prints, all 100% text match
  • Solution: Art similarity breaks ties, correct print ranks #1

CORS Blocking Remote Images

  • Problem: Browser blocks Scryfall canvas access
  • Solution: Download images locally once, use cached versions

Roadmap: What's Next

Price Features

  • 7-day trend arrows (↑ spiking, ↓ dropping)
  • Foil pricing displayed separately
  • "Great deal" badges for below-market listings
  • Stock indicators ("In stock: 47 copies")

Scanner Enhancements

  • Rotation correction for tilted scans
  • Multi-card detection (scan full hand at once)
  • Set symbol recognition (bypass text OCR entirely)
  • Bulk mode (rapid-fire scanning for collection input)

Database Improvements

  • Database name search (third hybrid dimension)
  • Perceptual hashing (pre-compute art signatures for speed)
  • Commander-specific fields (show if card is commander-legal)

Try It Now

Navigate to /scan or click "📸 Scan" in the header. Upload a card photo or use your phone camera. Toggle debug mode to see how the hybrid search works. Check if prices appear for your cards.

The system works best with:

  • Clear lighting (avoid glare on foils)
  • Card name visible (OCR needs readable text)
  • Centered framing (art region at standard position)

For cards without local images, art matching falls back gracefully to text-only scoring—you'll still get correct results, just without the art-confirmed confidence boost.

Technical Debt Paid

This wasn't just feature work—we fixed underlying architecture:

  1. Type safety: Removed any usage, added strict CardMatch interface
  2. Error handling: CORS failures now silent (no console spam)
  3. Performance: Local cache eliminates 110k potential network requests
  4. Maintainability: Debug logging makes future improvements easier
  5. Documentation: Comprehensive knowledge log in AGENTS.md

Why This Approach Works

Hybrid search mirrors how humans identify cards: we read the name (OCR), recognize the art (similarity), and check the price (market data). By combining all three signals and weighting them intelligently, the scanner achieves better-than-human accuracy at machine speed.

The key insight: no single method is perfect, but voting systems are robust. When text OCR fails (short names, glare), art similarity rescues it. When art comparison fails (CORS, missing images), text matching carries the load. Pricing data adds instant value without slowing anything down.

Bottom line: Scan cards as fast as you can flip them. Get accurate printings, instant pricing, and one-click purchasing. No typing, no guessing, no network delays.