scry/AGENTS.md

Agent Instructions

Overview

Scry is a Magic: The Gathering card scanner app built with Expo (React Native) and Convex. It uses perceptual hashing to match photographed cards against a database of known card images from Scryfall.

Tech Stack:

• Frontend: Expo/React Native with Expo Router (file-based routing)
• Backend: Convex (serverless functions + real-time database)
• Image Processing: React Native Skia, fast-opencv
• Camera: Adaptive (expo-camera in dev, react-native-vision-camera in production)
• Auth: Convex Auth with Zitadel OIDC (GDPR-compliant, no PII stored)
• Package Manager: Bun (not npm/yarn)

Build Commands

Use just commands (defined in .justfile):

Task	Command	Notes
Start dev server	just dev	Runs Convex + Expo together
Expo only	just start	Just the Expo dev server
Convex only	just convex-dev	Just the Convex dev server
Run on Android	just android	Starts Android emulator
Install deps	just expo-install	Runs bun install
Migrate hashes	just expo-migrate	Migrate card hashes to Convex
Type check	just expo-typecheck	TypeScript check
Start emulator	just emu	Virtual camera (submodule)

Direct Bun Commands

bun install                    # Install dependencies
bun run dev                    # Convex + Expo hot reload
bun run dev:convex             # Convex dev server only
bun run dev:expo               # Expo dev server only
bun run android                # Run on Android
bun run migrate                # Migrate hashes to Convex
bun run typecheck              # TypeScript check
bunx convex dev                # Convex CLI

Project Structure

app/                           # Expo Router pages
├── _layout.tsx                # Root layout (Convex + HashCache providers)
├── modal.tsx                  # Card details modal
├── +not-found.tsx             # 404 page
└── (tabs)/                    # Tab navigation group
    ├── _layout.tsx            # Tab bar layout
    ├── index.tsx              # Collection tab (home)
    ├── scan.tsx               # Camera scan tab
    └── settings.tsx           # Settings tab

components/
├── camera/                    # Adaptive camera system
│   ├── index.tsx              # AdaptiveCamera wrapper
│   ├── ExpoCamera.tsx         # expo-camera (Expo Go)
│   └── VisionCamera.tsx       # react-native-vision-camera (production)
└── *.tsx                      # Shared UI components

convex/                        # Backend (Convex functions + schema)
├── schema.ts                  # Database schema
├── auth.ts                    # Zitadel OIDC configuration
├── http.ts                    # HTTP endpoints for auth
├── cards.ts                   # Card queries/mutations
├── collections.ts             # User collection functions
├── users.ts                   # User functions
├── scanHistory.ts             # Scan history functions
└── _generated/                # Auto-generated types

lib/
├── recognition/               # Card recognition pipeline
│   ├── recognitionService.ts  # Main recognition logic
│   ├── cardDetector.ts        # Edge detection, find card quad
│   ├── perspectiveCorrection.ts # Warp to rectangle
│   ├── clahe.ts               # CLAHE lighting normalization
│   ├── perceptualHash.ts      # 192-bit color hash (24 bytes)
│   ├── imageUtils.ts          # Resize, rotate, grayscale
│   ├── imageLoader.ts         # Load/resize images
│   └── skiaDecoder.ts         # Decode images with Skia
├── hooks/                     # React hooks
│   ├── useAuth.ts             # OAuth flow with expo-auth-session
│   ├── useCamera.ts           # Adaptive camera permissions
│   ├── useConvex.ts           # Convex data hooks
│   ├── useSync.ts             # Hash sync hook
│   └── useUserProfile.ts      # Fetch profile from Zitadel
├── context/                   # React contexts
│   └── HashCacheContext.tsx   # In-memory hash cache
└── db/                        # Local database utilities
    ├── localDatabase.ts       # SQLite wrapper
    └── syncService.ts         # Sync with Convex

scripts/
└── migrate-hashes.ts          # Migration script

TestImages/                    # Test images (225 files)

Architecture

Recognition Pipeline

Camera Image
     │
     ▼
┌─────────────────────┐
│ loadImageAsBase64   │  ← Resize to 480×640
└─────────────────────┘
     │
     ▼
┌─────────────────────┐
│ decodeImageBase64   │  ← Skia decodes to RGBA pixels
└─────────────────────┘
     │
     ▼
┌─────────────────────┐
│ detectCard          │  ← Edge detection, find card quad
│ (optional)          │
└─────────────────────┘
     │
     ▼
┌─────────────────────┐
│ warpPerspective     │  ← Warp detected quad to rectangle
└─────────────────────┘
     │
     ▼
┌─────────────────────┐
│ applyCLAHE          │  ← Lighting normalization
└─────────────────────┘
     │
     ▼
┌─────────────────────┐
│ computeColorHash    │  ← Compute 192-bit color hash (24 bytes)
└─────────────────────┘
     │
     ▼
┌─────────────────────┐
│ recognizeCard       │  ← Hamming distance match against cache
└─────────────────────┘

Data Model (Convex Schema)

Table	Purpose
users	Minimal auth (no PII, GDPR compliant)
cards	Card printings with 24-byte perceptual hashes
oracles	Abstract game cards (one per unique card name)
sets	MTG sets with metadata
collections	User card collections
scanHistory	Scan history with confidence scores
metadata	Sync metadata

GDPR Compliance

• Database stores no user PII - only auth subject ID
• User profile (name, email, image) fetched from Zitadel userinfo endpoint on demand
• Profile held in memory only, never persisted

Adaptive Camera System

The app detects its runtime environment and uses the appropriate camera:

• Expo Go (Constants.appOwnership === "expo"): Uses expo-camera
• Production builds: Uses react-native-vision-camera

Both expose the same CameraHandle interface with takePhoto().

Key Algorithms

Perceptual Hash (pHash)

Color-aware 192-bit hash:

1. Resize to 32×32
2. For each RGB channel:
   • Compute 2D DCT
   • Extract 8×8 low-frequency coefficients (skip DC)
   • Compare each to median → 63 bits per channel
3. Concatenate R, G, B hashes → 24 bytes (192 bits)

Matching uses Hamming distance with threshold ≤25 bits and minimum confidence 85%.

CLAHE (Contrast Limited Adaptive Histogram Equalization)

Applied in LAB color space to L channel only:

• Tile-based histogram equalization (8×8 tiles)
• Clip limit prevents over-amplification of noise
• Bilinear interpolation between tiles for smooth output

Code Conventions

General

• TypeScript: Strict mode enabled
• Formatting: Prettier defaults
• Package Manager: Bun (never use npm/yarn)

React/React Native

• Functional components with hooks
• Expo Router for navigation (file-based)
• Convex hooks for data (useQuery, useMutation)

Naming

• Hooks: useCardHashes, useCameraPermission
• Components: PascalCase (AdaptiveCamera, ScanScreen)
• Files: camelCase for modules, PascalCase for components
• Convex functions: camelCase (cards.ts, getByScryfallId)

Convex Backend

• Queries are reactive and cached
• Mutations are transactional
• Use v. validators for all arguments
• Index frequently queried fields

Environment Variables

Convex Backend (convex/.env.local)

AUTH_ZITADEL_ID=your-client-id
AUTH_ZITADEL_SECRET=your-client-secret
AUTH_ZITADEL_ISSUER=https://your-zitadel-instance

Client Side (.env)

EXPO_PUBLIC_CONVEX_URL=https://your-deployment.convex.cloud
EXPO_PUBLIC_ZITADEL_ISSUER=https://your-zitadel-instance

Common Tasks

Adding a New Convex Function

1. Add function to appropriate file in convex/ (e.g., cards.ts)
2. Run bunx convex dev to regenerate types
3. Import from convex/_generated/api in client code

Testing Recognition

1. Add test images to TestImages/
2. Use the scan tab in the app
3. Check console logs for [Scry] prefixed messages

Debugging Camera Issues

• In Expo Go: Uses expo-camera, check for "Dev mode" indicator
• In production: Uses Vision Camera, requires EAS build

Dependencies

Core

• expo ~54.0.33
• expo-router ~6.0.23
• convex ^1.31.7
• @convex-dev/auth ^0.0.90
• react 19.1.0
• react-native 0.81.5

Image Processing

• @shopify/react-native-skia ^2.4.18
• react-native-fast-opencv ^0.4.7

Camera

• expo-camera ^17.0.10 (Expo Go)
• react-native-vision-camera ^4.7.3 (production)

Auth

• expo-auth-session ^7.0.10
• expo-secure-store ^15.0.8

External Resources

• Scryfall API - Card data source
• Convex Docs - Backend documentation
• Expo Router - Navigation
• docs/CARD_RECOGNITION.md - Recognition architecture