scry/AGENTS.md

# Agent Instructions

## Overview

Scry is a Magic: The Gathering card scanner app built with Expo (React Native) and Convex. It uses perceptual hashing to match photographed cards against a database of known card images from Scryfall.

**Tech Stack:**
- **Frontend**: Expo/React Native with Expo Router (file-based routing)
- **Backend**: Convex (serverless functions + real-time database)
- **Image Processing**: React Native Skia, fast-opencv
- **Camera**: Adaptive (expo-camera in dev, react-native-vision-camera in production)
- **Auth**: Convex Auth with Zitadel OIDC (GDPR-compliant, no PII stored)
- **Package Manager**: Bun (not npm/yarn)

## Build Commands

Use `just` commands (defined in `.justfile`):

| Task | Command | Notes |
|------|---------|-------|
| Start dev server | `just dev` | Runs Convex + Expo together |
| Expo only | `just start` | Just the Expo dev server |
| Convex only | `just convex-dev` | Just the Convex dev server |
| Run on Android | `just android` | Starts Android emulator |
| Install deps | `just expo-install` | Runs `bun install` |
| Migrate hashes | `just expo-migrate` | Migrate card hashes to Convex |
| Type check | `just expo-typecheck` | TypeScript check |
| Start emulator | `just emu` | Virtual camera (submodule) |

### Direct Bun Commands

```bash
bun install                    # Install dependencies
bun run dev                    # Convex + Expo hot reload
bun run dev:convex             # Convex dev server only
bun run dev:expo               # Expo dev server only
bun run android                # Run on Android
bun run migrate                # Migrate hashes to Convex
bun run typecheck              # TypeScript check
bunx convex dev                # Convex CLI
```

## Project Structure

```
app/                           # Expo Router pages
├── _layout.tsx                # Root layout (Convex + HashCache providers)
├── modal.tsx                  # Card details modal
├── +not-found.tsx             # 404 page
└── (tabs)/                    # Tab navigation group
    ├── _layout.tsx            # Tab bar layout
    ├── index.tsx              # Collection tab (home)
    ├── scan.tsx               # Camera scan tab
    └── settings.tsx           # Settings tab

components/
├── camera/                    # Adaptive camera system
│   ├── index.tsx              # AdaptiveCamera wrapper
│   ├── ExpoCamera.tsx         # expo-camera (Expo Go)
│   └── VisionCamera.tsx       # react-native-vision-camera (production)
└── *.tsx                      # Shared UI components

convex/                        # Backend (Convex functions + schema)
├── schema.ts                  # Database schema
├── auth.ts                    # Zitadel OIDC configuration
├── http.ts                    # HTTP endpoints for auth
├── cards.ts                   # Card queries/mutations
├── collections.ts             # User collection functions
├── users.ts                   # User functions
├── scanHistory.ts             # Scan history functions
└── _generated/                # Auto-generated types

lib/
├── recognition/               # Card recognition pipeline
│   ├── recognitionService.ts  # Main recognition logic
│   ├── cardDetector.ts        # Edge detection, find card quad
│   ├── perspectiveCorrection.ts # Warp to rectangle
│   ├── clahe.ts               # CLAHE lighting normalization
│   ├── perceptualHash.ts      # 192-bit color hash (24 bytes)
│   ├── imageUtils.ts          # Resize, rotate, grayscale
│   ├── imageLoader.ts         # Load/resize images
│   └── skiaDecoder.ts         # Decode images with Skia
├── hooks/                     # React hooks
│   ├── useAuth.ts             # OAuth flow with expo-auth-session
│   ├── useCamera.ts           # Adaptive camera permissions
│   ├── useConvex.ts           # Convex data hooks
│   ├── useSync.ts             # Hash sync hook
│   └── useUserProfile.ts      # Fetch profile from Zitadel
├── context/                   # React contexts
│   └── HashCacheContext.tsx   # In-memory hash cache
└── db/                        # Local database utilities
    ├── localDatabase.ts       # SQLite wrapper
    └── syncService.ts         # Sync with Convex

scripts/
└── migrate-hashes.ts          # Migration script

TestImages/                    # Test images (225 files)
```

## Architecture

### Recognition Pipeline

```
Camera Image
     │
     ▼
┌─────────────────────┐
│ loadImageAsBase64   │  ← Resize to 480×640
└─────────────────────┘
     │
     ▼
┌─────────────────────┐
│ decodeImageBase64   │  ← Skia decodes to RGBA pixels
└─────────────────────┘
     │
     ▼
┌─────────────────────┐
│ detectCard          │  ← Edge detection, find card quad
│ (optional)          │
└─────────────────────┘
     │
     ▼
┌─────────────────────┐
│ warpPerspective     │  ← Warp detected quad to rectangle
└─────────────────────┘
     │
     ▼
┌─────────────────────┐
│ applyCLAHE          │  ← Lighting normalization
└─────────────────────┘
     │
     ▼
┌─────────────────────┐
│ computeColorHash    │  ← Compute 192-bit color hash (24 bytes)
└─────────────────────┘
     │
     ▼
┌─────────────────────┐
│ recognizeCard       │  ← Hamming distance match against cache
└─────────────────────┘
```

### Data Model (Convex Schema)

| Table | Purpose |
|-------|---------|
| `users` | Minimal auth (no PII, GDPR compliant) |
| `cards` | Card printings with 24-byte perceptual hashes |
| `oracles` | Abstract game cards (one per unique card name) |
| `sets` | MTG sets with metadata |
| `collections` | User card collections |
| `scanHistory` | Scan history with confidence scores |
| `metadata` | Sync metadata |

### GDPR Compliance

- Database stores **no user PII** - only auth subject ID
- User profile (name, email, image) fetched from Zitadel userinfo endpoint on demand
- Profile held in memory only, never persisted

### Adaptive Camera System

The app detects its runtime environment and uses the appropriate camera:

- **Expo Go** (`Constants.appOwnership === "expo"`): Uses `expo-camera`
- **Production builds**: Uses `react-native-vision-camera`

Both expose the same `CameraHandle` interface with `takePhoto()`.

## Key Algorithms

### Perceptual Hash (pHash)

Color-aware 192-bit hash:
1. Resize to 32×32
2. For each RGB channel:
   - Compute 2D DCT
   - Extract 8×8 low-frequency coefficients (skip DC)
   - Compare each to median → 63 bits per channel
3. Concatenate R, G, B hashes → 24 bytes (192 bits)

Matching uses Hamming distance with threshold ≤25 bits and minimum confidence 85%.

### CLAHE (Contrast Limited Adaptive Histogram Equalization)

Applied in LAB color space to L channel only:
- Tile-based histogram equalization (8×8 tiles)
- Clip limit prevents over-amplification of noise
- Bilinear interpolation between tiles for smooth output

## Code Conventions

### General

- **TypeScript**: Strict mode enabled
- **Formatting**: Prettier defaults
- **Package Manager**: Bun (never use npm/yarn)

### React/React Native

- Functional components with hooks
- Expo Router for navigation (file-based)
- Convex hooks for data (`useQuery`, `useMutation`)

### Naming

- Hooks: `useCardHashes`, `useCameraPermission`
- Components: PascalCase (`AdaptiveCamera`, `ScanScreen`)
- Files: camelCase for modules, PascalCase for components
- Convex functions: camelCase (`cards.ts`, `getByScryfallId`)

### Convex Backend

- Queries are reactive and cached
- Mutations are transactional
- Use `v.` validators for all arguments
- Index frequently queried fields

## Environment Variables

### Convex Backend (`convex/.env.local`)

```bash
AUTH_ZITADEL_ID=your-client-id
AUTH_ZITADEL_SECRET=your-client-secret
AUTH_ZITADEL_ISSUER=https://your-zitadel-instance
```

### Client Side (`.env`)

```bash
EXPO_PUBLIC_CONVEX_URL=https://your-deployment.convex.cloud
EXPO_PUBLIC_ZITADEL_ISSUER=https://your-zitadel-instance
```

## Common Tasks

### Adding a New Convex Function

1. Add function to appropriate file in `convex/` (e.g., `cards.ts`)
2. Run `bunx convex dev` to regenerate types
3. Import from `convex/_generated/api` in client code

### Testing Recognition

1. Add test images to `TestImages/`
2. Use the scan tab in the app
3. Check console logs for `[Scry]` prefixed messages

### Debugging Camera Issues

- In Expo Go: Uses `expo-camera`, check for "Dev mode" indicator
- In production: Uses Vision Camera, requires EAS build

## Dependencies

### Core

- `expo` ~54.0.33
- `expo-router` ~6.0.23
- `convex` ^1.31.7
- `@convex-dev/auth` ^0.0.90
- `react` 19.1.0
- `react-native` 0.81.5

### Image Processing

- `@shopify/react-native-skia` ^2.4.18
- `react-native-fast-opencv` ^0.4.7

### Camera

- `expo-camera` ^17.0.10 (Expo Go)
- `react-native-vision-camera` ^4.7.3 (production)

### Auth

- `expo-auth-session` ^7.0.10
- `expo-secure-store` ^15.0.8

## External Resources

- [Scryfall API](https://scryfall.com/docs/api) - Card data source
- [Convex Docs](https://docs.convex.dev/) - Backend documentation
- [Expo Router](https://docs.expo.dev/router/introduction/) - Navigation
- [docs/CARD_RECOGNITION.md](docs/CARD_RECOGNITION.md) - Recognition architecture