Add debug output support and refactor DbGenerator CLI
- Add RecognitionOptions with DebugOutputDirectory for saving pipeline
stages (input, detection, perspective correction, CLAHE preprocessing)
- Wire up IOptions<RecognitionOptions> via DI in MauiProgram
- Extract GenerateCommand from Program.cs using Spectre.Console.Cli
- Add priority card support with preferred set matching (Alpha/Beta)
- Expand card_hashes.db with more cards for better recognition coverage
- Update AGENTS.md with comprehensive project documentation
💘 Generated with Crush
Assisted-by: Claude Opus 4.5 via Crush <crush@charm.land>
This commit is contained in:
Chris Kruining
parent
54ba7496c6
commit
56499d5af9
14 changed files with 1010 additions and 456 deletions
301
AGENTS.md
301
AGENTS.md
|
|
@ -1,30 +1,291 @@
|
|||
# Agent Instructions
|
||||
|
||||
## Build and Task Commands
|
||||
## Overview
|
||||
|
||||
Prefer using `just` commands over raw dotnet/build commands:
|
||||
Scry is a Magic: The Gathering card scanner app built with .NET MAUI. It uses perceptual hashing to match photographed cards against a database of known card images from Scryfall.
|
||||
|
||||
| Task | Command |
|
||||
|------|---------|
|
||||
| Build project | `just build` |
|
||||
| Run tests | `just test` |
|
||||
| Generate card database | `just gen-db` |
|
||||
| Publish app | `just publish` |
|
||||
| Run full dev workflow | `just dev` |
|
||||
**Key components:**
|
||||
- Mobile scanning app (MAUI/Android)
|
||||
- Card recognition via perceptual hashing (not OCR)
|
||||
- SQLite database with pre-computed hashes
|
||||
- Scryfall API integration for card data
|
||||
|
||||
## Build Commands
|
||||
|
||||
Use `just` commands (defined in `.justfile`):
|
||||
|
||||
| Task | Command | Notes |
|
||||
|------|---------|-------|
|
||||
| Build project | `just build` | Builds Android debug |
|
||||
| Run tests | `just test` | Runs xUnit tests |
|
||||
| Generate card database | `just gen-db` | Downloads from Scryfall, computes hashes |
|
||||
| Publish app | `just publish` | Creates release APK |
|
||||
| Hot reload dev | `just dev` | Uses `dotnet watch` |
|
||||
| Start emulator | `just emu` | Virtual camera with Serra Angel |
|
||||
| Install to device | `just install` | Installs release APK |
|
||||
|
||||
### Database Generator Options
|
||||
|
||||
```bash
|
||||
just gen-db # Default: 500 cards with test images
|
||||
dotnet run --project tools/DbGenerator -- -c 1000 # More cards
|
||||
dotnet run --project tools/DbGenerator -- --force # Rebuild from scratch
|
||||
dotnet run --project tools/DbGenerator -- --no-test-cards # Skip priority test cards
|
||||
```
|
||||
|
||||
## Project Structure
|
||||
|
||||
- `src/Scry.App` - MAUI mobile app
|
||||
- `src/Scry.Core` - Core library (recognition, hashing, database)
|
||||
- `test/Scry.Tests` - Unit tests
|
||||
- `tools/DbGenerator` - Card hash database generator
|
||||
```
|
||||
src/
|
||||
├── Scry.App/ # MAUI mobile app (Android target)
|
||||
│ ├── Views/ # XAML pages (ScanPage, CollectionPage, etc.)
|
||||
│ ├── ViewModels/ # MVVM ViewModels using CommunityToolkit.Mvvm
|
||||
│ ├── Services/ # App-layer services (ICardRecognitionService, ICardRepository)
|
||||
│ ├── Converters/ # XAML value converters
|
||||
│ ├── Models/ # App-specific models (CollectionEntry)
|
||||
│ └── Resources/Raw/ # Bundled card_hashes.db
|
||||
│
|
||||
└── Scry.Core/ # Platform-independent core library
|
||||
├── Recognition/ # CardRecognitionService, RecognitionOptions
|
||||
├── Imaging/ # PerceptualHash, ImagePreprocessor, CardDetector
|
||||
├── Data/ # CardDatabase (SQLite)
|
||||
├── Models/ # Card, Oracle, Set, ScanResult
|
||||
└── Scryfall/ # ScryfallClient for API/bulk data
|
||||
|
||||
## Database Generator
|
||||
test/
|
||||
└── Scry.Tests/ # xUnit tests
|
||||
|
||||
The `just gen-db` command:
|
||||
- Builds the DbGenerator tool
|
||||
- Runs it against `src/Scry.App/Resources/Raw/card_hashes.db`
|
||||
- Supports incremental updates (only downloads missing cards)
|
||||
- Prefers LEA/LEB (Alpha/Beta) sets for reference_alpha test cards
|
||||
tools/
|
||||
└── DbGenerator/ # CLI tool to generate card_hashes.db
|
||||
|
||||
Use `--force` flag to rebuild from scratch if needed.
|
||||
TestImages/ # Test images organized by category
|
||||
├── reference_alpha/ # Alpha/Beta cards for testing
|
||||
├── single_cards/ # Individual card photos
|
||||
├── varying_quality/ # Different lighting/quality
|
||||
├── hands/ # Cards held in hand
|
||||
├── foil/ # Foil cards with glare
|
||||
└── ... # More categories
|
||||
```
|
||||
|
||||
## Architecture
|
||||
|
||||
### Recognition Pipeline
|
||||
|
||||
```
|
||||
Camera Image
|
||||
│
|
||||
▼
|
||||
┌─────────────────────┐
|
||||
│ CardDetector │ ← Edge detection, find card quad
|
||||
│ (optional) │
|
||||
└─────────────────────┘
|
||||
│
|
||||
▼
|
||||
┌─────────────────────┐
|
||||
│ PerspectiveCorrection│ ← Warp to rectangle
|
||||
└─────────────────────┘
|
||||
│
|
||||
▼
|
||||
┌─────────────────────┐
|
||||
│ ImagePreprocessor │ ← CLAHE for lighting normalization
|
||||
│ (ApplyClahe) │
|
||||
└─────────────────────┘
|
||||
│
|
||||
▼
|
||||
┌─────────────────────┐
|
||||
│ PerceptualHash │ ← Compute 192-bit color hash (24 bytes)
|
||||
│ (ComputeColorHash) │
|
||||
└─────────────────────┘
|
||||
│
|
||||
▼
|
||||
┌─────────────────────┐
|
||||
│ CardRecognitionService│ ← Hamming distance match against DB
|
||||
└─────────────────────┘
|
||||
```
|
||||
|
||||
### Data Model
|
||||
|
||||
Three-table schema mirroring Scryfall:
|
||||
|
||||
- **oracles** - Abstract game cards (one per unique card name)
|
||||
- **sets** - MTG sets with metadata
|
||||
- **cards** - Printings with perceptual hashes (one per unique artwork)
|
||||
|
||||
The `Card` model includes denormalized Oracle fields for convenience.
|
||||
|
||||
### Key Classes
|
||||
|
||||
| Class | Purpose |
|
||||
|-------|---------|
|
||||
| `CardRecognitionService` | Main recognition logic, caches DB, handles rotation matching |
|
||||
| `PerceptualHash` | DCT-based color hash (192-bit = 8 bytes × 3 RGB channels) |
|
||||
| `ImagePreprocessor` | CLAHE, resize, grayscale conversions |
|
||||
| `CardDetector` | Edge detection + contour analysis to find card boundaries |
|
||||
| `PerspectiveCorrection` | Warp detected quad to rectangle |
|
||||
| `CardDatabase` | SQLite wrapper with batch insert, queries |
|
||||
| `ScryfallClient` | Bulk data streaming, image downloads |
|
||||
|
||||
## Code Conventions
|
||||
|
||||
### General
|
||||
|
||||
- **Target**: .NET 10.0 (net10.0-android for app, net10.0 for Core/tools)
|
||||
- **Nullable**: Enabled everywhere (`<Nullable>enable</Nullable>`)
|
||||
- **Warnings as errors**: `<TreatWarningsAsErrors>true</TreatWarningsAsErrors>`
|
||||
- **Central package management**: Versions in `Directory.Packages.props`
|
||||
|
||||
### C# Style
|
||||
|
||||
- Records for data models (`record Card`, `record ScanResult`)
|
||||
- `required` properties for non-nullable required fields
|
||||
- Extension methods for conversions (`ScryfallCard.ToCard()`)
|
||||
- Static classes for pure functions (`PerceptualHash`, `ImagePreprocessor`)
|
||||
- `using` declarations (not `using` blocks) for disposables
|
||||
- File-scoped namespaces
|
||||
- Primary constructors where appropriate
|
||||
- `CancellationToken` parameter on all async methods
|
||||
|
||||
### MVVM (App layer)
|
||||
|
||||
- `CommunityToolkit.Mvvm` for source generators
|
||||
- `[ObservableProperty]` attributes for bindable properties
|
||||
- `[RelayCommand]` for commands
|
||||
- ViewModels in `Scry.ViewModels` namespace
|
||||
|
||||
### Naming
|
||||
|
||||
- Services: `ICardRecognitionService`, `CardRecognitionService`
|
||||
- Database methods: `GetCardsWithHashAsync`, `InsertCardBatchAsync`
|
||||
- Hash methods: `ComputeColorHash`, `HammingDistance`
|
||||
- Test methods: `RecognizeAsync_ExactMatch_ReturnsSuccess`
|
||||
|
||||
## Testing
|
||||
|
||||
Tests are in `test/Scry.Tests` using xUnit.
|
||||
|
||||
```bash
|
||||
just test # Run all tests
|
||||
dotnet test --filter "FullyQualifiedName~PerceptualHash" # Filter by name
|
||||
```
|
||||
|
||||
### Test Categories
|
||||
|
||||
| Test Class | Tests |
|
||||
|------------|-------|
|
||||
| `PerceptualHashTests` | Hash computation, Hamming distance |
|
||||
| `CardRecognitionTests` | End-to-end recognition |
|
||||
| `CardDatabaseTests` | SQLite CRUD operations |
|
||||
| `ImagePreprocessorTests` | CLAHE, resize |
|
||||
| `RobustnessAnalysisTests` | Multiple image variations |
|
||||
|
||||
### Test Images
|
||||
|
||||
TestImages directory contains categorized reference images:
|
||||
- `reference_alpha/` - Alpha/Beta cards (matching DbGenerator priority cards)
|
||||
- `single_cards/` - Clean single card photos
|
||||
- `varying_quality/` - Different lighting/blur conditions
|
||||
|
||||
## Key Algorithms
|
||||
|
||||
### Perceptual Hash (pHash)
|
||||
|
||||
Color-aware 192-bit hash:
|
||||
1. Resize to 32×32
|
||||
2. For each RGB channel:
|
||||
- Compute 2D DCT
|
||||
- Extract 8×8 low-frequency coefficients (skip DC)
|
||||
- Compare each to median → 63 bits per channel
|
||||
3. Concatenate R, G, B hashes → 24 bytes (192 bits)
|
||||
|
||||
Matching uses Hamming distance with threshold ≤25 bits and minimum confidence 85%.
|
||||
|
||||
### CLAHE (Contrast Limited Adaptive Histogram Equalization)
|
||||
|
||||
Applied in LAB color space to L channel only:
|
||||
- Tile-based histogram equalization (8×8 tiles)
|
||||
- Clip limit prevents over-amplification of noise
|
||||
- Bilinear interpolation between tiles for smooth output
|
||||
|
||||
### Card Detection
|
||||
|
||||
Pure SkiaSharp implementation:
|
||||
1. Grayscale → Gaussian blur → Canny edge detection
|
||||
2. Contour tracing via flood fill
|
||||
3. Douglas-Peucker simplification → Convex hull
|
||||
4. Find best quadrilateral matching MTG aspect ratio (88/63 ≈ 1.397)
|
||||
5. Order corners: top-left, top-right, bottom-right, bottom-left
|
||||
|
||||
## Debug Mode
|
||||
|
||||
Set `RecognitionOptions.DebugOutputDirectory` to save pipeline stages:
|
||||
- `01_input.png` - Original image
|
||||
- `02_detection.png` - Card detection visualization
|
||||
- `03_perspective_corrected.png` - Warped card
|
||||
- `05_clahe_*.png` - After CLAHE preprocessing
|
||||
|
||||
On Android: `/sdcard/Download/scry-debug` (pull with `adb pull`)
|
||||
|
||||
## Dependencies
|
||||
|
||||
### Core Library (Scry.Core)
|
||||
|
||||
- **SkiaSharp** - Image processing, DCT, edge detection
|
||||
- **Microsoft.Data.Sqlite** - SQLite database
|
||||
- **Microsoft.Extensions.Options** - Options pattern
|
||||
|
||||
### App (Scry.App)
|
||||
|
||||
- **CommunityToolkit.Maui** - MAUI extensions
|
||||
- **CommunityToolkit.Maui.Camera** - Camera integration
|
||||
- **CommunityToolkit.Mvvm** - MVVM source generators
|
||||
|
||||
### DbGenerator Tool
|
||||
|
||||
- **Spectre.Console** / **Spectre.Console.Cli** - Rich terminal UI
|
||||
|
||||
## Common Tasks
|
||||
|
||||
### Adding a New Card to Priority Test Set
|
||||
|
||||
1. Add image to `TestImages/reference_alpha/` or appropriate folder
|
||||
2. Add entry to `GenerateCommand.PriorityCardsWithSets` dictionary
|
||||
3. Run `just gen-db` to regenerate database
|
||||
|
||||
### Debugging Recognition Failures
|
||||
|
||||
1. Enable debug output in `MauiProgram.cs`:
|
||||
```csharp
|
||||
options.DebugOutputDirectory = "/sdcard/Download/scry-debug";
|
||||
```
|
||||
2. Run recognition
|
||||
3. Pull debug images: `adb pull /sdcard/Download/scry-debug`
|
||||
4. Compare `05_clahe_*.png` with reference images in database
|
||||
|
||||
### Modifying Hash Algorithm
|
||||
|
||||
1. Update `PerceptualHash.ComputeColorHash()`
|
||||
2. Update `CardRecognitionService.ColorHashBits` constant
|
||||
3. Regenerate database: `just gen-db --force`
|
||||
4. Run tests: `just test`
|
||||
|
||||
## Gotchas
|
||||
|
||||
1. **Hash size is 24 bytes (192 bits)** - 3 RGB channels × 8 bytes each
|
||||
2. **Confidence threshold is 85%** - Configurable in `CardRecognitionService.MinConfidence`
|
||||
3. **Card detection is optional** - Controlled by `RecognitionOptions.EnableCardDetection`
|
||||
4. **Rotation matching tries 4 orientations** - Controlled by `RecognitionOptions.EnableRotationMatching`
|
||||
5. **Database is bundled in APK** - Copied on first run to app data directory
|
||||
6. **Multi-face cards** - Only front face image is used for hashing
|
||||
7. **Rate limiting** - DbGenerator uses 50ms delay between Scryfall image downloads
|
||||
|
||||
## CI/CD
|
||||
|
||||
Forgejo Actions workflow (`.forgejo/workflows/release.yml`):
|
||||
- Builds for win-x64, linux-x64, osx-x64
|
||||
- Creates "standard" and "embedded" (with APK) variants
|
||||
- Publishes to Forgejo releases
|
||||
|
||||
## External Resources
|
||||
|
||||
- [Scryfall API](https://scryfall.com/docs/api) - Card data source
|
||||
- [CARD_RECOGNITION.md](docs/CARD_RECOGNITION.md) - Detailed architecture doc
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue