🎬 Annotation Tool Overview

Professional Video Annotation for Tennis AI Training

AceSense Annotate is a cross-platform desktop application for creating high-quality training data for tennis AI models. Built with Tauri + React for native performance.

---

🎯 Purpose

:::info Why This Exists

• Training Data: Create labeled datasets for shot detection and event classification (point, box, and temporal-event annotations)
• Speed: Professional annotators need keyboard-first workflows :::

---

🛠️ Tech Stack

Technology	Version	Purpose
🦀 Tauri	2.0+	Desktop shell (frameless window, fs/dialog/opener plugins). No decoding/persistence logic lives in Rust.
⚛️ React	18.3	UI framework — owns video, canvas overlay, annotation state, persistence
⚡ Vite	6.4	Build tool
📘 TypeScript	5.6	Type safety
🎬 @ffmpeg/ffmpeg	0.12	Client-side WASM format conversion (MP4 fallback when a video won't play) — not Rust decoding
📈 PostHog	—	Product analytics

---

🏗️ Architecture

The Rust backend (src-tauri/src/lib.rs) only wires the shell: frameless window setup plus fs/dialog/opener plugins. Video playback, the Canvas2D annotation overlay, WASM format conversion, and project persistence (via @tauri-apps/plugin-fs) all run in the React frontend. The app also runs in a plain browser (web mode) with a file-picker fallback.

---

📂 Project Structure

acesense-annotate/
├── src/                      # React frontend
│   ├── App.tsx              # Main workspace
│   ├── components/
│   │   ├── VideoViewport.tsx   # <video> + Canvas2D annotation overlay
│   │   ├── Timeline.tsx        # Frame scrubber
│   │   ├── PropertiesPanel.tsx # Property editor
│   │   ├── ItemsPanel.tsx      # Annotation list
│   │   ├── ContextModal.tsx    # Tennis context picker
│   │   ├── ShortcutModal.tsx   # In-app shortcut cheat sheet (`?`)
│   │   ├── ErrorBoundary.tsx   # Crash guard
│   │   └── VideoConverter.tsx  # @ffmpeg/ffmpeg WASM converter
│   ├── types.ts             # TypeScript types (Tool, Annotation, TemporalEvent, ProjectData)
│   └── utils.ts             # Helpers (allowed-events, fps detection)
├── src-tauri/               # Tauri shell
│   ├── src/lib.rs           # Plugin registration + frameless window setup
│   ├── Cargo.toml           # Dependencies
│   └── tauri.conf.json      # App config
├── public/                  # Static assets
└── package.json

---

🎮 Keyboard Shortcuts

Playback Controls

Key	Action
Space	Play / Pause
← / →	Frame step
Shift + ← / →	Jump 10 frames
J / K / L	Reverse / Pause / Forward (J/L ramp 2×→4× on repeat)

Annotation Tools

Key	Tool	Behavior
P	Point	Click to place a point (x, y, time)
B	Box	Click-drag to draw (x1, y1, x2, y2, time)
T	Temporal Event	Activate event tool, or close the open event
1–6	Set event type	serve / shot / bounce / point_end / net_hit / out (selected event)
Esc	Select / deselect	Back to select tool
?	Help	Toggle the shortcut cheat sheet

See the full Keyboard Reference. There is no skeleton tool or ML-suggestion review.

---

🎾 Tennis Context System

Before annotating, select the tennis context to unlock appropriate tools:

Context	Allowed Annotations
Match	serve, shot, bounce, net_hit, out, point_end
Practice	shot, bounce
Training (off-court)	shot only
Highlight clip	shot, point_end

---

📋 Annotation Schema

The persisted shapes come from src/types.ts. There is no pose/skeleton schema.

Temporal Event

{
  "id": "evt_014",
  "type": "event",
  "eventType": "shot",
  "startTime": 12.21,
  "endTime": 12.94,
  "shotType": "forehand",
  "confidence": 0.88
}

eventType ∈ serve | shot | bounce | net_hit | out | point_end. shotType (forehand | backhand | unknown) and confidence are optional. endTime is null while an event is open.

Point / Box annotations

{ "id": "pt_001",  "type": "point", "time": 3.417, "x": 0.51, "y": 0.18 }
{ "id": "box_002", "type": "box",   "time": 4.000, "x1": 0.40, "y1": 0.30, "x2": 0.62, "y2": 0.70 }

The full saved file is a ProjectData object (video, fps, video_meta, events, annotations).

---

🚀 Development

# Install dependencies
pnpm install

# Start dev server (web preview, runs in a plain browser too)
pnpm dev

# Run desktop app
pnpm tauri dev

# Type-check / test
pnpm typecheck
pnpm test

# Build production app
pnpm tauri build

---

🔗 Related

• GPU Backend - Uses annotated data for training
• Frontend - Displays AI analysis results