Skip to main content

🤖 GPU Backend Setup Guide

Run the AI analysis engine locally

Prerequisites​

ToolVersionNotes
Python3.11.xRequired exactly — CatBoost + MediaPipe do not support 3.12+
uvLatestPython package + venv manager (this workstation's standard)
CUDA11.8+Optional, for NVIDIA GPU acceleration
GitLatestRepository access

:::warning Python Version Python 3.11.x is a hard requirement. CatBoost (bounce detection) and MediaPipe (pose estimation) do not have compatible wheels for Python 3.12+. :::

Hardware Recommendations​

ComponentMinimumRecommended
GPUCPU-only (slow)NVIDIA RTX 3080+ or Apple Silicon (MPS)
RAM8 GB16 GB
Storage5 GB (models + output)10 GB
VRAMN/A8 GB+

Installation​

1. Clone the Repository​

git clone <repo-url>
cd acesense-gpu-backend

2. Create a Virtual Environment​

uv venv --python 3.11
source .venv/bin/activate  # On Windows: .venv\Scripts\activate

3. Install Dependencies​

uv pip install -r requirements/requirements.txt

:::note RunPod image vs local The RunPod serverless Docker image is built on a prebuilt PyTorch/CUDA base and installs requirements/base.txt (plus linux-cuda.txt) — it does not use requirements.txt. requirements/ is split per platform: base.txt, linux-cpu.txt, linux-cuda.txt, mac.txt. :::

Running Locally​

Basic Analysis​

python main.py --path_input_video input_videos/sample.mp4

Development Mode (with cache)​

Dev cache stores intermediate results so subsequent runs skip expensive ML inference:

# First run — builds cache (~90s for 30s video)
python main.py --path_input_video input_videos/sample.mp4 --dev

# Subsequent runs — uses cache (~3s, 96% faster)
python main.py --path_input_video input_videos/sample.mp4 --dev

All CLI Options​

python main.py --path_input_video VIDEO.mp4 [OPTIONS]

Options:
  --path_output_video OUTPUT.mp4    Output video path
  --dev                              Enable dev cache (fast iteration)
  --save-shot-videos                 Save individual shot videos
  --no-save-shot-videos             Skip shot videos (faster)
  --cloud                            Upload to Firebase Storage
  --firebase_bucket BUCKET          Firebase bucket name
  --firebase_credentials PATH       Path to service account JSON

Cloud Upload​

python main.py \
  --path_input_video video.mp4 \
  --cloud \
  --firebase_bucket your-bucket.firebasestorage.app \
  --firebase_credentials firebasecred/firebase-admin.json

Project Structure​

acesense-gpu-backend/
├── main.py                  # Unified entry point — auto-detects serverless vs local
│                            #   via src.serverless (is_serverless_mode / start_serverless)
├── src/                     # Core processing modules
│   ├── pipeline/            # ACTIVE orchestrator: game_processor.py::process_video(sport, ...),
│   │                        #   phase6_outputs.py (emits insights.json), registry, stages glue
│   ├── stages/              # Stage implementations: ball_tracking/, surface/, events/, physics/
│   ├── core/                # Shared constants
│   ├── insights/            # Insight engine (candidates, scorer, ranker, templates)
│   ├── serverless/          # Vendor-isolated serverless runtime adapter (RunPod)
│   ├── tracking/            # Ball tracking (TrackNet)
│   ├── surface/             # Court detection
│   ├── players/             # Player detection
│   ├── events/              # Bounce & shot detection
│   ├── classification/      # Shot classification
│   ├── analysis/ fusion/    # Game analytics + sensor fusion
│   ├── export/              # analysis_export.py (schema 1.5.0), report generation
│   ├── io/                  # Video I/O, Firebase
│   ├── render/              # Visualization overlays
│   ├── preprocessing/       # Frame prep / resize
│   └── utils/               # Utilities & dev cache
├── games/                   # Tennis config + models
├── input_videos/            # Local input clips (ignored)
├── tests/                   # Test suite
└── requirements/            # Dependencies (base + per-platform)

Pre-Trained Models​

Models are included in the repository:

ModelFilePurpose
TrackNetgames/tennis/models/tracknet_ball.ptBall tracking
CourtDetectorNetgames/tennis/models/courtnet_keypoints.ptCourt detection
CatBoostgames/tennis/models/bounce_classifier.cbmBounce detection

Processing Pipeline​

Video Input
    ↓
Ball Tracking (TrackNet)
    ↓
Court Detection (CourtDetectorNet)
    ↓
Player Detection (FasterRCNN + MediaPipe)
    ↓
Bounce Detection (CatBoost)
    ↓
Shot Detection (Pose Analysis)
    ↓
Shot Classification (Sequence Classifier)
    ↓
Game Analysis (Stats + Recommendations)
    ↓
Export (Video, PDF, JSON, Minimaps)

Output​

Analysis creates a timestamped directory:

.dev/output/sample_video_19122030/
├── output_video.mp4              # Annotated video
├── analysis_report.pdf           # Performance report
├── sample_video_analysis.json    # Canonical analysis JSON (schema 1.5.0, embeds "insights")
├── insights.json                 # Ranked insight candidates (Phase 6, also embedded in analysis JSON)
├── ball_trajectory_3d.json       # Physics-based 3D trajectory
├── events_timeline.json          # Event frames
└── shots/                        # Per-shot metadata/pose

Dev Cache​

# Cache location
.dev/cache/<video_hash>/
├── ball_tracking.pkl
├── court_detection.pkl
├── player_detection.pkl
├── bounces.pkl
└── shots.pkl

# Clear cache
rm -rf .dev/cache/

Running Tests​

pytest tests/ -v

Performance Benchmarks (RTX 3080)​

StageTime
Ball Tracking~30s
Court Detection~10s
Player Detection~40s (CPU bottleneck on Apple Silicon)
Analysis & Export~10s
Total~90s for 30s video
With Dev Cache~3s (96% faster)

Troubleshooting​

ErrorSolution
catboost requires Python 3.11.xUse Python 3.11.x exactly
Device: cpu (MPS/CUDA not detected)Check GPU drivers; CPU fallback works but is slower
CUDA out of memoryReduce video resolution or batch size
ModuleNotFoundErrorEnsure venv is activated and deps installed
FFmpeg not foundInstall FFmpeg: brew install ffmpeg (macOS) or apt install ffmpeg (Linux)