📐 Architecture Overview

Distributed, Event-Driven, GPU-Accelerated

:::tip TL;DR

Client (Flutter) requests an upload path, then uploads one video file → Firebase Storage
A Cloud Function triggers on upload, creates/updates a job, and dispatches it to RunPod serverless GPU workers
GPU workers run the Python AI pipeline, write results back to Storage, and a result trigger marks the Firestore job complete
The app gets real-time updates via Firestore listeners — no polling :::

AceSense is built on a modern cloud-native architecture designed for scalability, reliability, and cost-efficiency. This document provides a comprehensive overview of the system design.

🎯 Design Principles

:::info Core Principles

Event-Driven: All processing is triggered by events (uploads, completions)
Stateless Compute: Each uploaded video job is processed independently
Horizontal Scaling: GPU instances scale based on demand
Fail-Safe: Failed processing can be retried without side effects
Cost-Optimized: Pay only for actual GPU usage on RunPod :::

🏗️ High-Level Architecture

📊 Data Flow

Video Upload Flow

🔧 Component Deep Dive

Client Applications

Component	Technology	Responsibility
UI Layer	Flutter Widgets	Screen rendering, user interaction
State Layer	Provider	App state, business logic
Service Layer	Dart	Firebase SDK, REST calls
Data Layer	Dart Models	Type-safe data structures
Camera	Platform Channels	Native camera access

Firebase Services

See firebase-functions/overview.md for the complete function list, trigger types, and the GDPR-flow (export / delete) details.

GPU Processing Pipeline

The pipeline uses a Stage Strategy Pattern: four abstract base stages (BallTracker, SurfaceDetector, EventDetector, PhysicsModel) defined in src/pipeline/stage_base.py, with concrete implementations under src/stages/. The checked-in config is games/tennis/config.yaml, and the entry point is process_video(sport="tennis", ...) in src/pipeline/game_processor.py. See GPU Backend for implementation detail.

Status (2026-05-05): Phase 6 output generation is ported. The modular pipeline emits: events_timeline.json, ball_trajectory_3d.json, annotated video, PDF report, per-shot directories, trimmed match video, and insights.json (AI insight engine — ranked insight candidates with text and implication). MediaPipe pose sequences, ML shot classification probabilities, and swing motion analysis require a GPU environment; they are absent in CPU-only runs but emitted as per-shot data when running on GPU.

Phase 6 output artifacts

Artifact	Description
`events_timeline.json`	Ordered list of match events (shots, bounces, serves, point ends) with frame indices and metadata
`ball_trajectory_3d.json`	Ball position in 3D court space per frame, derived via homography from 2D detections
Annotated video	Source video with overlaid court keypoints, ball track, player bounding boxes, and event labels
PDF report	8-page editorial match report (Match Pass cover + 7 body pages) built by `src/export/report_generator.py`
Per-shot directories	One sub-directory per detected shot containing trimmed clip, pose data, and shot-level JSON
Trimmed match video	Match-length video with dead time removed
`insights.json`	AI insight engine output — ranked candidates with `featured` (top insight) and `secondary` list, each with `text`, `implication`, `level` (2=comparison, 3=implication), and `score`

📬 User Feedback Pipeline

Out-of-band channel for users to flag bugs, request features, or rate docs pages. App feedback writes directly to Firestore; docs feedback posts to the admin-gated docs server, which writes with Admin SDK. It does not touch the GPU pipeline, RunPod, or the jobs collection.

Flow

Collections & schemas

Collection	Producer	Key fields
`feedback/{id}`	Flutter `FirestoreService.submitFeedback()`	`text` (string, 1-10000), `email`, `uid`, `createdAt`
`docs_feedback/{id}`	`docsServer` `/__/feedback`	`helpful` (bool), optional `note` (string ≤ 2000), `path`, `createdAt`, `userEmail`

Cloud Functions

Both are v2 onDocumentCreated triggers in acesense-auth-function/feedback/index.ts, re-exported from index.ts.

Function	Trigger	Behavior
`onFeedbackCreated`	`feedback/{id}` onCreate	Always alerts (skips silently when `text` is empty). Email contains text, user email, UID, Firestore deeplink.
`onDocsFeedbackCreated`	`docs_feedback/{id}` onCreate	Alerts on 👎 always; alerts on 👍 only when a `note` is attached, to avoid inbox spam.

Both build a brand-styled HTML + plaintext email and call the shared enqueueEmail() helper, which writes to mail/{id} — picked up by the Trigger Email from Firestore extension and dispatched via SMTP (Resend, verified acesense.io) to admin@acesense.io.

Firestore rules

feedback/ is admin-read and authenticated-create. docs_feedback/ is admin-read and server-created only; the Docusaurus widget posts to /__/feedback, and docsServer verifies the admin session before writing with Admin SDK. Update and delete are denied so feedback can't be tampered with after submission.

match /feedback/{id} {
  allow read: if isAdmin();
  allow create: if request.auth != null
    && request.resource.data.text is string
    && request.resource.data.text.size() > 0
    && request.resource.data.text.size() <= 10000;
  allow update, delete: if false;
}

match /docs_feedback/{id} {
  allow read: if isAdmin();
  allow create, update, delete: if false;
}

Source files

acesense-auth-function/feedback/index.ts — both Cloud Functions
acesense-auth-function/index.ts — re-exports
acesense-frontend/firestore.rules — feedback/ + docs_feedback/ rule blocks
acesense-frontend/lib/services/firestore_service.dart — submitFeedback() (writes to feedback/)
acesense-docs/functions/index.js — verifies docs admin sessions and writes docs_feedback/
acesense-docs/website/src/components/DocFeedback.tsx — posts to /__/feedback

🔐 Security Architecture

Layer	Mechanism	Details
Transport	TLS 1.3	All traffic encrypted
Authentication	Firebase Auth	Google OAuth 2.0
Authorization	Security Rules	Document-level access control
Storage	GCS Encryption	AES-256 at rest
Secrets	Secret Manager	API keys, credentials

📈 Scalability

Auto-Scaling Configuration

Metric	Scale Up Threshold	Scale Down Threshold
Queue Depth	> 10 pending jobs	< 2 pending jobs
Processing Time	> 5 min average	N/A
GPU Utilization	> 80%	< 20% for 10 min
Max Workers	10	1 (always-on)

🔗 Integration Points

System	Integration	Protocol
RunPod	Job dispatch & status	REST API
Firebase	Auth, Storage, Firestore	SDK
FCM	Push notifications	Firebase SDK
Analytics	Usage tracking	Firebase Analytics
api.acesense.io	Public REST + MCP surface	HTTPS rewrite → `apiServer` / `mcpServer` Cloud Functions

Public API surface (`api.acesense.io`)

The acesense-api-hosting repository owns a Firebase Hosting site (target api, site acesense-prod-api) that fronts api.acesense.io. It carries no application code — firebase.json rewrites traffic to two Cloud Functions defined in acesense-auth-function:

https://api.acesense.io/v1/** → apiServer (Express, europe-west1); Firebase Hosting preserves the /v1 prefix for Express routing.
https://api.acesense.io/mcp → mcpServer (europe-west1); Model Context Protocol endpoint consumed by external agents.
API keys are minted in the Admin panel (/admin-api-keys), stored in Firestore, and scoped per user.

See architecture/api-reference.md for full request/response schemas.

🎯 Next Steps

🚀 New to the codebase? → 5-Minute Quickstart
🔧 Ready to deploy? → Deployment Guide
🔌 Integrating? → API Reference
🔑 Configuring secrets? → Configuration

Frontend — Flutter app implementation
GPU Backend — AI processing pipeline
Firebase Functions — Cloud functions
Annotate — Annotation tool

Was this page helpful?

🎯 Design Principles​

🏗️ High-Level Architecture​

📊 Data Flow​

Video Upload Flow​

🔧 Component Deep Dive​

Client Applications​

Firebase Services​

GPU Processing Pipeline​

Phase 6 output artifacts​

📬 User Feedback Pipeline​

Flow​

Collections & schemas​

Cloud Functions​

Firestore rules​

Source files​

🔐 Security Architecture​

📈 Scalability​

Auto-Scaling Configuration​

🔗 Integration Points​

Public API surface (api.acesense.io)​

🎯 Next Steps​

📚 Related Documentation​