🤖 AI Model + Data Card

EU AI Act transparency (Art. 13 / Art. 52) — plain-English disclosure of the AI pipeline AceSense uses to analyse tennis videos.

This is the answer to "what AI is in here, and what does it do?" for regulators, partners, and curious users.

1. Classification under the EU AI Act

Risk tier: minimal / low-risk.

❌ Not a prohibited practice (Art. 5) — no social scoring, no real-time biometric ID in public spaces, no behavioural manipulation.
❌ Not high-risk (Art. 6 + Annex III) — no employment / credit / education / law-enforcement decisions.
❌ Not subject to transparency obligations for specific AI systems (Art. 52) in the strict sense (no deepfakes, no emotion recognition, no chatbot pretending to be human).

We still publish this card voluntarily because (a) the Act encourages it, (b) subject-access requests may reference it, and (c) it's good practice.

2. What the system does

Takes a tennis video (single-angle, baseline perspective preferred) and produces:

Per-shot analysis — shot type, speed, bounce location, angle, depth.
Court detection — line positions, service box boundaries.
Player detection — body pose per frame (shoulders, elbows, wrists, hips, knees, ankles, etc.).
Ball tracking — 2D trajectory across frames, impact frames.
Whole-video AI read (Gemini) — a general video model (Google Gemini on Vertex AI) watches the full clip and produces a standardized, schema-constrained report: timestamped shot events, per-player skill ratings graded against a fixed written rubric, strengths and improvement notes with practice drills. Players are referenced only by court position and a neutral appearance cue (e.g. "red shirt") — the prompt forbids inferring names or identity.
Merged report — a deterministic (non-AI) rule set combines the two engines; measured GPU numbers lead only when the court was solved, and every merged figure is labeled with the engine that produced it.
Summary report — totals, distributions, coaching notes.

What it does NOT do:

Does not identify the player by name, face, or voice.
Does not match one player's pose against another.
Does not generate synthetic (deepfake) content.
Does not produce medical or diagnostic output.
Does not make legal / employment / financial decisions about anyone.
Does not process audio.

3. Pipeline

Video upload
    ↓
FFmpeg split → 5-min chunks (3 s overlap)
    ↓
Parallel GPU jobs (RunPod, EU/US)          Gemini whole-video read (parallel)
    ├── TrackNet  → ball trajectory            gemini-3.7-flash on Vertex AI
    ├── FasterRCNN + MediaPipe Pose            (global endpoint — inference may
    ├── Court homography (OpenCV)               run outside the EU; video read
    ├── CatBoost → shot classification          in place via gs:// URI, schema-
    └── Heuristic → bounce detection            constrained JSON output)
    ↓                                              ↓
Per-chunk analysis JSON                    jobs/{id}.videoAnalysis (va2)
    ↓                                              ↓
mergeChunkResults → combined analysis      mergeAnalysisReports (deterministic
    ↓                                       rules, no model call)
Storage write → result_path                        ↓
    ↓                                      jobs/{id}.mergedReport (vm2)
onResultUploaded trigger → Firestore jobs/{id}.status = 'done'

Component details

Component	Source	Role	Licence
TrackNet v2	Chen et al., NCTU (PyTorch reimplementation)	Ball position heatmap → 2D trajectory	MIT
MediaPipe Pose	Google	33-point skeleton per player per frame	Apache 2.0
FasterRCNN (torchvision)	PyTorch official	Player bounding boxes	BSD
CatBoost shot classifier	Trained in-house on annotated internal dataset	Shot-type label from pose + ball trajectory features	Apache 2.0
Court detection	Classical OpenCV (Hough + homography)	Pixel → real-world court metric mapping	BSD
Gemini (gemini-3.7-flash)	Google, hosted on Vertex AI (global endpoint)	Whole-video read: timestamped shot events, per-player ratings + coaching notes. Schema-constrained output; identity inference forbidden by prompt; temperature 0	Google Cloud terms (hosted API — no weights)

The CV models run on RunPod GPU pods; the Gemini read is a hosted Vertex AI API call (no video bytes transit our functions — the model reads the storage object directly). None runs on-device.

4. Training data (model cards for in-house-trained components only)

CatBoost shot classifier

Property	Value
Dataset source	Internal annotations produced via the `acesense-annotate` desktop app from publicly-available professional match footage (broadcast YouTube / ATP highlights), plus consented user uploads that opted in during the beta
Approximate size	~18 000 shots across ~200 matches
Demographics represented	Broadly mixed — pro tour demographics skew towards certain body types + skill levels; may under-represent recreational + junior players
Labels	13 shot types (serve flat/slice/kick, forehand topspin/slice/flat, backhand topspin/slice, forehand volley, backhand volley, overhead, lob, dropshot)
Features	Pose velocity, ball approach angle, contact frame pose, court position at contact
Reported accuracy	~78% top-1 on held-out pro-match validation set; drops to ~65% on recreational footage
Known failure modes	Low-light indoor courts; camera angles other than baseline-centered; players wearing same-color clothing as the ball
Bias assessment	Under-representation of junior + wheelchair tennis in the training set; the model's accuracy on those groups is untested

Court homography

Purely classical computer vision — no training data, no bias.

TrackNet / MediaPipe / FasterRCNN

Pre-trained third-party models. Refer to their upstream model cards for training data + bias information:

5. Intended use + misuse

Intended

A recreational or club-level tennis player reviewing their own practice.
A coach reviewing their student's video.
Self-directed improvement over weeks / months.

Out-of-scope / discouraged

Match-officiating (the line / bounce accuracy is not ITF-approved).
Medical / physiotherapy diagnosis.
Uploading videos of people who have not consented to AI analysis (e.g. opposing players in a match).
Scouting / selection decisions about third parties.

In-app: the AI consent dialog tells the user only their own performance is analysed. The coaching report includes a disclaimer: "AI-generated. Use alongside human judgement."

6. Human oversight

The output is always advisory. No automated decision is taken about the user.
Users can dispute any shot label via Settings → Send Feedback.
No automated-decision-making that produces legal or similarly significant effects per GDPR Art. 22.

7. Robustness + monitoring

Per-video confidence scores are exposed inside the analysis JSON. Low-confidence shots are flagged in the UI.
Weekly aggregated dashboards track model drift (served via admin panel).
User-reported errors (via Send Feedback) feed a quarterly retraining cadence.

8. Known limitations

Single-angle footage is strongly preferred. Panning cameras degrade accuracy significantly.
Lighting — indoor low-light can halve ball-tracking accuracy.
Clothing — outfits in a color close to the ball (neon yellow / lime) hurt tracking.
Doubles — current models are tuned for singles; doubles matches produce noisy player-identity assignments.
Non-standard courts — clay-court line detection is less robust than hard-court.

These are disclosed in-app via a "Best results" tip card on the upload screen.

9. Update log

Date	Change
2026-04-24	Initial publication
2026-08-17	Added the Gemini whole-video read (Vertex AI, global endpoint) and the deterministic GPU × Gemini merged report; DPIA sub-processor row updated to match

Was this page helpful?

1. Classification under the EU AI Act​

2. What the system does​

3. Pipeline​

Component details​

4. Training data (model cards for in-house-trained components only)​

CatBoost shot classifier​

Court homography​

TrackNet / MediaPipe / FasterRCNN​

5. Intended use + misuse​

Intended​

Out-of-scope / discouraged​

6. Human oversight​

7. Robustness + monitoring​

8. Known limitations​

9. Update log​