Skip to main content

Job Document Schema

Each processing request has one jobs/{jobId} document. App uploads, REST-created jobs, and MCP-created jobs share the collection, but source and billing fields can differ.

Initial app-upload document​

requestUploadPath creates the document with the Admin SDK after auth, App Check, consent, entitlement, metadata, and quota checks:

{
  userId: string,
  sessionId: string,           // generated by the server
  status: "pending",
  progress: 0,
  fileName: string,            // sanitized display/source name
  videoPath: string,
  resultPath: string,
  chunkIndex: 0,
  totalChunks: 1,
  isChunked: false,
  quotaSlotReserved: true,
  quotaSlotReleased: false,
  metadata: {
    fileSize?: number,
    duration?: number,
    width?: number,
    height?: number,
    format?: string,
    codec?: string | null
  },
  createdAt: Timestamp,
  updatedAt: Timestamp
}

The current Flutter flow uploads one object even though legacy chunk fields remain for compatibility. The issued paths are:

videos/{uid}/{sessionId}/{sessionId}_chunk_001.mp4
results/{uid}/{sessionId}/{sessionId}_combined.json

Lifecycle​

Some backend paths move directly between adjacent operational states, so consumers must tolerate pending, uploaded, queued, processing/running, done, and failed. The public REST/MCP surface normalizes these into queued, processing, done, and failed.

Fields added during processing​

Depending on the source and result, server code may add:

  • dispatch identifiers and provider metadata;
  • progress, processing timestamps, and completion timestamps;
  • videoUrl, resultUrl, pdfReportUrl, or other signed/generated artifact references;
  • errorCode and errorMessage on failure;
  • API-key, cost-reservation, and refund markers for commerce jobs;
  • insights/report-generation and reclassification markers;
  • quotaSlotReleased and its timestamp at terminal cleanup;
  • retry counters — retryCount/lastRetryAt/lastRetryBy for an admin retry, infraRetryCount/lastInfraRetryAt for the automatic infrastructure retry, and backfilledAt for a backfill. Any of these being set makes the completion email use "report updated" copy instead of the first-time "report ready" — see isReanalysis(). A job that already told its owner it FAILED must not come back as a fresh completion.

These are server-owned. Code should treat optional provider/result fields as absent until the relevant stage has completed.

Client permissions​

  • Owners can read their own documents.
  • Owners can update only userTitle, userTags, userNotes, isFavorite, and updatedAt.
  • Clients cannot create jobs or change status/result/owner fields.
  • Owners delete through deleteSession; direct deletion is admin-only.

Recovery and idempotency​

Terminal transitions use shared helpers so retries do not release quota or refund commerce cost twice. reportUploadFailure only acts while an owned job is still pending. reconcileStuckJobs fails pending/uploaded work after two hours and processing work after six hours. Administrators retry through adminRetryJob, not by editing the document.

retryInfraFailedJobs retries automatically every 10 minutes, for GPU_JOB_FAILED and STUCK_JOB_TIMEOUT — codes that can mean our infrastructure dropped a job whose video is still in Storage. GPU_REJECTED means the pipeline ran and rejected the clip, so retrying burns GPU time to reach the same answer. It is capped per job and per tick and skips work older than a week.

Retry requires processingStartedAt. processVideoOnUpload writes it in the same update that sets status: "processing", immediately after the pre-flight gate, so its presence is exact proof the job was screened and dispatched once already. Without it the job never cleared moderation and filmability, and this path dispatches straight to a worker — so CLIENT_UPLOAD_FAILED stays excluded even though a small number of those jobs do have their video in Storage. Recovering those means re-entering the upload path, not this one.

reconcileStuckJobs checks Storage before failing an overdue pending job. If the object is present the finalize event was lost rather than the upload abandoned, so it copies the object onto itself — a new generation re-emits object.finalized — and leaves the job pending, which is the only state processVideoOnUpload claims. Recovery therefore re-runs pre-flight. Bounded by uploadRecoveryCount.

A requeue must refresh createdAt with a server timestamp. The stuck-job sweeper fails any active job whose createdAt is over ten minutes old, so a job put back with its original timestamp is killed within the minute — and an ISO string leaves createdAt.toDate undefined, which makes it un-sweepable instead.

When adding a field, update the server writer, TypeScript/Dart readers, Firestore rules if it is client-editable, tests, and this page in the same change.