Media Storage And Delivery

Wavemap’s media architecture supports Artist, Event, Venue, Event Series, and Event Series Edition image authoring through shared storage and delivery boundaries. Each entity still owns its persistence, routes, authorization, authoring workflow, read projections, and product language.

Use this page when changing an implemented media-owning entity, media upload or sync flows, storage adapter selection, local media execution modes, image ingest, media delivery fields, or deployed-dev media validation.

Use Media Workflow And Validation for the practical lane choice between mocked storage, local emulators, deployed-dev media proof, browser media smoke, and discrepancy reporting.

Current Scope

The current media platform is intentionally bounded:

Artist, Event, Venue, Event Series, and Event Series Edition image authoring, persistence, and public reads.
Server-mediated uploads through backend media endpoints.
Public render URLs returned by the backend.
One active storage locator per asset.
Canonical image plus a first-pass thumbnail and blur preview derivative.
AWS S3 as the primary serious-validation path.
Azure retained as a continuity path, not the roadmap driver.
Mocked storage and provider emulators for local and test confidence.

The current platform does not try to solve a universal cross-entity media model, video/audio processing, direct-to-storage uploads, signed URLs, private delivery, multi-locator replicas, or a broad variant matrix.

Core Decisions

Keep frontend upload draft state separate from persisted media records.
Keep scalar entity create/update requests focused on JSON entity details.
Route real media writes through dedicated entity-owned media workflows.
- At some later point, explore the possibility of a process of writing media directly to storage, bypassing proxying it through the application server.
Return render-ready media fields such as profileImageURL, media[].src, and thumbnail/preview fields to the frontend.
Store provider-agnostic storage locator fields as canonical identity rather than provider-shaped URLs.
Keep media kind, purpose, and lifecycle status as separate concepts.
Keep artist_media, event_media, venue_media, event_series_media, and event_series_edition_media as entity-owned canonical collections; shared infrastructure does not imply a universal media table or route family.
Keep storage provider, execution target, and application environment as separate runtime concepts.
Use the active write provider for new uploads, but use each stored locator’s provider for reads, deletes, and cleanup.
Keep media ingest behind a dedicated backend seam so validation, normalization, and derivative generation can evolve without changing upload transport or frontend reads.

Vocabulary

Term	Meaning
Media draft	Temporary frontend upload-selection state before an asset is persisted.
Media asset	The logical persisted media record associated with an entity.
Storage locator	Provider-agnostic identity for the underlying stored object.
Render URL	The URL the backend returns for frontend display.
Kind	Technical media category, such as image, video, or audio.
Purpose	Product meaning, such as profile, gallery, clip, or track.
Status	Lifecycle state, such as pending, processing, ready, or failed.
Artifact kind	Stored object meaning inside the media pipeline, such as canonical, thumbnail, or poster.
Execution target	Runtime target for a provider implementation, such as cloud, emulator, or mocked.

Layer Boundaries

Layer	Owns
Frontend draft layer	Temporary selected files, browser previews, form ordering, and UI workflow state before submission.
Frontend workflow layer	Sequencing entity creation/update, upload calls, sync calls, retry posture, and UI-facing workflow state.
Backend media routes	Application-facing media write contracts, request validation, and route-level response envelopes.
Image ingest layer	File inspection, validation, normalization, metadata policy, and derivative generation.
Database layer	Canonical metadata, lifecycle status, sort order, purpose, and storage locator fields.
Storage adapter layer	Provider-specific upload, delete, public URL resolution, legacy URL parsing, and emulator support.
Frontend read layer	Render-ready media fields and layout metadata, not provider-specific storage details.

Endpoint hooks should stay close to individual API routes. Feature-level orchestration hooks should own sequencing, diffing, retry posture, and UI-facing workflow state.

Canonical Record Shape

The first-pass media record is shaped around canonical metadata plus locator identity:

type TCanonicalImageMediaRecord = {
  ID: string
  entityID: string
  kind: "image" | "video" | "audio"
  purpose: "profile" | "gallery" | "clip" | "track"
  status: "pending" | "processing" | "ready" | "failed"
  originalFileName: string
  mimeType: string
  byteSize: number
  width?: number
  height?: number
  durationMs?: number
  altText?: string
  sortOrder: number
  storageProvider: "azure" | "s3" | "mocked"
  storageLocation: string
  storageKey: string
  thumbnailStorageProvider?: "azure" | "s3" | "mocked"
  thumbnailStorageLocation?: string
  thumbnailStorageKey?: string
  thumbnailURL?: string
  blurDataURL?: string
}

The exact implementation can evolve, but these boundaries should remain stable:

The record stores canonical metadata and locator identity, not a provider URL as the source of truth.
purpose carries product meaning instead of a narrow flag like isProfileImage.
sortOrder describes persisted collection order, not upload-batch order.
Thumbnail locator fields follow the same provider-agnostic direction as canonical media.
Transitional read conveniences can remain while the canonical locator model settles.

The example is conceptual. Live persistence remains entity-shaped through the owner column on each entity media table; application code must not treat entityID as a real universal database field.

Entity-Owned Columns

Artist, Event, Venue, Event Series, and Event Series Edition media deliberately share infrastructure rather than domain ownership:

Owner	Product Language	Persistence	Workflow And Public Composition
Artist	Profile and Gallery	`artist_media`	Artist-owned Add/Edit recovery and portrait/card/detail densities.
Event	Poster and Gallery	`event_media`	Event-owned Add/Edit recovery and compact/full-poster densities.
Venue	Profile and Gallery	`venue_media`	Venue-owned Add/Edit recovery and Index/Details projections.
Event Series	Profile and Gallery	`event_series_media`	Series-owned Add/Edit recovery and Index/Details projections.
Event Series Edition	Profile and Gallery	`event_series_edition_media`	Edition-owned Add/Edit recovery and nested Details projections.

All five columns use public media IDs, image kind, purpose, lifecycle status, purpose-local ordering, optional alt text, render-ready full and thumbnail URLs, intrinsic dimensions, blur previews, and provider-neutral storage locators. They use the shared image-ingest and storage-adapter seams for writes while retaining entity-owned permissions, routes, tables, storage prefixes, collection semantics, and recovery workflows. Entity handlers remain responsible for parent resolution, media ownership, authorization, and response projections.

Transitional Simplicity

The first pass should keep existing product flows stable while the canonical media model settles.

Allowed transition posture:

Keep returning profileImageURL and media[].src as read conveniences so frontend surfaces do not need broad churn.
Keep profileImageURL derived from persisted profile-purpose media rather than restoring an entity-table URL as a source of truth.
Keep legacy URL fallback and legacy URL parsing inside the backend/storage boundary while older rows still require it.
Keep mediaTypeID as the active database representation of media kind while additive canonical fields land.
Keep image handling bounded to the five implemented entity-owned columns.
Keep create flows parent-first: create the entity, then attach media through its dedicated workflow.
Keep edit flows bounded: patch entity details and synchronize changed media through media-focused requests. Hydration, no-op detection, deletion/restore, and revision coordination remain explicit entity-workflow responsibilities.
Keep one backend storage utility/adapter layer small enough to own upload, delete, public URL resolution, and legacy URL parsing before expanding into a broader provider plugin system.

Do not turn the transition into a second source of truth. Transitional fields and fallbacks are there to preserve product behavior while canonical locators take over.

Storage Runtime Model

Media runtime selection separates three concerns:

Concern	Examples	Persistence Boundary
Storage provider	`s3`, `azure`, `mocked`	Stored on media locators because it identifies how to read/delete the stored object.
Execution target	`cloud`, `emulator`, `mocked`	Runtime config only. Do not persist emulator names as providers.
Application environment	`development`, `test`, `dev`	Runtime/deploy context. Use it to reject unsafe combinations.

Emulators are execution targets of real provider adapters:

S3Mock is the emulator target for S3-shaped local development work.
Azurite is the emulator target for Azure-shaped continuity work.
Neither S3Mock nor Azurite should appear as persisted storageProvider values.

Mocked storage is the deliberate exception. It is a first-class storage double for development and test app logic, so locators created by the mocked adapter can honestly use storageProvider: "mocked" rather than pretending to be S3 or Azure.

Execution Modes

Mode	Use For	Avoid For
Mocked storage	Day-to-day app logic, unit tests, most request tests, non-storage work.	Provider behavior, object-store path construction, IAM, bucket policy, public URLs.
Emulator-backed S3	Local AWS-shaped adapter behavior through S3Mock.	Real IAM, CloudFront, or deployed-dev proof.
Azure emulator	Azure continuity smoke through Azurite.	Driving the AWS-first roadmap.
Real AWS `dev` media	Serious end-to-end validation, IAM, bucket policy, S3/CloudFront proof.	Ordinary local work or broad reset scripts.
Future production	User-facing durable media.	Disposable reset assumptions from deployed `dev`.

Local reset behavior must follow the same split:

Mocked mode has no durable provider state.
Emulator mode can reset local emulator media state intentionally.
Cloud mode must never be targeted by generic local reset commands.

The deployed dev environment uses real S3 storage behind the app delivery path. See Deployed Dev Environment and Deployed Dev Lifecycle for the operator-facing lifecycle boundary.

Adapter Rules

The storage adapter contract should stay provider-neutral above provider-specific modules.

Adapters own:

Uploading stored objects.
Deleting stored objects.
Resolving public render URLs.
Parsing legacy provider URLs where migration needs it.
Supporting cloud or emulator execution targets inside the provider implementation.

Application media code owns:

Product meanings such as profile, gallery, canonical, thumbnail, and blur preview.
Entity/purpose/kind/sort metadata.
Choosing the active write adapter from runtime config.
Resolving read/delete behavior from the persisted locator’s provider.

This split enables gradual provider migration. New uploads can move to S3 while older assets still resolve from Azure locators, because reads and deletes follow the stored locator rather than the current active write provider.

URL And Delivery Boundaries

Frontend reads should receive render-ready values, not storage-provider internals.

Current public read fields may include:

src for full-display media.
thumbnailSrc or thumbnailURL for small surfaces.
blurDataURL for tiny preview or blurred placeholder presentation.
width and height when the backend can provide trustworthy dimensions.

Surface rules:

Showcase, gallery, card, search-result, and table surfaces should prefer thumbnail fields.
Carousel and large-detail surfaces should use the full-display src.
Blurred background treatments should prefer a tiny preview or blur payload rather than loading the full image again.
Only the immediately important above-the-fold asset should be eager by default.

A backend-owned media resolution route remains deferred. Direct render URLs are still the simplest current shape, but the frontend contract should not require provider-specific URL knowledge.

Image Ingest Policy

The backend image-ingest seam sits before storage finalization.

Current first-pass posture:

Validate non-empty file presence.
Enforce supported MIME types.
Decode the image and verify the decoded format matches the upload contract.
Reject uploads over the hard request byte ceiling.
Reject images over the hard decoded-dimension ceiling.
Downscale valid-but-large images to the canonical delivery target.
Auto-orient before stripping metadata so render orientation does not depend on EXIF.
Re-encode to the same allowed output format.
Strip embedded metadata from public-facing outputs through Sharp defaults.
Persist canonical metadata for the stored canonical asset.
Generate a stored thumbnail and an inline blur preview from the normalized canonical image.

Client-side preflight is advisory. Server-side validation is authoritative.

The upload transport, canonical media record, and frontend read contract should stay stable while validation, normalization, thumbnail generation, preview generation, and future async processing evolve behind the ingest seam.

Lifecycle And Reconciliation

The current image workflows are storage-first and insert media rows as ready only after required storage work succeeds.

Current rule:

Validate and ingest the uploaded image synchronously.
Upload the canonical asset and required first-pass derivatives.
Insert the media row as ready.
If storage fails before the row exists, return failure without creating a media row.
If DB finalization fails after request-owned provider objects were created, attempt compensating deletes.

The richer pending / processing / failed lifecycle remains reserved for future flows where the application intentionally creates non-ready rows, such as upload sessions, async processing, moderation, malware scanning, video transcoding, or retryable finalization.

Until non-ready rows exist intentionally, stale pending / failed reporting is lower signal. Operator visibility comes from the shared pending-cleanup inventory and read-only database/provider discrepancy reports.

Shared Media Cleanup And Recovery

collectMediaCleanupReferences owns canonical/thumbnail enumeration, including stored thumbnails without a URL, locator resolution and deduplication. Fully external URLs are excluded from deletion; partial owned references remain visible as unresolved work. Reconciliation uses the same artifact enumeration while preserving domain reporting policy.

Artist, Event, Venue, Event Series and Edition media deletion/sync, plus whole-Artist and whole-Event deletion, stage recovery work in media_cleanup inside the transaction that removes canonical database references. The shared table contains only owner type/public ID, storage reference, creation time and retry diagnostics. It has no entity foreign key or retained entity content. Migration 0052_shared_media_cleanup renames the former Event table and preserves pending references and diagnostics, backfilling their owner type as events.

Domain handlers retain authorization, parent/media ownership, revisions, relationship changes, canonical media tables and public DTOs. stageMediaCleanup performs no provider I/O. After commit, attemptPendingMediaCleanup attempts at most 25 objects for that owner; failure cannot turn a committed entity mutation into an API failure. Whole-Event deletion retains its thin compatibility facade and mediaCleanup.status response (complete or pending).

cleanupMedia powers the shared manual recovery command and immediate retries. It locks each selected pending entry independently while the stored provider adapter deletes its object. Successful entries are immediately removed; failures retain sanitized diagnostics. Concurrent attempts skip locked entries. If provider deletion succeeds but database acknowledgement fails, retry uses the adapters’ idempotent deletion behavior. No response or operator report exposes storage references.

cleanupUploadedMedia shares request-owned upload compensation across the five canonical media families. It attempts all distinct created objects after thumbnail upload or media-row insertion fails, and records only failed objects in an independent transaction when the database is available. It preserves the original request failure. A database outage, process crash before recording recovery, or uncertain media-row insert acknowledgement still requires discrepancy inspection; this storage-first flow does not provide an atomic transaction across the provider and database.

The existing optional media compensation in user/admin registration also uses the shared executor. Those older flows upload inside their registration transaction and omit an independent recovery write while it is open. This avoids nested-connection contention and recovery rows disappearing with rollback; their broader registration failure lifecycle is unchanged. Failed compensation there is diagnosed for manual inspection, not promised as durable recovery.

This pending inventory has no age-based purge or scheduled worker. Operators own failed-work follow-through; the command bounds each attempt, not the total backlog or provider request duration. Successful work does not accumulate as an audit.

Reconciliation shares infrastructure without erasing entity policy. The backend reconciliation kernel owns inspection target validation, entity-prefix construction, storage-scope filtering, database/storage comparison, discrepancy classification, and summary reduction. Artist, Event, Venue, Event Series, and Event Series Edition adapters remain thin but explicit owners of:

Entity-specific database reads and parent identity.
Storage path prefixes and public report shapes.
Domain labels used in operator-facing discrepancy messages.
Transitional URL policy: Event, Venue, Series, and Edition accept fully external URL-backed rows without provider locators, while Artist expects every rendered media URL to have a canonical provider locator. Partial locators remain reportable for every entity because they cannot be resolved or cleaned up reliably.

Keep this split when adding another media-owning entity. Extend the kernel for storage-comparison mechanics that are genuinely identical; keep authorization, persistence, routes, collection semantics, and migration policy in the owning entity adapter.

Deployed-Dev Boundary

In deployed dev:

The media bucket is private.
Browser-facing media is served through the deployed app CloudFront distribution under /media/*, not through direct public bucket access.
Runtime media access belongs to the runtime host role, not frontend/browser IAM credentials.
Runtime media config should flow through deployed-dev runtime config, SSM, and host-side env rendering.
GitHub Actions should not decrypt app runtime secrets or own media runtime credentials.
Database reset does not delete S3 media objects and can create application-orphaned media.
Media bucket replacement or stack teardown can delete media objects because the deployed-dev bucket is disposable at the infrastructure lifecycle level.

Use Configuration And Secrets when adding or changing runtime media config. Use Runbooks for the read-only deployed-dev discrepancy report and Media Bucket Replacement before any deployed-dev bucket replacement or destroy.

Deliberate Deferrals

Keep these out of the current architecture unless a concrete product or operational need appears:

A universal cross-entity media table, public route family, or monolithic authoring workflow.
Direct-to-storage upload sessions.
Signed URL or private-delivery complexity.
Multi-locator replicas for the same asset.
Video/audio upload and processing workflows.
Broad responsive variant matrices.
Crop-intent modeling or crop-aware derivative generation.
Automated repair or scheduled cleanup before report data justifies it.
Local emulator orchestration through Pulumi.