Object Storage

S3-compatible blob storage for frames, PMTiles, track logs, and Parquet artifacts.

At a glance

Property	Value
Compatibility	AWS S3 / GCS / MinIO (any S3-compatible service)
Deployed	Externally — provider-managed (cloud) or MinIO in-cluster (on-prem)
Client	AWS SDK for .NET (AWSSDK.S3)
Access	Presigned URLs (mobile + web), service credentials (Worker)

Bucket layout

A single bucket per environment (configurable per-org override possible):

<env-bucket>/
├── frames/
│   └── {track_id}/
│       └── {frame_id}.jpg
├── log/
│   └── {track_id}                  ← optional debug log file from mobile
├── pmtiles/
│   ├── coverage/
│   │   └── {org_id}/
│   │       └── {YYYY-MM-DD}.pmtiles
│   └── custom-layers/
│       └── {org_id}/
│           └── {layer_id}/
│               └── {YYYY-MM-DD}.pmtiles
└── parquet/                         ← Gen S3 datasource only
    └── {dataset}/...

Key design choices

Presigned URLs everywhere mobile / web touches data

Neither Sense API nor Gen API ever proxies bytes. Frames upload directly from the mobile app to S3 via presigned PUT (15-minute TTL). PMTiles fetch is via presigned GET with a longer TTL (1 hour) since the browser needs range reads.

This decouples upload throughput from API capacity entirely.

Service credentials for Worker, not presigned

The Worker reads frames during PMTiles generation and the Citylens initial migration. It uses long-lived service credentials, not presigned URLs — internally to the cluster, presigned would add latency without benefit.

Path layout encodes ownership and time

• frames/{track_id}/... — track ID is UUIDv7 (time-encoded), so a path-prefix listing gives you all frames for a track and indirectly all frames in a time window.
• pmtiles/.../{org_id}/... — IAM-style bucket policies can scope access by prefix.
• pmtiles/.../{date}.pmtiles — old versions stay reachable; lifecycle policy expires them after N days.

Lifecycle policies

Prefix	Policy
frames/...	Indefinite retention (operational data)
log/...	30 days
pmtiles/coverage/...	Keep last 7 generations per org; expire others
pmtiles/custom-layers/...	Keep last 7 generations per (org, layer); expire others
parquet/...	Customer-managed

Configuration

"S3": {
  "ServiceUrl": "https://s3.amazonaws.com",   // or MinIO endpoint
  "Region": "us-east-1",
  "Bucket": "axion-prod-data",
  "AccessKey": "...",                          // service credentials (Worker)
  "SecretKey": "...",
  "UsePathStyle": false,                       // true for MinIO
  "PresignedUrlTtlSeconds": 900                // 15 minutes default
}

On-prem deployments

For on-prem, MinIO is the standard choice. Charts in axion.infra/services/minio (when present) deploy a single MinIO instance with persistent volumes. Multi-site replication is not in scope for this layer — customers using on-prem are typically single-site.

CORS

CORS rules on the bucket allow:

• PUT from the mobile origin (presigned URLs).
• GET from the web origins (presigned PMTiles fetches).

Misconfigured CORS is the #1 cause of "the upload returns 403 even with a fresh presigned URL." Confirm via the S3 CLI's get-bucket-cors.

Observability

• S3 access logs (provider-side), shipped to a sibling bucket if compliance requires it.
• Per-presign issuance recorded as a span on the API; ties into the request's distributed trace.

Cost notes

• Frames are the dominant cost driver. A typical city-day produces N GiB; verify against the production footprint and pick the storage class accordingly:
• Standard for the active month.
• Intelligent-Tiering (or equivalent) for older months.
• PMTiles are small relative to frames — keep on Standard.