Flow — PMTiles Generation (Coverage + Custom Layers)

How the Planner web map gets its coverage layer and custom admin/priority layers.

Why PMTiles

The Planner web app needs to render large vector layers — coverage hexagons, territory polygons, priority areas, custom admin boundaries — across a wide zoom range. Two options:

• Tile server (Mapbox/MapTiler stack): another stateful service to run, scale, and authorize.
• PMTiles (protomaps.com/docs/pmtiles): a single binary file with HTTP range-read indexing. Serve it from S3, fetch ranges from the browser via the PMTiles JS protocol, no tile server needed.

We chose PMTiles. The Planner web app reads them via MapLibre's PMTiles protocol; Sense API issues a presigned URL for the active file.

There are two flavors of PMTiles in the platform:

Flavor	Job	Source	Refresh cadence
Coverage	CoverageSyncJob	ClickHouse frames (recent H3 cells)	Configurable (default daily)
Custom layers	CustomLayersSyncJob	External RDBMS (admin areas, priority polygons)	Configurable (default daily)

Both feature-flagged via CoverageSyncOptions.Enabled and CustomLayerSyncOptions.Enabled.

Sequence

Why this shape

Why generate offline, not on-the-fly

Tippecanoe is CPU-heavy (typical run: tens of seconds to minutes for a city-sized layer). Doing it per request would either explode latency or require a tile-server cache. PMTiles + S3 range reads gives us the cache for free, at the cost of slightly stale layers — acceptable when the data changes daily, not by the second.

Why FlatGeobuf as the intermediate

Tippecanoe accepts GeoJSON or FlatGeobuf. FlatGeobuf is ~10× smaller on disk and ~5× faster to read. For million-feature layers, GeoJSON would be the bottleneck.

Why one PMTiles file per org per date

• Per org → no cross-tenant data leakage; presigned URLs are per-tenant.
• Per date → previous versions remain reachable for audit / rollback. Lifecycle policy on the bucket prunes after N days.

Why Postgres tracks the latest key

The PMTiles file location is opaque to the API — it just looks up the row, issues a presigned URL, and trusts the Worker to keep s3_pmtiles_key current. This means the API has no knowledge of how PMTiles are produced.

Why tippecanoe runs in the Worker pod (not a sidecar)

A separate tippecanoe pod would mean shipping the input FlatGeobuf over the network and pulling the output back. Doing it in-process keeps the pipeline local. The trade-off is that the Worker needs the tippecanoe binary baked into its image and elevated CPU quota during runs.

Failure modes

Failure	What happens	Recovery
ClickHouse query times out	Job fails; Hangfire records failure.	Hangfire retries (default 3 attempts with backoff); operator can re-trigger from the dashboard.
External RDBMS unreachable	Custom layers job fails; coverage continues.	Same.
tippecanoe OOM (huge city, low quota)	Worker pod OOMKilled.	Increase pod memory; tippecanoe options --drop-densest-as-needed and zoom limit reduce output size.
S3 PUT fails	Job fails before Postgres update.	Retry; Postgres still points at the previous PMTiles, so the map keeps rendering the older layer.
Postgres update fails after S3 PUT	New file in S3, but Postgres still points at the old one.	Job retries; idempotent because it overwrites the row with s3_pmtiles_key = the just-uploaded key.
Browser request races a PMTiles update	Browser may get a presigned URL for an old file.	Acceptable — old file lives in S3 until the lifecycle policy expires it.

Resource notes

• Coverage job: ClickHouse query is the dominant cost. Use a narrow time window (since parameter) and rely on frames partitioning + the H3 skip index to keep it fast.
• Custom layers job: tippecanoe is the dominant cost. CPU-bound, single-threaded per file. Run with requests.cpu: 2, limits.cpu: 4.
• Disk: tippecanoe produces and reads big intermediates. Mount a emptyDir of ~10 GiB on the Worker pod for tmp.

Code references

• CoverageSyncJob, CustomLayersSyncJob: axion.sense.backend/src/Axion.Sense.Worker/Jobs/
• ClickHouse query for H3 cells: see axion.sense.backend/docs/CoverageSync.md for the canonical SQL.
• Web-side rendering: axion.sense.web/src/widgets/map-dashboard/ (MapLibre + PMTiles protocol).