ogdc_runner.workflow.viz_workflow module

ogdc_runner.workflow.viz_workflow.create_3dtile_parallel() → None

Convert staged GeoPackage vector tiles to Cesium 3D tiles (B3DM/GLB).

Return type:

None

ogdc_runner.workflow.viz_workflow.create_composite_z_parallel() → None

Create composite (parent) raster tiles from child GeoTIFFs at z+1.

Return type:

None

ogdc_runner.workflow.viz_workflow.create_web_tile_parallel() → None

Convert GeoTIFFs to PNG web tiles.

Return type:

None

ogdc_runner.workflow.viz_workflow.discover_all_geotiffs() → None

Discover all GeoTIFF files across every z-level for web-tile fan-out.

Each partition entry is a dict {"z": int, "path": str} so the worker receives the correct z-level.

Return type:

None

ogdc_runner.workflow.viz_workflow.discover_parent_tiles() → None

Discover parent tile IDs at z-level from child GeoTIFFs at z+1.

Outputs a JSON array-of-arrays of {"z": int, "path": str} dicts for Argo withParam fan-out.

Return type:

None

ogdc_runner.workflow.viz_workflow.discover_staged_tiles() → None

Discover staged GeoPackage tiles at max z-level and emit chunked partition manifests.

Outputs a JSON array-of-arrays to stdout for Argo withParam fan-out.

Return type:

None

ogdc_runner.workflow.viz_workflow.make_and_submit_viz_workflow(recipe_config: RecipeConfig, wait: bool = False) → str

Create and submit a parallel Argo workflow for viz processing on K8s.

Implements a 5-stage parallel pipeline with strict z-level ordering:

Stage 1 — Stage input files → max-z GeoPackage tiles (with_param fan-out) Stage 2 — Rasterise max-z GeoPackage tiles → GeoTIFFs (discovery → with_param) Stage 3 — Build composite parent tiles from max-z-1→min-z (sequential z, with_param tiles) Stage 4 — Convert GeoTIFFs → PNG web tiles (discovery → with_param) Stage 5 — Convert staged vectors → Cesium 3D tiles (discovery reuse → with_param)

All stages use Argo withParam for dynamic fan-out so the number of DAG nodes stays bounded regardless of dataset size.

For extreme-scale datasets (> ~10 M files) Stage 1 currently embeds the partition manifest in the workflow spec. A future hardening step writes manifests to the workflow PVC during setup and passes only partition indices via withParam to stay within etcd size limits.

Parameters:

• recipe_config (RecipeConfig) – Recipe configuration with workflow.type == "visualization".

• wait (bool) – If True, block until the workflow completes. Defaults to False for async submission.

Return type:

str

Returns:

The Argo workflow name (e.g. "my-recipe-visualization-xyz12").

Raises:

OgdcInvalidRecipeConfig – If the workflow type is not "visualization".

ogdc_runner.workflow.viz_workflow.rasterize_max_z_parallel() → None

Rasterise a partition of staged GeoPackage vector files to GeoTIFF at max z-level.

Return type:

None

ogdc_runner.workflow.viz_workflow.run_viz_serial() → None

Run the enabled visualization stages sequentially in one pod.

Return type:

None

ogdc_runner.workflow.viz_workflow.stage_file_parallel() → None

Stage input files into vector tiles at max z-level.

Processes one partition of input file URLs. The partition-manifest parameter is a JSON-serialised list of file paths / URLs.

Return type:

None