ops prefect
Prefect flow and deployment management. Prefect is the primary flow orchestrator for all data pipelines in the Wasteology portfolio. These commands are now the single CLI surface for the complete flow lifecycle — an agent (or engineer) can probe an API, smoke-test locally, ship a new image, deploy, run with row-count gating, and diagnose stalls entirely through ops prefect without touching the Prefect UI or bare az / prefect CLIs.
Prefect runs the daily import flows, transformations, and scheduled jobs for every owned project. When something fails, ops prefect runs --state FAILED is usually the fastest path to the root cause.
Subcommands
| Subcommand | Description |
|---|---|
deployments | List Prefect deployments |
runs | List recent flow runs |
trigger | Trigger a deployment run (simple, no gating) |
inspect | Inspect a specific flow run |
cancel | Cancel a running flow |
blocks | List Prefect Cloud blocks |
run | Trigger a deployment with optional watch + row-count gate |
deploy | Register or update a deployment from a YAML or script |
ship | Build image → deploy → gated run in one command |
run-local | Run a flow file locally for pre-ship smoke testing |
probe | Authenticated API probe using a Prefect credentials block |
doctor | Diagnose "scheduled but never starts" failures |
ops prefect runs
List recent flow runs, optionally filtered by state, flow prefix, or count. This is the most-used subcommand — keep it close.
Flags
| Flag | Type | Description | Default |
|---|---|---|---|
--limit-l | INTEGER | Maximum number of runs to display | 20 |
--state-s | TEXT | Filter by state (COMPLETED, FAILED, RUNNING, CRASHED, CANCELLED) | — |
--filter-f | TEXT | Filter to flows whose name contains this prefix | — |
--json | flag | Output as JSON for scripts and AI agents | — |
Examples
--state FAILED is the single most useful flag here. Run it first thing in the morning to see what broke overnight.
ops prefect deployments
List all Prefect deployments — the named, schedulable instances of flows.
ops prefect trigger
Trigger a deployment to start a new flow run immediately, outside of its normal schedule. This is the simple, ungated variant — use ops prefect run if you need watch mode or row-count verification.
Triggered runs use the deployment's configured environment. Double-check you're not pointing at production before triggering ad-hoc runs.
ops prefect inspect
Inspect a single flow run in detail — parameters, state transitions, logs pointer, timing.
ops prefect cancel
Cancel a flow run that's currently in flight. Useful when you spot a bad input or need to stop a runaway job.
ops prefect blocks
List Prefect Cloud blocks — credentials, secrets, and configuration objects registered with the workspace. Replaces running bare prefect block ls outside of ops.
Flags
| Flag | Type | Description | Default |
|---|---|---|---|
--filter-f | TEXT | Show only blocks whose name or type contains this string | — |
--json | flag | Output as JSON | — |
Examples
ops prefect run
Trigger a named deployment and optionally wait for it to finish. With --watch, the command exits non-zero unless the run reaches Completed. Add --verify-count to also gate on a minimum row count in a target table — useful for ensuring an ETL actually landed data before declaring success.
This is the production-safe successor to trigger: it enforces outcomes, not just submission.
Flags
| Flag | Type | Description | Default |
|---|---|---|---|
--params | JSON | JSON string of parameters to pass to the deployment | — |
--watch | flag | Poll until the run reaches a terminal state; exit non-zero on anything other than Completed | — |
--timeout | INTEGER | Seconds to wait when --watch is set before timing out | 300 |
--verify-count | TEXT | schema.table to row-count after the run; requires --watch | — |
--min-rows | INTEGER | Minimum acceptable row count for --verify-count | 1 |
--conn | TEXT | Sling connection name to use for --verify-count | WASTE_PG_DEV |
--json | flag | Output as JSON | — |
ops prefect deploy
Register or update a Prefect deployment from a YAML spec file or a Python deploy script, without leaving wg-orchestration. The command runs inside the owning repo (prefect-azure-infra by default) via the cross-project path plumbing already configured in the registry.
Flags
| Flag | Type | Description | Default |
|---|---|---|---|
--repo | TEXT | Project registry alias for the repo that owns the deployment file | prefect-azure-infra |
--json | flag | Output as JSON | — |
ops prefect ship
Build and push the shared Prefect container image, deploy, then trigger a gated run — all in one command. Designed for CieTrade flow releases where every step must succeed before the next one starts.
Flags
| Flag | Type | Description | Default |
|---|---|---|---|
--deploy | TEXT | Deployment YAML or script to apply after the image push | — |
--run | TEXT | "flow/deployment" to trigger after deploy | — |
--watch | flag | Wait for the triggered run to reach a terminal state | — |
--verify-count | TEXT | schema.table to row-count after the run | — |
--min-rows | INTEGER | Minimum acceptable row count for --verify-count | 1 |
--conn | TEXT | Sling connection name to use for --verify-count | WASTE_PG_DEV |
--confirm | flag | REQUIRED. Acknowledges blast-radius before rebuilding the shared image | — |
--json | flag | Output as JSON | — |
--confirm is required because rebuilding the shared CieTrade image redeploys all CieTrade flows simultaneously. Any broken dependency in the new image takes down every CieTrade flow at once. Always smoke-test with ops prefect run-local before calling ship.
ops prefect run-local
Run a flow file directly (uv run python <flow_file>) inside the owning repo, without pushing an image or touching Prefect Cloud. Use this for pre-ship smoke testing: verify the flow logic is sound before committing to a full ship.
Flags
| Flag | Type | Description | Default |
|---|---|---|---|
--repo | TEXT | Project registry alias for the repo that owns the flow file | prefect-azure-infra |
--timeout | INTEGER | Seconds before the local run is killed | 300 |
--json | flag | Output as JSON | — |
Run ops prefect run-local → confirm it succeeds → then call ops prefect ship --confirm. This two-step pattern catches most regressions before the shared image is rebuilt.
ops prefect probe
Send an authenticated HTTP request to a JSON API using credentials stored in a Prefect credentials block, and validate the response. Exits non-zero if the response returns fewer records than --min-records. Use this as a lockout gate before deploying or running a flow that depends on the upstream API.
Flags
| Flag | Type | Description | Default |
|---|---|---|---|
--block | TEXT | Prefect credentials block in "block-type/block-name" format | — |
--user-field | TEXT | Field name in the block that holds the username/user ID | user_id |
--user-param | TEXT | Query-parameter name to send the user value as | UserID |
--secret-field | TEXT | Field name in the block that holds the secret/API key | auth_key |
--param | TEXT (repeatable) | Additional KEY=VALUE query parameters | — |
--min-records | INTEGER | Minimum number of records the response must contain; exits non-zero below this threshold | 1 |
--json | flag | Output as JSON | — |
ops prefect doctor
Run the "Scheduled but never starts" checklist for a deployment. Checks the entrypoint path, service-principal permissions, ACI quota, and image tag — the four most common causes of a deployment that queues forever. Exits non-zero on any hard failure. Gracefully skips az-dependent checks when the Azure CLI is not available.
Flags
| Flag | Type | Description | Default |
|---|---|---|---|
--deployment | TEXT | "flow/deployment" to diagnose; omit to run workspace-level checks only | — |
--pool | flag | Also check work pool health and agent heartbeat | — |
--acr | flag | Also verify that the ACR is reachable and the service principal can pull images | — |
--image | flag | Also resolve and validate the image tag referenced by the deployment | — |
--json | flag | Output as JSON | — |
If a deployment shows Scheduled in the Prefect UI but no run ever appears as Running, run ops prefect doctor --deployment "<flow>/<dep>" --pool --acr --image before digging into logs. It surfaces the four most common root causes in under a minute.