Skip to content

[air] Add air list/get provisioned_capacity commands - #6595

Open
vinchenzo-db wants to merge 2 commits into
mainfrom
air-provisioned-capacity
Open

[air] Add air list/get provisioned_capacity commands#6595
vinchenzo-db wants to merge 2 commits into
mainfrom
air-provisioned-capacity

Conversation

@vinchenzo-db

@vinchenzo-db vinchenzo-db commented Sep 9, 2026

Copy link
Copy Markdown
Contributor

Adds two read commands for pre-provisioned AI Runtime capacity reservations (the same reservation a run targets via compute.provisioned_capacity_id):

  • air list provisioned_capacity — the workspace's reservations (id, accelerator type, reserved accelerators).
  • air get provisioned_capacity <id> — one reservation plus its accelerator usage summary (used / idle).

Both call AiWorkflowService's purpose-built public read API (GET /api/2.0/ai-training/provisioned-capacities[/{id}]) via a raw client.Do, matching the sibling air list workflows call. The platform tracks these internally as "guaranteed capacity"; the public surface (and this CLI) calls them "provisioned capacity". Usage is populated only by Get, so the list view omits it. Output uses the same JSON envelope + text pattern as the other air commands, and the SAFE-gate the server returns before rollout is surfaced as a clear FEATURE_DISABLED error rather than a retryable one.

Tests: unit tests for the projections, resource-name handling, int64-as-string decoding, pagination, and the JSON/text/not-found/feature-disabled paths; an acceptance test drives the real binary against a mocked endpoint (text + JSON goldens). Verified live against e2-dogfood: the CLI reaches the deployed handler and classifies its responses correctly (the API is SAFE-gated off there, so the successful list/get shape is covered by the goldens).

Changes

Why

Tests

e2e test

$ cd /home/v.chen/databricks-cli-wt-pcap                                                                                                                    
go build -o /tmp/air ./
# v.chen at ip-10-90-27-126 in ~/databricks-cli-wt-pcap (git:air-provisioned-capacity) [20:01:15]
$ /tmp/air experimental air list provisioned_capacity -p eng-nephos-e2
ID                                       ACCELERATOR    RESERVED
cap-uitest-nephos-a10g-test              GPU_1xA10      3
# v.chen at ip-10-90-27-126 in ~/databricks-cli-wt-pcap (git:air-provisioned-capacity) [20:07:54]
$ /tmp/air experimental air list provisioned_capacity -p eng-nephos-e2 -o json
{
  "v": 1,
  "ts": "2026-09-11T20:08:12Z",
  "data": {
    "provisioned_capacities": [
      {
        "provisioned_capacity_id": "cap-uitest-nephos-a10g-test",
        "accelerator_type": "GPU_1xA10",
        "reserved_accelerators": 3
      }
    ]
  }
}
# v.chen at ip-10-90-27-126 in ~/databricks-cli-wt-pcap (git:air-provisioned-capacity) [20:08:12]
$ /tmp/air experimental air get provisioned_capacity cap-uitest-nephos-a10g-test -p eng-nephos-e2 -o json
{
  "v": 1,
  "ts": "2026-09-11T20:08:40Z",
  "data": {
    "provisioned_capacity_id": "cap-uitest-nephos-a10g-test",
    "accelerator_type": "GPU_1xA10",
    "reserved_accelerators": 3,
    "used_accelerators": 0,
    "idle_accelerators": 3
  }
}
# v.chen at ip-10-90-27-126 in ~/databricks-cli-wt-pcap (git:air-provisioned-capacity) [20:08:40]
$ /tmp/air experimental air list --help
/tmp/air experimental air get --help
List your active runs for the current profile (use --all-status for finished runs)

Usage:
  databricks experimental air list [flags]
  databricks experimental air list [command]

Available Commands:
  provisioned_capacity List the pre-provisioned AI Runtime capacity reservations for the current workspace

Flags:
      --all-status           Show runs in all states (default: active only)
      --all-users            Show runs from all users
      --filter stringArray   Filter runs, e.g. experiment=foo* (repeatable)
  -h, --help                 help for list
      --limit int            Maximum number of runs to show (default 20)

Global Flags:
      --debug            enable debug logging
  -o, --output type      output type: text or json (default text)
  -p, --profile string   ~/.databrickscfg profile
  -t, --target string    bundle target to use (if applicable)

Use "databricks experimental air list [command] --help" for more information about a command.
Show status, configuration, and timing details for a specific run

Usage:
  databricks experimental air get JOB_RUN_ID [flags]
  databricks experimental air get [command]

Available Commands:
  provisioned_capacity Show a pre-provisioned AI Runtime capacity reservation, including its accelerator usage

Flags:
  -h, --help   help for get

Global Flags:
      --debug            enable debug logging
  -o, --output type      output type: text or json (default text)
  -p, --profile string   ~/.databrickscfg profile
  -t, --target string    bundle target to use (if applicable)

Use "databricks experimental air get [command] --help" for more information about a command.
# v.chen at ip-10-90-27-126 in ~/databricks-cli-wt-pcap (git:air-provisioned-capacity) [20:08:54]
$ 

Adds two read commands for pre-provisioned AI Runtime capacity reservations
(the same reservation a run targets via compute.provisioned_capacity_id):

- `air list provisioned_capacity` — the workspace's reservations
  (id, accelerator type, reserved accelerators).
- `air get provisioned_capacity <id>` — one reservation plus its accelerator
  usage summary (used / idle).

Both call AiWorkflowService's purpose-built public read API
(GET /api/2.0/ai-training/provisioned-capacities[/{id}]) via a raw client.Do,
matching the sibling `air list` workflows call. The platform tracks these
internally as "guaranteed capacity"; the public surface (and this CLI) calls
them "provisioned capacity". Usage is populated only by Get, so the list view
omits it. Output uses the same JSON envelope + text pattern as the other air
commands, and the SAFE-gate the server returns before rollout is surfaced as a
clear FEATURE_DISABLED error rather than a retryable one.

Tests: unit tests for the projections, resource-name handling, int64-as-string
decoding, pagination, and the JSON/text/not-found/feature-disabled paths; an
acceptance test drives the real binary against a mocked endpoint (text + JSON
goldens). Verified live against e2-dogfood: the CLI reaches the deployed
handler and classifies its responses correctly (the API is SAFE-gated off there,
so the successful list/get shape is covered by the goldens).

Co-authored-by: Isaac <no-reply@databricks.com>
@eng-dev-ecosystem-bot

eng-dev-ecosystem-bot commented Sep 9, 2026

Copy link
Copy Markdown
Collaborator

Integration test report

Commit: 204b91e

Run: 34642427409

Env 🔄​flaky 💚​RECOVERED ✅​pass 🙈​skip Time
💚​ aws linux 1 275 17 6:28
💚​ aws windows 1 277 15 5:56
💚​ azure linux 1 274 17 6:24
💚​ azure windows 1 276 15 4:47
🔄​ gcp linux 1 1 274 17 7:23
💚​ gcp windows 1 277 15 6:21
Test Name aws linux aws windows azure linux azure windows gcp linux gcp windows
💚​ TestAccept 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R
🔄​ TestSyncIncrementalSyncFileToPythonNotebook ✅​p ✅​p ✅​p ✅​p 🔄​f ✅​p
Top 3 slowest tests (at least 2 minutes):
duration env testname
4:37 gcp windows TestAccept
3:24 azure windows TestAccept
3:22 aws windows TestAccept

- Rename the command from `provisioned_capacity` to `provisioned-capacity`:
  the repo-wide TestCommandsDontUseUnderscoreInName forbids `_` in a cobra
  command name (underscore is the command separator in logs).
- Replace the pagination loop's index counter with an integer range loop
  (golangci-lint intrange).
- Regenerate the affected acceptance goldens (help + provisioned-capacity).

Co-authored-by: Isaac <no-reply@databricks.com>

@ben-hansen-db ben-hansen-db left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some blocking comments, let's align first before merge.

return nil, fmt.Errorf("failed to create API client: %w", err)
}
var pc provisionedCapacity
if err := apiClient.Do(ctx, http.MethodGet, provisionedCapacityPath+"/"+id, nil, nil, nil, &pc); err != nil {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

claude:

Both raw workspace calls omit the required workspace routing header. Unified hosts cannot reliably route these requests to Config.WorkspaceID.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

suggests instead

headers := auth.WorkspaceIDHeaders(w.Config)
apiClient.Do(ctx, http.MethodGet, provisionedCapacityPath, headers, nil, query, &resp)


pc, err := getProvisionedCapacity(ctx, w, id)
if err != nil {
if errors.Is(err, apierr.ErrResourceDoesNotExist) {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what about other not found errors or http 404?

apierr.ErrNotFound is broader

databricks experimental air list [command]

Available Commands:
provisioned-capacity List the pre-provisioned AI Runtime capacity reservations for the current workspace

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is very long... is the the phrase we want to use databricks air list provisioned-capacity??

databricks air list capacity or reservations, pools, commitments

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants