feat: introduce API-based execution engine as an alternative to legacy shell script orchestration

.github/workflows/ci.yml

@@ -0,0 +1,38 @@
+name: Dataproc Custom Image
+
+on:
+  push:
+    branches: [ main ]
+  pull_request:
+    branches: [ main ]
+
+jobs:
+  test:
+    runs-on: ubuntu-latest
+    strategy:
+      matrix:
+        python-version: ["3.10", "3.11", "3.12"]
+
+    steps:
+    - name: Checkout Repository
+      uses: actions/checkout@v6
+
+    - name: Set up Python ${{ matrix.python-version }}
+      uses: actions/setup-python@v6
+      with:
+        python-version: ${{ matrix.python-version }}
+        cache: 'pip'
+
+    - name: Install dependencies
+      run: |
+        python -m pip install --upgrade pip
+        pip install -r requirements.txt
+
+    - name: Run Ruff check (Linter)
+      run: ruff check .
+
+    - name: Run Ruff format check
+      run: ruff format --check .
+
+    - name: Run Unit Tests
+      run: python -m unittest discover -s tests -p "test_*.py"

README.md

@@ -133,6 +133,10 @@ python generate_custom_image.py \
     default value of 300 seconds will be used.
 *   **--dry-run**: Dry run mode which only validates input and generates
     workflow script without creating image. Disabled by default.
+*   **--execution-engine**: The orchestration backend used to manage the custom
+    image creation. Supported values are `cli` (runs legacy generated shell-script
+    via local gcloud commands) and `api` (uses native Google Cloud Python clients).
+    Defaults to `cli`.
 *   **--trusted-cert**: a certificate in DER format to be inserted
     into the custom image's EFI boot sector.  Can be generated by
     reading examples/secure-boot/README.md.  This argument is mutually
@@ -183,13 +187,13 @@ cluster property values in the file.
 
 #### Create a custom image
 
-Create a custom image with name `custom-image-1-5-9` with Dataproc version
-`1.5.9-debian10`:
+Create a custom image with name `custom-image-2-2-0` with Dataproc version
+`2.2.0-debian12`:
 
 ```shell
 python generate_custom_image.py \
-    --image-name custom-image-1-5-9 \
-    --dataproc-version 1.5.9-debian10 \
+    --image-name custom-image-2-2-0 \
+    --dataproc-version 2.2.0-debian12 \
     --customization-script ~/custom-script.sh \
     --metadata 'key1=value1,key2=value2' \
     --zone us-central1-f \
@@ -200,10 +204,37 @@ python generate_custom_image.py \
 
 ```shell
 python generate_custom_image.py \
-    --image-name custom-image-1-5-9 \
-    --dataproc-version 1.5.9-debian10 \
+    --image-name custom-image-2-2-0 \
+    --dataproc-version 2.2.0-debian12 \
     --customization-script ~/custom-script.sh \
     --zone us-central1-f \
     --gcs-bucket gs://my-test-bucket \
     --no-smoke-test
 ```
+
+### API-based Execution Engine (Optional)
+
+The tool supports an optional API-based execution engine that communicates directly with Google Cloud services using official Python Client Libraries instead of generating and running local shell scripts.
+
+#### Setup
+
+To use the API-based execution engine, install the required packages:
+
+```shell
+pip install -r requirements.txt
+```
+
+#### Running in API Mode
+
+To invoke the custom image generator using the API engine, pass `--execution-engine api`:
+
+```shell
+python generate_custom_image.py \
+    --image-name custom-image-2-2-0 \
+    --dataproc-version 2.2.0-debian12 \
+    --customization-script ~/custom-script.sh \
+    --zone us-central1-f \
+    --gcs-bucket gs://my-test-bucket \
+    --execution-engine api
+```
+