diff --git a/.github/workflows/dev.yml b/.github/workflows/dev.yml index 7f38a2ea..24349702 100644 --- a/.github/workflows/dev.yml +++ b/.github/workflows/dev.yml @@ -58,6 +58,11 @@ jobs: run: python ./test/unit_test/keyboard/hotkey_test.py # Mouse tests + # These three drive the real mouse and the real exit path on a hosted + # runner with no desk in front of it, so a failure says more about the + # runner's session than about the change under test. They are demo + # scripts (CLAUDE.md: the *_test.py files run on import), not the CI + # gate -- that is pytest-headless in quality.yml. - name: Test Mouse Module run: python ./test/unit_test/mouse/mouse_test.py continue-on-error: true diff --git a/.github/workflows/platform-smoke.yml b/.github/workflows/platform-smoke.yml index f22bb1a4..3d7abbdb 100644 --- a/.github/workflows/platform-smoke.yml +++ b/.github/workflows/platform-smoke.yml @@ -16,30 +16,38 @@ jobs: matrix: # arm64 is not a rounding error on the desktop any more, and the # dependency set is where it shows. macos-14 is already arm64; - # ubuntu-22.04-arm adds Linux, and it passes. + # ubuntu-22.04-arm adds Linux, and windows-11-arm is back. # - # windows-11-arm is deliberately absent, and it was measured rather - # than assumed. Two dependencies have no win_arm64 wheel, and both - # have to go before the runner is worth adding back: + # windows-11-arm installs a smaller dependency set than every other + # square, and that is deliberate rather than accidental. Two packages + # publish no win_arm64 wheel, so pyproject.toml marks them off this + # one platform: # - # opencv-python — no win_arm64 wheel in any version, so pip falls - # back to building from source and CMake cannot + # opencv-python — no win_arm64 wheel in any version, so pip fell + # back to building from source and CMake could not # configure for ARM64. Twelve minutes, then failure. # cryptography — wheels stop at 46.0.3; 46.0.4 onwards ship none. # Our floor is >=48.0.1 and that is a security floor # (GHSA-537c-gmf6-5ccf), so it cannot be lowered. # - # Neither is a CI problem to work around — the package genuinely - # cannot be installed on Windows arm64 today. Re-check without a - # runner, in about ten seconds: + # Nothing in the package imports either one at import time, so what + # this square proves is real: the install succeeds and the stable API + # works. Image matching, action signing, ACME and encrypted recording + # do not work there, and Progress.md says so. Re-check upstream in + # about ten seconds, no runner required: # # pip install --dry-run --only-binary=:all: --platform win_arm64 \ # --python-version 3.12 --target /tmp/probe \ # 'opencv-python>=4.8,<6' 'cryptography>=48.0.1' # - # Recorded in Progress.md; add the runner back when both resolve. - os: [windows-2022, ubuntu-22.04, macos-14, ubuntu-22.04-arm] + # When both resolve, drop the markers from pyproject.toml. + os: [windows-2022, ubuntu-22.04, macos-14, ubuntu-22.04-arm, windows-11-arm] python-version: ["3.10", "3.14"] + exclude: + # CPython's official Windows arm64 builds start at 3.11, so + # setup-python has no 3.10 interpreter to fetch on this runner. + - os: windows-11-arm + python-version: "3.10" runs-on: ${{ matrix.os }} steps: - uses: actions/checkout@v4 diff --git a/.github/workflows/stable.yml b/.github/workflows/stable.yml index 02b068cd..bccd9f48 100644 --- a/.github/workflows/stable.yml +++ b/.github/workflows/stable.yml @@ -62,6 +62,11 @@ jobs: run: python ./test/unit_test/keyboard/hotkey_test.py # Mouse tests + # These three drive the real mouse and the real exit path on a hosted + # runner with no desk in front of it, so a failure says more about the + # runner's session than about the change under test. They are demo + # scripts (CLAUDE.md: the *_test.py files run on import), not the CI + # gate -- that is pytest-headless in quality.yml. - name: Test Mouse Module run: python ./test/unit_test/mouse/mouse_test.py continue-on-error: true diff --git a/CHANGELOG.md b/CHANGELOG.md index 7af7c5f6..06d2167a 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -10,6 +10,18 @@ only when documented here with a migration path. ### Added +- **Windows on arm64 installs.** `opencv-python`, `cryptography` and + `je_open_cv` now carry the environment marker + `sys_platform != 'win32' or platform_machine != 'ARM64'`, because none of + the three publishes a `win_arm64` wheel and `pip install je_auto_control` + therefore failed on that platform before any of this code ran. Every other + platform resolves exactly the same dependency set as before. On Windows + arm64, the features that need those wheels — `find_image*`, the OpenCV + `screenshot()`, the secret vault, action-file encryption, ACME/TLS and + encrypted recording — raise a `RuntimeError` or `ImportError` naming the + missing wheel rather than a bare `ModuleNotFoundError`. Python 3.11 is the + floor there, since CPython publishes no official Windows arm64 build for + 3.10. - **The macOS recorder works.** `record()`, `stop_record()`, `stop_record_timeline()`, the `AC_record*` commands, the `ac_record_*` MCP tools and `je_auto_control record` all run on macOS now; they used to refuse diff --git a/Progress.md b/Progress.md index 009c2bae..fe0d2f81 100644 --- a/Progress.md +++ b/Progress.md @@ -50,53 +50,69 @@ --- -## Windows arm64 裝不起來——是兩個上游,不是一個 +## Windows arm64:裝得起來了,但少了影像與加密 -`BLOCKED` — 上游(`opencv-python` 沒有 win_arm64 wheel;`cryptography` 在安全下限之上也沒有) +`TODO` — 上游仍未發 wheel(`opencv-python`、`cryptography`),但安裝本身不再是卡點 -`windows-11-arm` 加進 `platform-smoke.yml` 的矩陣跑了一次, -結果是實測而不是推測:**opencv-python 並沒有發 -win_arm64 wheel**,pip 回退到從原碼建,CMake 在 ARM64 上 -configure 不起來,花了十二分鐘失敗。所以那一格已從矩陣 -移除,並把原因寫在 workflow 的註解裡。 +這一項曾經是 `BLOCKED`,而那個判斷只對一半。上游確實沒有發 wheel, +這件事到今天(2026-08-20)重新實測依舊成立;但「裝不起來」卡的不是程式, +是 `pyproject.toml` 無條件要求那兩個套件。實測:把 `cryptography`、`cv2`、 +`je_open_cv`、`numpy`、`PIL` 五個全擋掉之後,`import je_auto_control`、executor、 +MCP 工具表、`cli`、`api.generate_code`、`api.create_failure_bundle` **全部照常跑**。 -門面已經不在 module scope import OpenCV 了(見 -[WHATS_NEW.md](WHATS_NEW.md)),但這裡卡的不是 import -而是 **pip 裝不起來**:那幾個套件仍列在 `pyproject.toml` -的 `dependencies`,`pip install -e .` 第一步就會去建它們。 +所以修法是一個 PEP 508 環境標記,三個相依共用同一個: -### 2026-08-20 重新實測:當初只數到一半 +``` +sys_platform != 'win32' or platform_machine != 'ARM64' +``` + +`windows-11-arm` 已經回到 `platform-smoke.yml` 的矩陣(只跑 3.14,CPython 的 +官方 win-arm64 build 從 3.11 才有)。其他平台拿到的東西一個位元都沒變。 + +### 還沒有答案的:Windows arm64 上這些功能不能用 -不必開 runner——`pip` 可以替別的平台解析,十秒就給出答案: +裝得起來不等於功能齊。該平台上以下四組會拋帶提示的錯誤,而不是默默失效: -| 依賴 | win_arm64 | 實測 | +| 功能 | 缺的是 | 錯誤形式 | | --- | --- | --- | -| `opencv-python>=4.8,<6` | **沒有** | 任何版本都沒有,pip 回的是 `from versions: none`。`je_open_cv` 自己是純 Python,但它相依 opencv-python,所以一起卡。 | +| 影像比對、截圖轉 BGR、螢幕錄影 | `opencv-python`/`je_open_cv` | `utils/cv2_utils/optional.py` 的 `require_cv2()`/`require_je_open_cv()` 拋 `RuntimeError` | +| 動作檔加密(`action_signing`) | `cryptography` | `_fernet_types()` 拋 `RuntimeError`(簽章本身是 HMAC,不受影響) | +| 秘密金庫(`${secrets.NAME}`) | `cryptography` | 同上 | +| ACME/TLS 發證、加密錄影 | `cryptography` | 模組層 `ImportError` 轉述(照 `webrtc_transport` 慣例) | + +這四組在 arm64 上能不能回來,**完全取決於上游**: + +| 依賴 | win_arm64 | 實測(2026-08-20) | +| --- | --- | --- | +| `opencv-python>=4.8,<6` | **沒有** | 任何版本都沒有,pip 回的是 `from versions: none`。`je_open_cv` 自己是純 Python,但相依 opencv-python,所以一起卡——標記也必須一起下。 | | `cryptography>=48.0.1` | **沒有** | wheel 只出到 **46.0.3**,46.0.4 起上游就不再發 win_arm64。而 `>=48.0.1` 是 347ec1e 為了 GHSA-537c-gmf6-5ccf(high)訂的**安全下限**,不能為了 arm64 降回去。 | -| `pillow==12.3.0` | 有 | `pillow-12.3.0-cp3xx-win_arm64.whl` 一直都在。**原本這裡寫「把 OpenCV/Pillow 移到 extra」,Pillow 那半是猜的,它從來不是卡點。** | -| `mss`/`defusedxml`/`je_open_cv` | 有 | 純 Python。 | +| `pillow==12.3.0` | 有 | `pillow-12.3.0-cp3xx-win_arm64.whl` 一直都在。**曾經被寫成卡點,那是猜的,它從來不是。** | +| `mss`/`defusedxml` | 有 | 純 Python。這三個加上 Pillow 就是 arm64 實際裝到的全部。 | | `PySide6==6.11.1`/`qt-material==2.17` | 有 | `[gui]` extra 在 arm64 上裝得起來。 | | `aiortc` | **沒有** | 卡在傳遞相依 `google-crc32c`,與本專案的選擇無關;`av` 自己有 wheel。 | -**所以原本那句「把 OpenCV/Pillow 移到 optional extra,arm64 就能只裝輸入的部分」 -是不成立的**——就算 OpenCV 移走,`cryptography` 還是會把 `pip install` 擋在同一個 -地方,而它的下限是安全下限,沒有往下讓的空間。要真的讓 arm64 裝得起來,**兩個都得 -離開必裝集合**;`cryptography` 今天被六個模組用到(`acme_v2`、`tls_acme`、 -`action_signing`、`secrets`、`remote_desktop` 的加密錄影),那是比 OpenCV 更大的 -相容性決定。沒有人要求之前不做。 - 重驗指令(不需要 arm64 機器,也不需要 runner): ```bash pip install --dry-run --only-binary=:all: --platform win_arm64 --python-version 3.12 --target /tmp/probe 'opencv-python>=4.8,<6' 'cryptography>=48.0.1' ``` -兩行 `ERROR: No matching distribution` 就是現況。哪天其中一行不見了,就是上游發了 -wheel,那時把 `windows-11-arm` 加回 `platform-smoke.yml` 的矩陣。 +兩行 `ERROR: No matching distribution` 就是現況。**哪天其中一行不見了,就把 +`pyproject.toml` 上那個標記拿掉**(三行一起), +`test/unit_test/headless/test_arm64_dependency_markers.py` 會帶著你改完。 -**Linux arm64 是好的**——`ubuntu-22.04-arm` 兩個 Python 版本 -都綠,macOS 本來就是 arm64。所以卡住的只有 Windows -這一個組合。 +注意一個驗證上的陷阱:**`pip --platform` 不會換掉 marker 的評估環境**, +它只影響 wheel 相容性標籤,所以拿本機做 `--dry-run` **驗不到標記的效果**(兩個 +套件依舊會被要求)。能驗的是兩件事:直接評估 marker(上面那支測試在做的), +以及 `windows-11-arm` 那一格自己綠。 + +### 一個刻意的取捨:cv2 只包兩扇門 + +`cv2` 在 33 個檔、共 76 句 import,全部是函式內 lazy。這次**只**在兩個大家一定會 +經過的門換成 `require_cv2()`/`require_je_open_cv()`:`wrapper/auto_control_screen.py`(截圖) +與 `utils/cv2_utils/template_detection.py`(樣板比對)。其餘七十幾句維持原樣,在 arm64 上 +會得到 `ModuleNotFoundError: No module named 'cv2'`。全包一輪是大面積 diff,且對呼叫端 +並沒有多提供可以行動的資訊——哪天語意不足再說。 ## Wayland:剩下的都不是「缺一台機器」 @@ -181,3 +197,53 @@ capability enum 值與 variadic `ei_seat_bind_capabilities`、event-type enum **緩解**:驗不到的擷取部分有逃生門——`JE_AUTOCONTROL_WAYLAND_CAPTURE_COMMAND` 讓操作者 直接指定自己的擷取指令(`{output}` 會被換成暫存 PNG 路徑),優先於所有偵測。 + +--- + +## libei 的 `ei_unref` 在半開交握上會 SIGSEGV + +`BLOCKED` — 上游(libei 1.3.901) + +`linux_wayland/libei.py` 的 `_teardown` **刻意每個行程漏一個 context 與一個 fd**, +因為對一個還沒完成交握的 handle 呼叫 `ei_unref` 會直接 SIGSEGV。 +這是在驅動使用者桌面的函式庫裡的 crash,所以寧可漏也不能當。 + +**這條本來就該在這裡。** 兩支 verify 腳本都會印 `*** REVISIT ***` 並叫讀者 +來翻 `Progress.md`,而這裡一直什麼都沒寫: + +- `docker/libei_verify.py`:「The workaround in `LibeiBackend._teardown` can probably go」 +- `docker/eis_verify.py`:「`ei_unref` now SEGFAULTS on a live context too」 + +重驗方式就是跑那兩支腳本(`eis-verification` job 已經在跑);哪天 banner 不再 +出現,就把 `_teardown` 的迴避拿掉。形狀與 arm64 那條一樣:卡上游、有一行重驗。 + +--- + +## 三個 Qt thread-marshal 測試永久跳過中 + +`TODO` — 需要子行程隔離,做法已知 + +`test/unit_test/headless/test_r3_gui_thread_marshal.py` 裡三個測試被無條件 skip: +`test_panel_signals_expose_file_received`、`test_webrtc_received_file_marshaled_to_gui`、 +`test_thumbnail_poll_thread_is_reaped`。skip 理由自己寫著「needs subprocess +isolation (see test_actions_menu_gui) … skip until then」——那句「until then」就是這條。 + +跟 `CLAUDE.md` §Testing 記的 0xC0000409 `__fastfail` 是同一個家族: +worker→GUI 的 teardown 在共用的 pytest 行程裡把整個監獸帶走。 +`test_actions_menu_gui.py` 已經示範過解法(把建 widget 的部分丟進子行程), +這三個只是還沒改過去。沒改之前,這三條路徑沒有任何回歸保護。 + +--- + +## 兩個講好要爬、還沒爬的門檻 + +`TODO` — 兩者都寫在 `pyproject.toml` 的註解裡,但不在任何待辦清單上 + +- **覆蓋率**:`fail_under = 35`,註解寫著「Raise toward 70 as legacy modules are + brought under the stable API contract」。目標是 70,今天是 35,中間沒有計畫。 +- **mypy 範圍**:CI 只型別檢查兩條路徑(`quality.yml` 的 + `mypy je_auto_control/api je_auto_control/utils/failure_bundle`)。註解寫著 + 「followed legacy modules are analysed for signatures but not reported until they + join the contract」——同樣是講好要擴、還沒擴。 + +兩者都不是一次做得完的事,但放在這裡至少讓「下一步是什麼」有一個地方可寫。 diff --git a/README.md b/README.md index d57eb80d..0114f4df 100644 --- a/README.md +++ b/README.md @@ -54,7 +54,19 @@ Optional extras, installed only when you need them: | `fuzzy` / `locale` | `rapidfuzz` matching, `babel` locale parsing | | `s3` / `audio` | S3 artifact store, system volume control | -**Requirements:** Python ≥ 3.10. On Linux, install build prerequisites first: +**Windows on arm64** installs and runs, minus what upstream cannot ship +there: neither `opencv-python` nor `cryptography` publishes a `win_arm64` +wheel. So `find_image*`, `screenshot()` (the OpenCV/BGR one — the Pillow +capture still works), the secret vault, action-file encryption, ACME/TLS +and encrypted recording each raise a message naming the missing wheel +instead of failing obscurely. Mouse, keyboard, screen size, window +management, the accessibility tree, the action executor, the MCP/REST/TCP +servers and the GUI all work — measured, not assumed. Every other platform +is unaffected. + +**Requirements:** Python ≥ 3.10 (≥ 3.11 on Windows arm64, which is where +CPython's official builds for it start). On Linux, install build +prerequisites first: ```bash sudo apt-get install cmake libssl-dev diff --git a/README/README_zh-CN.md b/README/README_zh-CN.md index 452e56c8..8f153676 100644 --- a/README/README_zh-CN.md +++ b/README/README_zh-CN.md @@ -48,7 +48,15 @@ pip install je_auto_control[gui] # 加上 PySide6 桌面应用 | `fuzzy` / `locale` | `rapidfuzz` 模糊匹配、`babel` 区域解析 | | `s3` / `audio` | S3 制品存储、系统音量控制 | -**系统需求:** Python ≥ 3.10。Linux 请先安装构建依赖: +**Windows arm64** 装得起来也跑得起来,少的是上游在那里发不出来的那些: +`opencv-python` 与 `cryptography` 都没发 `win_arm64` wheel。所以 `find_image*`、 +`screenshot()`(OpenCV/BGR 那一支——Pillow 截图仍可用)、密钥金库、动作文件加密、 +ACME/TLS 与加密录影会抛出指名缺哪个 wheel 的错误,而不是难以追查的失败。 +鼠标、键盘、屏幕尺寸、窗口管理、无障碍树、动作执行器、MCP/REST/TCP 服务器 +与 GUI 都正常——这是实测的,不是推论的。其他平台不受影响。 + +**系统需求:** Python ≥ 3.10(Windows arm64≥ 3.11,CPython 官方构建从那里开始)。 +Linux 请先安装构建依赖: ```bash sudo apt-get install cmake libssl-dev diff --git a/README/README_zh-TW.md b/README/README_zh-TW.md index 97e31d4b..7c15911a 100644 --- a/README/README_zh-TW.md +++ b/README/README_zh-TW.md @@ -48,7 +48,15 @@ pip install je_auto_control[gui] # 加上 PySide6 桌面應用程式 | `fuzzy` / `locale` | `rapidfuzz` 模糊比對、`babel` 地區解析 | | `s3` / `audio` | S3 產出物儲存、系統音量控制 | -**系統需求:** Python ≥ 3.10。Linux 請先安裝建置前置套件: +**Windows arm64** 裝得起來也跑得起來,少的是上游在那裡發不出來的那些: +`opencv-python` 與 `cryptography` 都沒發 `win_arm64` wheel。所以 `find_image*`、 +`screenshot()`(OpenCV/BGR 那一支——Pillow 截圖仍可用)、秘密金庫、動作檔加密、 +ACME/TLS 與加密錄影會拋出指名缺哪個 wheel 的錯誤,而不是難以追查的失敗。 +滑鼠、鍵盤、螢幕尺寸、視窗管理、無障礙樹、動作執行器、MCP/REST/TCP 伺服器 +與 GUI 都正常——這是實測的,不是推論的。其他平台不受影響。 + +**系統需求:** Python ≥ 3.10(Windows arm64≥ 3.11,CPython 官方建置從那裡開始)。 +Linux 請先安裝建置前置套件: ```bash sudo apt-get install cmake libssl-dev diff --git a/README/WHATS_NEW_zh-CN.md b/README/WHATS_NEW_zh-CN.md index 0a07ea53..5fba5d44 100644 --- a/README/WHATS_NEW_zh-CN.md +++ b/README/WHATS_NEW_zh-CN.md @@ -2,6 +2,40 @@ ## 本次更新 (2026-08-20) — 声称支援的平台,這回真的量過了 +### Windows arm64 从来不是代码的问题 + +这一项原本标成 `BLOCKED`,而那只对一半。两个依赖确实没发 +`win_arm64` wheel,到今天依旧没有:`opencv-python` 任何版本都没有, +`cryptography` 停在 46.0.3,而本项目的下限 `>=48.0.1` 是安全下限 +(GHSA-537c-gmf6-5ccf),不能往下让。所以 `pip install` 回退到从源码构建 +OpenCV,CMake 在 ARM64 上 configure 不起来,十二分钟后失败。这一半是量过的, +也是那一格当初离开矩阵的原因。 + +没量的是真正重要的那一半。**包里没有任何东西在 import 时需要那两个 +wheel。** 在子进程里挡掉 `cryptography`、`cv2`、`je_open_cv`、`numpy`、`PIL` +之后,`import je_auto_control` 依旧绑定全部 1,238 个公开名称,executor、MCP +工具表、CLI、`api.generate_code`、`api.create_failure_bundle` 也都 import 得起来并跑得动。 +卡点完全在 `pyproject.toml` 的依赖清单上。 + +**所以修法是一个标记,不是一套架构。** 三个依赖带上 +`sys_platform != 'win32' or platform_machine != 'ARM64'`:`opencv-python`、 +`cryptography`,以及 `je_open_cv`——它自己是纯 Python,但依赖 OpenCV, +不一起标就会从侧门把 OpenCV 拉回来。Pillow 刻意不标:它一直都有 +`win_arm64` wheel,先前把它写成卡点那句是猜的。`windows-11-arm` 已回到 +`platform-smoke.yml`,只跑 3.14,因为 CPython 的官方 Windows arm64 build 从 3.11 才有。 + +**Windows arm64 放弃了什么,现在写在错误信息本身里。** +`find_image*`、OpenCV 那支 `screenshot()`、密钥金库、动作文件加密、ACME/TLS +与加密录影都会抛出同时指名“缺哪个 wheel”与“哪个平台”的信息,而不是一句 +读起来像安装坏掉的 `ModuleNotFoundError`。`utils/cv2_utils/optional.py` 的两个 +取用口盖住每条图像路径必经的两扇门;其余七十几句 lazy `import cv2` 刻意不动, +因为包起来并没有多给调用方可以行动的信息。 + +把关的是一支刻意不华丽的测试: +`test/unit_test/headless/test_arm64_dependency_markers.py` 读 `pyproject.toml`, +对五组平台实际评估那个 marker——有人把它拿掉,或“顺手”把 Pillow 也收进去, +都会当场红,而不是让一个 arm64 使用者少一个能用的功能。 + 整套测试一直只在 `windows-2022` 跑,另加容器裡一次 Linux 執行。macOS 只跑兩行指令。Wayland 有五個 job 對真的對等體 讀回輸入;X11——兩條 Linux 路徑中更老、部署更廣的那一 @@ -61,7 +95,9 @@ AX 樹也走得出真的元素。這是先量再斷言的,而且探針在期 system」,七個 X11 後端模組又各自帶一份同樣的 Linux 專屬守衛。 新的 `utils/platform_id` 是唯一的判定點,`freebsd` job 在 runner 裡開 真的 FreeBSD 14 VM,在真的 X server 上 import X11 模組並把游標移完讀回。 -`ubuntu-22.04-arm` 加进 smoke 矩阵且全绿。`windows-11-arm` 试过后拿掉了,而重新实测又挖出当初漏掉的**第二个**卡点:opencv-python 任何版本都没发 `win_arm64` wheel,cryptography 则从 46.0.4 起不再发,而本项目的下限 `>=48.0.1` 是安全下限(GHSA-537c-gmf6-5ccf),不能为了凑 wheel 往下让。原本跟 OpenCV 并列的 Pillow 其实一直都有 arm64 wheel,从来不是卡点。这些都不需要 arm64 机器就验得到——`pip install --dry-run --only-binary=:all: --platform win_arm64` 十秒给答案,指令已连同结论记在 `Progress.md`。 +`ubuntu-22.04-arm` 加进 smoke 矩阵且全绿。(**同一天已被取代**——见本档最上方 +“Windows arm64 从来不是代码的问题”:runner 已经回来了,因为卡的是依赖清单而不是代码。) +`windows-11-arm` 试过后拿掉了,而重新实测又挖出当初漏掉的**第二个**卡点:opencv-python 任何版本都没发 `win_arm64` wheel,cryptography 则从 46.0.4 起不再发,而本项目的下限 `>=48.0.1` 是安全下限(GHSA-537c-gmf6-5ccf),不能为了凑 wheel 往下让。原本跟 OpenCV 并列的 Pillow 其实一直都有 arm64 wheel,从来不是卡点。这些都不需要 arm64 机器就验得到——`pip install --dry-run --only-binary=:all: --platform win_arm64` 十秒给答案,指令已连同结论记在 `Progress.md`。 ## 本次更新 (2026-08-19) — Wayland 两个等人拍板的取舍,拍板了 diff --git a/README/WHATS_NEW_zh-TW.md b/README/WHATS_NEW_zh-TW.md index ef67d949..928718be 100644 --- a/README/WHATS_NEW_zh-TW.md +++ b/README/WHATS_NEW_zh-TW.md @@ -2,6 +2,40 @@ ## 本次更新 (2026-08-20) — 嬣稱支援的平台,這回真的量過了 +### Windows arm64 從來不是程式的問題 + +這一項原本標成 `BLOCKED`,而那只對一半。兩個相依確實沒發 +`win_arm64` wheel,到今天依舊沒有:`opencv-python` 任何版本都沒有, +`cryptography` 停在 46.0.3,而本專案的下限 `>=48.0.1` 是安全下限 +(GHSA-537c-gmf6-5ccf),不能往下讓。所以 `pip install` 回退到從原碼建 +OpenCV,CMake 在 ARM64 上 configure 不起來,十二分鐘後失敗。這一半是量過的, +也是那一格當初離開矩陣的原因。 + +沒量的是真正重要的那一半。**套件裡沒有任何東西在 import 時需要那兩個 +wheel。** 在子行程裡擋掉 `cryptography`、`cv2`、`je_open_cv`、`numpy`、`PIL` +之後,`import je_auto_control` 依舊綁定全部 1,238 個公開名稱,executor、MCP +工具表、CLI、`api.generate_code`、`api.create_failure_bundle` 也都 import 得起來並跑得動。 +卡點完全在 `pyproject.toml` 的相依清單上。 + +**所以修法是一個標記,不是一套架構。** 三個相依帶上 +`sys_platform != 'win32' or platform_machine != 'ARM64'`:`opencv-python`、 +`cryptography`,以及 `je_open_cv`——它自己是純 Python,但相依 OpenCV, +不一起標就會從側門把 OpenCV 拉回來。Pillow 刻意不標:它一直都有 +`win_arm64` wheel,先前把它寫成卡點那句是猜的。`windows-11-arm` 已回到 +`platform-smoke.yml`,只跑 3.14,因為 CPython 的官方 Windows arm64 build 從 3.11 才有。 + +**Windows arm64 放棄了什麼,現在寫在錯誤訊息本身裡。** +`find_image*`、OpenCV 那支 `screenshot()`、秘密金庫、動作檔加密、ACME/TLS +與加密錄影都會拋出同時指名「缺哪個 wheel」與「哪個平台」的訊息,而不是一句 +讀起來像安裝壞掉的 `ModuleNotFoundError`。`utils/cv2_utils/optional.py` 的兩個 +取用口蓋住每條影像路徑必經的兩扇門;其餘七十幾句 lazy `import cv2` 刻意不動, +因為包起來並沒有多給呼叫端可以行動的資訊。 + +把關的是一支刻意不華麗的測試: +`test/unit_test/headless/test_arm64_dependency_markers.py` 讀 `pyproject.toml`, +對五組平台實際評估那個 marker——有人把它拿掉,或「順手」把 Pillow 也收進去, +都會當場紅,而不是讓一個 arm64 使用者少一個能用的功能。 + 整套測試一直只在 `windows-2022` 跑,另加容器裡一次 Linux 執行。macOS 只跑兩行指令。Wayland 有五個 job 對真的對等體 讀回輸入;X11——兩條 Linux 路徑中更老、部署更廣的那一 @@ -61,7 +95,9 @@ AX 樹也走得出真的元素。這是先量再斷言的,而且探針在期 system」,七個 X11 後端模組又各自帶一份同樣的 Linux 專屬守衛。 新的 `utils/platform_id` 是唯一的判定點,`freebsd` job 在 runner 裡開 真的 FreeBSD 14 VM,在真的 X server 上 import X11 模組並把游標移完讀回。 -`ubuntu-22.04-arm` 加進 smoke 矩陣且全綠。`windows-11-arm` 試過後拿掉了,而重新實測又挖出當初漏掉的**第二個**卡點:opencv-python 任何版本都沒發 `win_arm64` wheel,cryptography 則從 46.0.4 起不再發,而本專案的下限 `>=48.0.1` 是安全下限(GHSA-537c-gmf6-5ccf),不能為了湊 wheel 往下讓。原本跟 OpenCV 並列的 Pillow 其實一直都有 arm64 wheel,從來不是卡點。這些都不需要 arm64 機器就驗得到——`pip install --dry-run --only-binary=:all: --platform win_arm64` 十秒給答案,指令已連同結論記在 `Progress.md`。 +`ubuntu-22.04-arm` 加進 smoke 矩陣且全綠。(**同一天已被取代**——見本檔最上方 +「Windows arm64 從來不是程式的問題」:runner 已經回來了,因為卡的是相依清單而不是程式。) +`windows-11-arm` 試過後拿掉了,而重新實測又挖出當初漏掉的**第二個**卡點:opencv-python 任何版本都沒發 `win_arm64` wheel,cryptography 則從 46.0.4 起不再發,而本專案的下限 `>=48.0.1` 是安全下限(GHSA-537c-gmf6-5ccf),不能為了湊 wheel 往下讓。原本跟 OpenCV 並列的 Pillow 其實一直都有 arm64 wheel,從來不是卡點。這些都不需要 arm64 機器就驗得到——`pip install --dry-run --only-binary=:all: --platform win_arm64` 十秒給答案,指令已連同結論記在 `Progress.md`。 ## 本次更新 (2026-08-19) — Wayland 兩個等人拍板的取捨,拍板了 diff --git a/WHATS_NEW.md b/WHATS_NEW.md index 457a16ed..b20a210c 100644 --- a/WHATS_NEW.md +++ b/WHATS_NEW.md @@ -2,6 +2,47 @@ ## What's new (2026-08-20) +### Windows arm64 Was Never a Code Problem + +The entry for this said `BLOCKED`, and that was half right. Two dependencies +publish no `win_arm64` wheel and still do not: `opencv-python` has none in any +version, and `cryptography` stopped at 46.0.3 while this project's floor of +`>=48.0.1` is a security floor (GHSA-537c-gmf6-5ccf) that cannot be lowered. +So `pip install` fell back to building OpenCV from source, CMake could not +configure for ARM64, and twelve minutes later the runner failed. That much was +measured, and it is why the runner left the matrix. + +What was not measured is the half that mattered. **Nothing in the package +needs either wheel at import time.** Blocking `cryptography`, `cv2`, +`je_open_cv`, `numpy` and `PIL` in a subprocess, `import je_auto_control` +still binds all 1,238 public names, and the executor, the MCP tool registry, +the CLI, `api.generate_code` and `api.create_failure_bundle` all import and +run. The blocker lived entirely in `pyproject.toml`'s dependency list. + +**So the fix is a marker, not an architecture.** Three requirements carry +`sys_platform != 'win32' or platform_machine != 'ARM64'` — `opencv-python`, +`cryptography`, and `je_open_cv`, which is pure Python but depends on OpenCV +and would otherwise drag it back in through the side door. Pillow is +deliberately not marked: it has always shipped `win_arm64` wheels and the +earlier note calling it a blocker was a guess. `windows-11-arm` is back in +`platform-smoke.yml`, on 3.14 only, because CPython's official Windows arm64 +builds start at 3.11. + +**What Windows arm64 gives up is now said out loud, in the error itself.** +`find_image*`, the OpenCV `screenshot()`, the secret vault, action-file +encryption, ACME/TLS and encrypted recording raise a message naming the +missing wheel and the platform, instead of a bare `ModuleNotFoundError` that +reads like a broken install. Two new accessors in +`utils/cv2_utils/optional.py` cover the two doors every image path goes +through; the other seventy-odd lazy `import cv2` sites are left alone on +purpose, because wrapping them buys the caller nothing it can act on. + +A deliberately unglamorous test guards the whole thing: +`test/unit_test/headless/test_arm64_dependency_markers.py` reads +`pyproject.toml` and evaluates the marker against five platforms, so removing +it — or "tidying" Pillow into it — fails loudly rather than costing an arm64 +user a working feature. + ### The Platforms This Project Claims, Now Measured The suite ran on `windows-2022` alone for its whole life, plus one Linux @@ -239,7 +280,10 @@ had to be loaded by file path to get even that far. See below: that turned out to be the wrong thing to work around, and the job now drives the whole backend. `ubuntu-22.04-arm` joins the smoke matrix and passes; `macos-14` was already -arm64. `windows-11-arm` was tried and removed, and re-measuring turned up a +arm64. (**Superseded the same day** — see "Windows arm64 Was Never a Code +Problem" at the top of this file: the runner is back, because the blocker +was the dependency list rather than the code.) `windows-11-arm` was tried +and removed, and re-measuring turned up a **second** blocker the first pass had missed: opencv-python publishes no `win_arm64` wheel in any version, and cryptography stopped publishing one after 46.0.3 — while this project's floor is `>=48.0.1`, a security floor @@ -490,7 +534,8 @@ recorded as needing a VM running a desktop that consumes libinput devices. `WLR_NO_HARDWARE_CURSORS=1`. Every locator, template match and OCR read goes through that capture, so the pointer punches a pointer-shaped hole in whatever it is sitting on. The check records the behaviour as measured, and - what to do about it is an open item in `Progress.md`. + the decision was to document it rather than work around it — see the + capture section below. ### The Screen-Capture Portal Could Never Have Worked, and a Real Bus Said So @@ -670,8 +715,9 @@ recorded as needing a VM running a desktop that consumes libinput devices. - **What this does not settle.** ydotool's `mousemove --absolute` has an origin of its own — it clamps to the compositor's top-left corner and sends the target as a relative delta — and whether that corner is the layout - origin still needs a compositor that consumes libinput devices. It stays - open in `Progress.md` rather than being changed on a guess. + origin needs a compositor that consumes libinput devices. That is what the + `seat-verification` job later built, and it answered this: the origin is + the layout's top-left corner, not the layout coordinate `(0, 0)`. ### The ydotool Path Was Reporting Success While Doing Nothing diff --git a/architecture_explore.md b/architecture_explore.md index edf5f599..5e245314 100644 --- a/architecture_explore.md +++ b/architecture_explore.md @@ -6,7 +6,7 @@ > 擷取每個模組的 docstring 與頂層公開名稱;統計數字取自實際檔案,非估算。 > 指令數與公開 API 數以 `executor.known_commands()` 與 `je_auto_control.__all__` 在工作樹上實測取得。 > -> **掃描時間**:2026-08-20 **版本**:`pyproject.toml` version `0.0.219` **分支**:`feat/cross-platform-verification` +> **掃描時間**:2026-08-20 **版本**:`pyproject.toml` version `0.0.220` **分支**:`feat/windows-arm64-install` --- @@ -19,8 +19,8 @@ iOS(WebDriverAgent)。核心能力是滑鼠/鍵盤控制、影像辨識、 | 指標 | 數值 | | --- | ---: | -| Python 模組總數(含周邊子專案) | 1,029 | -| 程式碼總行數 | 140,053 | +| Python 模組總數(含周邊子專案) | 1,030 | +| 程式碼總行數 | 140,157 | | `je_auto_control/utils/` 子套件數 | 310 | | `AC_*` 動作指令數(`known_commands()` 實測) | 773 | | 套件門面 `__all__` 公開名稱數 | 1,238 | @@ -174,7 +174,7 @@ socket server 有 8 MiB 讀取上限與 30 秒 handler timeout。 | `wrapper/_platform_wayland.py` | 57 | Wayland 後端組裝(libei/ydotool/grim)。 | | `wrapper/auto_control_mouse.py` | 366 | 滑鼠 API:位置讀寫、按下/放開/點擊、捲動、座標前處理、送訊息給指定視窗。 | | `wrapper/auto_control_keyboard.py` | 273 | 鍵盤 API:鍵表查詢、按下/放開/敲擊、`write` 字串、`hotkey` 組合鍵、按鍵狀態偵測。 | -| `wrapper/auto_control_screen.py` | 100 | 螢幕 API:`screen_size`、`screenshot`(可指定區域)、`get_pixel`。 | +| `wrapper/auto_control_screen.py` | 103 | 螢幕 API:`screen_size`、`screenshot`(可指定區域)、`get_pixel`。 | | `wrapper/auto_control_image.py` | 83 | 影像 API:`locate_all_image`、`locate_image_center`、`locate_and_click`。 | | `wrapper/auto_control_record.py` | 107 | 錄製 API:`record`/`stop_record`/`record_to_json`(支援 stop event 與逾時)。 | | `wrapper/auto_control_window.py` | 278 | 視窗管理門面:列舉、尋找、聚焦、等待、關閉、顯示狀態、幾何、所屬行程 PID、依行程列舉/最小化視窗、不搶焦點的投遞式輸入(目前僅 Windows 實作)。 | @@ -227,7 +227,7 @@ socket server 有 8 MiB 讀取上限與 30 秒 handler timeout。 | `uinput/keyboard.py` | 33 | uinput 鍵盤後端,介面與 X11 版一致。 | | `uinput/mouse.py` | 116 | uinput 滑鼠後端。 | -#### Linux Wayland(`linux_wayland/`,17 檔/2,835 行) +#### Linux Wayland(`linux_wayland/`,17 檔/2,836 行) | 模組 | 行數 | 職責 | | --- | ---: | --- | @@ -238,7 +238,7 @@ socket server 有 8 MiB 讀取上限與 30 秒 handler timeout。 | `_select_input.py` | 85 | 決定使用原生 libei 或 CLI shim;`active_backend()` 是 keyboard/mouse 的唯一入口,`emitted()` 讓被拒絕的單次發送退回 CLI。 | | `_layout.py` | 83 | 版面原點的共用查詢。擷取與輸入不是同一個座標空間,差的就是這個原點:libei 的 region offset 是 `uint32`(描述不了負原點),`ydotool mousemove --absolute` 的原點是合成器夾取的那個角落——兩條路都要減掉它,所以放在這裡而不是各自複製。讀數快取一秒——擷取那一側刻意不快取,但 ydotool 每次絕對移動都會問,不快取等於每次移動多開一個 `wlr-randr` 行程。 | | `oeffis.py` | 196 | liboeffis 綁定:跑完 RemoteDesktop portal 交握,交出 EIS fd。 | -| `libei.py` | 610 | libei 綁定與完整握手(seat 綁定能力 → 由事件取得 device → start_emulating → 每次發送後 frame)。另負責絕對指標的座標空間:讀回裝置的 region,把版面座標映射進去,沒有任何 region 涵蓋就拒絕(libei 對這種移動是靜靜丟掉的)。 | +| `libei.py` | 611 | libei 綁定與完整握手(seat 綁定能力 → 由事件取得 device → start_emulating → 每次發送後 frame)。另負責絕對指標的座標空間:讀回裝置的 region,把版面座標映射進去,沒有任何 region 涵蓋就拒絕(libei 對這種移動是靜靜丟掉的)。 | | `mouse.py` | 384 | 滑鼠後端:移動、按鈕與捲動都 libei 優先,退回 ydotool;送往 libei 時垂直捲動軸取負(kernel `REL_WHEEL` 與 `wl_pointer` 正負號相反)。退到 ydotool 的絕對移動會先減掉版面原點(`--absolute` 是相對於版面左上角,不是版面座標的 `(0, 0)`),並依 `pointer_accel_mode()` 處理指標加速度——倍率讀不回來,只有操作者知道,所以由 `JE_AUTOCONTROL_WAYLAND_POINTER_ACCEL` 宣告:未設定=每個行程警告一次後照送、`flat`=已關掉加速度故靜靜送出、`strict`=拒絕這次移動。 | | `keyboard.py` | 173 | 鍵盤後端:libei 優先,退回 ydotool/wtype。 | | `keymap.py` | 155 | 友善鍵名 → evdev key code。 | @@ -266,12 +266,12 @@ socket server 有 8 MiB 讀取上限與 30 秒 handler timeout。 ### 5.4.1 執行引擎與腳本資產 -> 24 個套件、約 12,882 行。 +> 24 個套件、約 12,900 行。 | 模組 | 行數 | 職責 | | --- | ---: | --- | | `utils/action_lint/` | 328 | action 檔 linter 與 JSON Schema 產生器(CI 用 `python -m` 進入點) | -| `utils/action_signing/` | 230 | action 檔 HMAC-SHA256 簽章與 Fernet 加密,`execute_files` 會強制驗簽 | +| `utils/action_signing/` | 248 | action 檔 HMAC-SHA256 簽章與 Fernet 加密,`execute_files` 會強制驗簽 | | `utils/checkpoint/` | 120 | 流程檢查點與續跑,讓長 action list 具持久性 | | `utils/codegen/` | 157 | 由 action list 產生可執行的 pytest / python / robot 測試碼 | | `utils/dag/` | 475 | 跨主機 DAG 編排器(圖模型 + runner) | @@ -365,7 +365,7 @@ socket server 有 8 MiB 讀取上限與 30 秒 handler timeout。 ### 5.4.5 影像辨識與畫面分析 -> 37 個套件、約 5,027 行。 +> 37 個套件、約 5,067 行。 | 模組 | 行數 | 職責 | | --- | ---: | --- | @@ -375,7 +375,7 @@ socket server 有 8 MiB 讀取上限與 30 秒 handler timeout。 | `utils/color_region/` | 79 | 以顏色定位畫面區域(遮罩 + 連通元件) | | `utils/color_stats/` | 95 | 區域顏色統計:平均色與主色 | | `utils/coordinate_space/` | 84 | 模型網格座標與實體像素之間的座標空間對映 | -| `utils/cv2_utils/` | 597 | OpenCV 基礎層:擷取後端選擇(`screen_grabber`,Pillow/mss 或平台後端)、截圖、樣板比對(走 `grab_logical`,涵蓋所有螢幕)、螢幕錄影、影片錄製、連通元件 | +| `utils/cv2_utils/` | 637 | OpenCV 基礎層:擷取後端選擇(`screen_grabber`,Pillow/mss 或平台後端)、截圖、樣板比對(走 `grab_logical`,涵蓋所有螢幕)、螢幕錄影、影片錄製、連通元件、影像堆疊的取用口(`optional`,Windows arm64 沒有 wheel 時語意報錯) | | `utils/edge_lines/` | 120 | 以 Hough 轉換偵測線條/格線/分隔線 | | `utils/edge_match/` | 112 | 邊緣形狀(Chamfer/距離轉換)樣板比對 | | `utils/feature_match/` | 129 | ORB 特徵比對:在旋轉/縮放/主題變更下定位樣板 | @@ -508,24 +508,24 @@ socket server 有 8 MiB 讀取上限與 30 秒 handler timeout。 ### 5.4.10 遠端桌面與 USB -> 6 個套件、約 17,720 行。 +> 6 個套件、約 17,726 行。 | 模組 | 行數 | 職責 | | --- | ---: | --- | | `utils/admin/` | 327 | 多主機管理主控台:平行輪詢 N 個 AutoControl REST 端點 | | `utils/config_sync/` | 245 | 透過訊令伺服器做跨機器設定同步 | | `utils/device_matrix/` | 138 | 行動裝置矩陣:同一 action list 於多台裝置平行執行 | -| `utils/remote_desktop/` | 11,840 | **遠端桌面子系統**(56 檔/11.7K LOC):TCP/WebSocket/WebRTC 三條傳輸路徑、主機與檢視端、訊令伺服器、TURN/中繼、多檢視者、錄影、信任清單、TOTP、稽核鏈 | +| `utils/remote_desktop/` | 11,846 | **遠端桌面子系統**(56 檔/11.7K LOC):TCP/WebSocket/WebRTC 三條傳輸路徑、主機與檢視端、訊令伺服器、TURN/中繼、多檢視者、錄影、信任清單、TOTP、稽核鏈 | | `utils/usb/` | 4,250 | 跨平台 USB 列舉/熱插拔/裝置直通(WinUSB、IOKit、libusb 後端 + ACL + WebRTC DataChannel 通道) | | `utils/usbip/` | 920 | USB/IP 線路協定主機端(協定封包、TCP 伺服器、libusb URB 後端) | ### 5.4.11 伺服器、網路協定與外部整合 -> 24 個套件、約 5,882 行。 +> 24 個套件、約 5,900 行。 | 模組 | 行數 | 職責 | | --- | ---: | --- | -| `utils/acme_v2/` | 586 | 完整 ACME v2 用戶端(RFC 8555),不依賴 certbot | +| `utils/acme_v2/` | 598 | 完整 ACME v2 用戶端(RFC 8555),不依賴 certbot | | `utils/chatops/` | 628 | Chat-ops bot:接收 Slack/Discord/webhook 的 slash 指令並路由到動作 | | `utils/cookie_jar/` | 103 | RFC 6265 cookie jar | | `utils/email_send/` | 116 | SMTP 寄信(email 觸發器的發送端搭檔) | @@ -546,7 +546,7 @@ socket server 有 8 MiB 讀取上限與 30 秒 handler timeout。 | `utils/rest_api/` | 1,739 | 純標準庫 REST 前端:路由、Bearer 驗證、限流、Prometheus 指標、OpenAPI 3.1 產生 | | `utils/socket_server/` | 131 | 執行 action JSON 的執行緒式 TCP 指令伺服器(預設綁 127.0.0.1) | | `utils/sse_client/` | 112 | Server-Sent Events 用戶端解析 | -| `utils/tls_acme/` | 441 | TLS 自動化:HTTP-01 挑戰伺服器、金鑰/CSR、自動續期 | +| `utils/tls_acme/` | 447 | TLS 自動化:HTTP-01 挑戰伺服器、金鑰/CSR、自動續期 | | `utils/url_canon/` | 115 | RFC 3986 URL 正規化與查詢字串工具 | | `utils/webrunner_bridge/` | 161 | 把 action JSON 橋接到 WebRunner(`je_web_runner`) | @@ -624,7 +624,7 @@ socket server 有 8 MiB 讀取上限與 30 秒 handler timeout。 ### 5.4.14 安全、機密與合規 -> 13 個套件、約 2,261 行。 +> 13 個套件、約 2,279 行。 | 模組 | 行數 | 職責 | | --- | ---: | --- | @@ -637,7 +637,7 @@ socket server 有 8 MiB 讀取上限與 30 秒 handler timeout。 | `utils/redaction/` | 457 | 截圖遮蔽層:規則偵測 + 政策 + 協調器(上傳 VLM 前先遮) | | `utils/sbom/` | 108 | SBOM(CycloneDX)產生 | | `utils/secret_ref/` | 126 | URI scheme 形式的值參照解析 | -| `utils/secrets/` | 251 | 加密機密儲存庫,供 `${secrets.NAME}` 解析 | +| `utils/secrets/` | 269 | 加密機密儲存庫,供 `${secrets.NAME}` 解析 | | `utils/secrets_scan/` | 98 | 掃描 action JSON/資料中應入庫卻硬編碼的機密 | | `utils/vex/` | 130 | OpenVEX 陳述撰寫與漏洞分類處置 | | `utils/vuln_scan/` | 188 | 以 OSV 比對 SBOM 元件的漏洞(純標準庫) | @@ -724,7 +724,7 @@ socket server 有 8 MiB 讀取上限與 30 秒 handler timeout。 | `rate_limit.py` | 48 | 工具呼叫的 token bucket 限流。 | | `__main__.py` | 87 | `je_auto_control_mcp` console script 進入點。 | -#### `utils/remote_desktop/`(11,840 行/56 檔) +#### `utils/remote_desktop/`(11,846 行/56 檔) 三條傳輸路徑並存:**TCP**(JPEG 影格)、**WebSocket**(同協定換傳輸)、**WebRTC**(aiortc 視訊 + DataChannel)。 @@ -748,7 +748,7 @@ socket server 有 8 MiB 讀取上限與 30 秒 handler timeout。 | `fingerprint.py` | 250 | TOFU 主機指紋驗證。 | | `turn_config.py` | 234 | coturn 設定產生器。 | | `presence.py` | 221 | 多檢視者的執行緒安全在場註冊表。 | -| `jpeg_recorder_encrypted.py` | 217 | AES-GCM 加密版 session 錄影。 | +| `jpeg_recorder_encrypted.py` | 223 | AES-GCM 加密版 session 錄影。 | | `address_book.py` | 209 | 檢視端的主機通訊錄。 | | `audio.py` / `webrtc_audio.py` / `webrtc_mic.py` | 206 / 190 / 152 | 音訊擷取播放、音訊軌、麥克風上行。 | | `webrtc_files.py` | 205 | 專屬 DataChannel 的分塊檔案傳輸。 | @@ -829,7 +829,7 @@ socket server 有 8 MiB 讀取上限與 30 秒 handler timeout。 | `semantic_recording/` | `enrich.py`(加錨點)、`replay.py`(換機重播)、`self_healing.py`(自癒重播) | | `tls_acme/` | `challenge.py`、`keys.py`、`renewal.py` | | `pytest_plugin/` | `plugin.py`(pytest11 進入點)、`keywords.py`、`bdd_steps.py`(Gherkin) | -| `cv2_utils/` | `screen_grabber.py`、`screenshot.py`、`template_detection.py`、`screen_record.py`、`video_recording.py`、`blobs.py` | +| `cv2_utils/` | `screen_grabber.py`、`screenshot.py`、`template_detection.py`、`screen_record.py`、`video_recording.py`、`blobs.py`、`optional.py` | | `action_lint/` | `linter.py`、`schema.py`、`__main__.py`(CI 使用) | | `time_travel/` | `controller.py`、`player.py` | | `dag/` | `graph.py`、`runner.py` | @@ -1021,17 +1021,17 @@ socket 預設綁 `127.0.0.1`;資源一律用 `with`。 | --- | ---: | ---: | | `gui/` | 89 | 26,542 | | `utils/mcp_server/` | 21 | 17,323 | -| `utils/remote_desktop/` | 56 | 11,840 | +| `utils/remote_desktop/` | 56 | 11,846 | | `utils/executor/` | 6 | 9,075 | | `utils/usb/` | 17 | 4,250 | | `je_auto_control/`(頂層 3 檔) | 3 | 2,363 | | `utils/accessibility/` | 13 | 2,818 | -| `wrapper/` | 3,065 | 3,013 新增 `window_backends/`:視窗管理的平台縫(`base` / `windows_backend` / `x11_backend` / `macos_backend` / `null_backend`)。放在 `wrapper/` 而不是 `utils/`,因為它必須 import `windows/`、`linux_with_x11/`、`osx/`,而 `utils/` 在分層上在那三者之上。 | +| `wrapper/` | 3,068 | 3,013 新增 `window_backends/`:視窗管理的平台縫(`base` / `windows_backend` / `x11_backend` / `macos_backend` / `null_backend`)。放在 `wrapper/` 而不是 `utils/`,因為它必須 import `windows/`、`linux_with_x11/`、`osx/`,而 `utils/` 在分層上在那三者之上。 | | `windows/` | 23 | 1,894 | | `utils/rest_api/` | 8 | 1,739 | | `utils/agent/` | 8 | 1,250 | | `linux_with_x11/` | 19 | 1,215 | -| `linux_wayland/` | 17 | 2,835 | +| `linux_wayland/` | 17 | 2,836 | | `utils/triggers/` | 4 | 1,146 | | `utils/ocr/` | 9 | 1,112 | | `utils/usbip/` | 5 | 920 | @@ -1039,6 +1039,6 @@ socket 預設綁 `127.0.0.1`;資源一律用 `with`。 | `osx/` | 17 | 907 | | `autocontrol-lsp/` | 8 | 744 | | `utils/hotkey/` | 7 | 727 | -| 其餘模組(約 286 個 `utils/` 子套件 + `android/`/`ios/`/周邊小工具) | 690 | 50,425 | -| **總計** | **1,023** | **139,988** | +| 其餘模組(約 286 個 `utils/` 子套件 + `android/`/`ios/`/周邊小工具) | 691 | 50,522 | +| **總計** | **1,024** | **140,092** | diff --git a/docker/Dockerfile.ydotool b/docker/Dockerfile.ydotool index 757dc916..f94b81f5 100644 --- a/docker/Dockerfile.ydotool +++ b/docker/Dockerfile.ydotool @@ -24,8 +24,8 @@ # What this image still cannot answer, and why: # * whether a compositor clamps the INT32_MIN reset that --absolute relies # on. The kernel side of that is checked here; the clamp is the -# compositor's behaviour, and it remains open in Progress.md rather than -# being quietly claimed. +# compositor's behaviour, and docker/Dockerfile.seat answers it against a +# real wlroots session that consumes these very devices. # The portal consent dialog and the EIS fd handover used to be listed here # too. They are xdg-desktop-portal's, not ydotool's, and they are now covered # by docker/Dockerfile.portal against the real liboeffis. diff --git a/je_auto_control/linux_wayland/libei.py b/je_auto_control/linux_wayland/libei.py index b37491ce..ddf00cc0 100644 --- a/je_auto_control/linux_wayland/libei.py +++ b/je_auto_control/linux_wayland/libei.py @@ -25,10 +25,11 @@ installed but unusable pays the probe once, not once per keystroke. The enum values are libei's, from ``libei.h``. They are the one part of this -module that a wrong guess would silently change — see ``Progress.md``. A -mismatch degrades safely rather than misfiring: capabilities that do not -match mean no device ever reports them, the handshake times out, and the CLI -takes over. +module that a wrong guess would silently change, so the ``eis-verification`` +job reads them back off a real ``libeis`` server rather than trusting the +header. A mismatch degrades safely rather than misfiring: capabilities that +do not match mean no device ever reports them, the handshake times out, and +the CLI takes over. """ from __future__ import annotations diff --git a/je_auto_control/utils/acme_v2/client.py b/je_auto_control/utils/acme_v2/client.py index 3809fcdf..f2ced862 100644 --- a/je_auto_control/utils/acme_v2/client.py +++ b/je_auto_control/utils/acme_v2/client.py @@ -8,7 +8,13 @@ from dataclasses import dataclass, field from typing import Any, Callable, Dict, List, Mapping, Optional, Sequence -from cryptography.hazmat.primitives.asymmetric import rsa +try: + from cryptography.hazmat.primitives.asymmetric import rsa +except ImportError as exc: # pragma: no cover - platform-dependent wheel + raise ImportError( + "The ACME client requires cryptography, which publishes no " + "Windows arm64 wheel: pip install cryptography" + ) from exc from je_auto_control.utils.acme_v2.jws import ( JwsError, csr_to_b64url, key_authorization, sign_compact, diff --git a/je_auto_control/utils/acme_v2/jws.py b/je_auto_control/utils/acme_v2/jws.py index 514d7ffd..b1e81d46 100644 --- a/je_auto_control/utils/acme_v2/jws.py +++ b/je_auto_control/utils/acme_v2/jws.py @@ -12,8 +12,14 @@ import json from typing import Any, Dict, Mapping, Optional -from cryptography.hazmat.primitives import hashes, serialization -from cryptography.hazmat.primitives.asymmetric import padding, rsa +try: + from cryptography.hazmat.primitives import hashes, serialization + from cryptography.hazmat.primitives.asymmetric import padding, rsa +except ImportError as exc: # pragma: no cover - platform-dependent wheel + raise ImportError( + "ACME JWS signing requires cryptography, which publishes no " + "Windows arm64 wheel: pip install cryptography" + ) from exc class JwsError(ValueError): diff --git a/je_auto_control/utils/action_signing/cipher.py b/je_auto_control/utils/action_signing/cipher.py index c7593bbb..29b3f792 100644 --- a/je_auto_control/utils/action_signing/cipher.py +++ b/je_auto_control/utils/action_signing/cipher.py @@ -23,13 +23,31 @@ KeyType = Optional[Union[bytes, str]] +def _fernet_types() -> tuple: + """Return ``(Fernet, InvalidToken)``, or explain why encryption is off. + + ``cryptography`` publishes no ``win_arm64`` wheel, so on Windows arm64 it + is absent by design rather than by accident -- see ``pyproject.toml``. A + bare ``ModuleNotFoundError`` there reads like a broken install, so name + what is missing and what it costs. + """ + try: + from cryptography.fernet import Fernet, InvalidToken + except ImportError as error: + raise RuntimeError( + "Action-file encryption requires cryptography (pip install cryptography). " + "It has no Windows arm64 wheel, so encryption is unavailable there." + ) from error + return Fernet, InvalidToken + + def _persistent_key() -> bytes: """Read the per-user Fernet key, creating it (0600) on first use.""" - from cryptography.fernet import Fernet + fernet_cls, _ = _fernet_types() if _DEFAULT_KEY_PATH.exists(): return _DEFAULT_KEY_PATH.read_bytes() _DEFAULT_KEY_PATH.parent.mkdir(parents=True, exist_ok=True) - generated = Fernet.generate_key() + generated = fernet_cls.generate_key() _DEFAULT_KEY_PATH.write_bytes(generated) try: os.chmod(_DEFAULT_KEY_PATH, 0o600) @@ -48,9 +66,9 @@ def _fernet_key(key: KeyType) -> bytes: def encrypt_action_file(path: Union[str, Path], key: KeyType = None) -> str: """Encrypt the file at ``path`` to ``.enc``; return the enc path.""" - from cryptography.fernet import Fernet + fernet_cls, _ = _fernet_types() target = Path(path) - token = Fernet(_fernet_key(key)).encrypt(target.read_bytes()) + token = fernet_cls(_fernet_key(key)).encrypt(target.read_bytes()) enc_path = target.with_name(target.name + _ENC_SUFFIX) enc_path.write_bytes(token) autocontrol_logger.info("encrypted action file %s", target) @@ -65,11 +83,11 @@ def decrypt_action_file(enc_path: Union[str, Path], key: KeyType = None, dropped. Raises :class:`AutoControlException` on a wrong key or a tampered file. """ - from cryptography.fernet import Fernet, InvalidToken + fernet_cls, invalid_token = _fernet_types() enc = Path(enc_path) try: - plaintext = Fernet(_fernet_key(key)).decrypt(enc.read_bytes()) - except InvalidToken as error: + plaintext = fernet_cls(_fernet_key(key)).decrypt(enc.read_bytes()) + except invalid_token as error: raise AutoControlException( f"cannot decrypt {enc_path!r}: wrong key or tampered file", ) from error diff --git a/je_auto_control/utils/cv2_utils/optional.py b/je_auto_control/utils/cv2_utils/optional.py new file mode 100644 index 00000000..fb4dfc41 --- /dev/null +++ b/je_auto_control/utils/cv2_utils/optional.py @@ -0,0 +1,39 @@ +"""Name the missing image-stack wheel instead of raising ``ModuleNotFoundError``. + +``opencv-python`` and ``je_open_cv`` are absent on Windows arm64 *by design*: +neither publishes a ``win_arm64`` wheel, so ``pyproject.toml`` marks both off +that one platform rather than letting ``pip install`` fail for everyone on it. +Everything that does not touch pixels still works there. + +These two accessors sit on the doors every image path goes through, so the +failure says which platform lacks the wheel instead of reading like a broken +install. The deeper modules keep their plain lazy ``import cv2`` — wrapping all +seventy-six of them would buy nothing the caller can act on. +""" +from typing import Any + +_CV2_HINT = ( + "Image matching requires opencv-python, which publishes no Windows arm64 " + "wheel: pip install opencv-python" +) + + +def require_cv2() -> Any: + """Return the ``cv2`` module, or explain why the image stack is absent.""" + try: + import cv2 + except ImportError as error: + raise RuntimeError(_CV2_HINT) from error + return cv2 + + +def require_je_open_cv() -> Any: + """Return ``je_open_cv.template_detection``, or explain why it is absent.""" + try: + from je_open_cv import template_detection + except ImportError as error: + raise RuntimeError( + "Template detection requires je_open_cv and opencv-python, which " + "publish no Windows arm64 wheel: pip install je_open_cv" + ) from error + return template_detection diff --git a/je_auto_control/utils/cv2_utils/template_detection.py b/je_auto_control/utils/cv2_utils/template_detection.py index 98f27bd5..9ac559de 100644 --- a/je_auto_control/utils/cv2_utils/template_detection.py +++ b/je_auto_control/utils/cv2_utils/template_detection.py @@ -8,6 +8,7 @@ """ from typing import Any, List, Optional, Sequence, Tuple +from je_auto_control.utils.cv2_utils.optional import require_je_open_cv from je_auto_control.utils.monitor_layout.logical_frame import grab_logical @@ -52,7 +53,7 @@ def find_image(image: Any, detect_threshold: float = 1.0, :param screen_region: Limit the search to (x, y, width, height) 限定搜尋範圍 :return: [found, [x1, y1, x2, y2]] 座標為螢幕座標 """ - from je_open_cv import template_detection + template_detection = require_je_open_cv() grab_image, origin_x, origin_y = grab_logical( screen_region, all_screens=all_screens) result = template_detection.find_object( @@ -79,7 +80,7 @@ def find_image_multi(image: Any, detect_threshold: float = 1.0, :param screen_region: Limit the search to (x, y, width, height) 限定搜尋範圍 :return: [found, [[x1, y1, x2, y2], ...]] 座標為螢幕座標 """ - from je_open_cv import template_detection + template_detection = require_je_open_cv() grab_image, origin_x, origin_y = grab_logical( screen_region, all_screens=all_screens) result = template_detection.find_multi_object( diff --git a/je_auto_control/utils/input_macro/input_macro.py b/je_auto_control/utils/input_macro/input_macro.py index b5f0b82d..e21f20ac 100644 --- a/je_auto_control/utils/input_macro/input_macro.py +++ b/je_auto_control/utils/input_macro/input_macro.py @@ -34,10 +34,10 @@ def _sink_click(event: Dict[str, Any]) -> None: def _sink_scroll(event: Dict[str, Any]) -> None: from je_auto_control.wrapper.auto_control_mouse import mouse_scroll # ``value`` is the DSL's name for it, ``delta`` the recorder's. The sign - # is kept: it is what decides the direction on Windows and macOS. Linux - # takes its direction from `scroll_direction` instead, which is an open - # cross-platform decision recorded in Progress.md, not something to - # settle here. + # is kept, and that is now enough: a negative value reverses the + # direction on every backend, X11 and Wayland included. They used to + # discard it and always scroll ``scroll_direction``, so a macro + # recorded on Windows replayed backwards there, silently. mouse_scroll(int(event.get("value", event.get("delta", 1)))) diff --git a/je_auto_control/utils/remote_desktop/jpeg_recorder_encrypted.py b/je_auto_control/utils/remote_desktop/jpeg_recorder_encrypted.py index 9da6b248..6c066083 100644 --- a/je_auto_control/utils/remote_desktop/jpeg_recorder_encrypted.py +++ b/je_auto_control/utils/remote_desktop/jpeg_recorder_encrypted.py @@ -22,7 +22,13 @@ from pathlib import Path from typing import Dict, List, Optional -from cryptography.hazmat.primitives.ciphers.aead import AESGCM +try: + from cryptography.hazmat.primitives.ciphers.aead import AESGCM +except ImportError as exc: # pragma: no cover - platform-dependent wheel + raise ImportError( + "Encrypted recording requires cryptography, which publishes no " + "Windows arm64 wheel: pip install cryptography" + ) from exc _MANIFEST_FILENAME = "manifest.json" _KEY_BYTES = 32 # AES-256 diff --git a/je_auto_control/utils/secrets/secret_store.py b/je_auto_control/utils/secrets/secret_store.py index dfb3545f..1dbe28b0 100644 --- a/je_auto_control/utils/secrets/secret_store.py +++ b/je_auto_control/utils/secrets/secret_store.py @@ -35,6 +35,24 @@ _SALT_BYTES = 16 +def _fernet_types() -> tuple: + """Return ``(Fernet, InvalidToken)``, or explain why the vault cannot open. + + ``cryptography`` publishes no ``win_arm64`` wheel, so on Windows arm64 it + is absent by design rather than by accident -- see ``pyproject.toml``. A + bare ``ModuleNotFoundError`` there reads like a broken install, so name + what is missing and what it costs. + """ + try: + from cryptography.fernet import Fernet, InvalidToken + except ImportError as error: + raise RuntimeError( + "The secret vault requires cryptography (pip install cryptography). " + "It has no Windows arm64 wheel, so the vault is unavailable there." + ) from error + return Fernet, InvalidToken + + class SecretStoreError(RuntimeError): """Raised when the vault file is corrupt or a passphrase is wrong.""" @@ -115,10 +133,10 @@ def initialize(self, passphrase: str) -> None: with self._lock: if self._path.exists(): raise SecretStoreError("vault already exists") - from cryptography.fernet import Fernet + fernet_cls, _ = _fernet_types() salt = os.urandom(_SALT_BYTES) key = _derive_key(passphrase, salt, _KEY_ITERATIONS) - fernet = Fernet(key) + fernet = fernet_cls(key) verifier = fernet.encrypt(_VERIFIER_PLAINTEXT).decode("ascii") payload = { "version": 1, @@ -137,16 +155,16 @@ def unlock(self, passphrase: str) -> bool: data = _load_vault(self._path) if data is None: raise SecretStoreError("vault does not exist") - from cryptography.fernet import Fernet, InvalidToken + fernet_cls, invalid_token = _fernet_types() salt = base64.b64decode(data["salt"]) iterations = int(data.get("iterations", _KEY_ITERATIONS)) key = _derive_key(passphrase, salt, iterations) - fernet = Fernet(key) + fernet = fernet_cls(key) try: if fernet.decrypt(data["verifier"].encode("ascii")) \ != _VERIFIER_PLAINTEXT: return False - except InvalidToken: + except invalid_token: return False self._fernet = fernet self._vault = data @@ -177,10 +195,10 @@ def get(self, name: str) -> Optional[str]: token = self._vault["items"].get(name) # type: ignore[index] if token is None: return None - from cryptography.fernet import InvalidToken + _, invalid_token = _fernet_types() try: return self._fernet.decrypt(token.encode("ascii")).decode("utf-8") - except InvalidToken as error: + except invalid_token as error: raise SecretStoreError( f"secret {name!r} failed integrity check" ) from error diff --git a/je_auto_control/utils/tls_acme/keys.py b/je_auto_control/utils/tls_acme/keys.py index 38216ad3..39d2f4c8 100644 --- a/je_auto_control/utils/tls_acme/keys.py +++ b/je_auto_control/utils/tls_acme/keys.py @@ -7,10 +7,16 @@ from pathlib import Path from typing import Optional, Sequence -from cryptography import x509 -from cryptography.hazmat.primitives import hashes, serialization -from cryptography.hazmat.primitives.asymmetric import rsa -from cryptography.x509.oid import NameOID as _NameOID +try: + from cryptography import x509 + from cryptography.hazmat.primitives import hashes, serialization + from cryptography.hazmat.primitives.asymmetric import rsa + from cryptography.x509.oid import NameOID as _NameOID +except ImportError as exc: # pragma: no cover - platform-dependent wheel + raise ImportError( + "TLS key and CSR generation requires cryptography, which publishes " + "no Windows arm64 wheel: pip install cryptography" + ) from exc _DEFAULT_KEY_BITS = 2048 diff --git a/je_auto_control/wrapper/auto_control_screen.py b/je_auto_control/wrapper/auto_control_screen.py index 46711836..2a5d25f6 100644 --- a/je_auto_control/wrapper/auto_control_screen.py +++ b/je_auto_control/wrapper/auto_control_screen.py @@ -1,6 +1,7 @@ import sys from typing import Tuple, List +from je_auto_control.utils.cv2_utils.optional import require_cv2 from je_auto_control.utils.cv2_utils.screenshot import pil_screenshot from je_auto_control.utils.exception.exception_tags import screen_get_size_error_message from je_auto_control.utils.exception.exception_tags import screen_screenshot_error_message @@ -42,7 +43,9 @@ def screenshot(file_path: str = None, screen_region: list = None) -> List[int]: # Spelled out rather than locals(): the imports below land in locals() too, # so the recorded parameters would carry two module objects. param = {"file_path": file_path, "screen_region": screen_region} - import cv2 # noqa: E402 # reason: kept off the facade's import path + # require_cv2 first: without the OpenCV wheel there is no NumPy either, + # so its ModuleNotFoundError would be the one the caller ended up seeing. + cv2 = require_cv2() import numpy as np # noqa: E402 # reason: kept off the facade's import path try: record_action_to_list("AC_screenshot", param) diff --git a/pyproject.toml b/pyproject.toml index b6d6ce48..9bd5a94a 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -14,10 +14,34 @@ description = "GUI Automation Framework" requires-python = ">=3.10" license-files = ["LICENSE"] dependencies = [ - "je_open_cv==0.0.22", + # Three requirements below carry the same marker, and it excludes exactly + # one square: Windows on arm64. Neither package publishes a win_arm64 + # wheel, so pip fell back to building from source and CMake could not + # configure for ARM64 -- twelve minutes, then failure, which is why the + # runner had to leave the matrix. Nothing in the package needs them at + # import time, so the marker is all that stood between this project and + # that machine. PEP 508 markers have no boolean "not", hence "A or B". + # + # opencv-python no win_arm64 wheel in any version. + # cryptography wheels stop at 46.0.3; our floor >=48.0.1 is a security + # floor (GHSA-537c-gmf6-5ccf) and cannot be lowered. + # je_open_cv pure Python, but it depends on opencv-python, so leaving + # it unmarked drags OpenCV back in through the side door. + # + # Pillow is deliberately NOT marked: it has always shipped win_arm64 + # wheels and was never a blocker. Re-check upstream in ten seconds, no + # runner required: + # + # pip install --dry-run --only-binary=:all: --platform win_arm64 \ + # --python-version 3.12 --target /tmp/probe \ + # 'opencv-python>=4.8,<6' 'cryptography>=48.0.1' + # + # When both resolve, drop the marker from all three lines. Progress.md + # records what Windows arm64 gives up in the meantime. + "je_open_cv==0.0.22; sys_platform != 'win32' or platform_machine != 'ARM64'", # je_open_cv leaves opencv-python unpinned; bound the major so a new # OpenCV release cannot silently change cv2 return shapes under us. - "opencv-python>=4.8,<6", + "opencv-python>=4.8,<6; sys_platform != 'win32' or platform_machine != 'ARM64'", "pillow==12.3.0", "pyobjc-core==12.2.1;platform_system=='Darwin'", "pyobjc==12.2.1;platform_system=='Darwin'", @@ -27,7 +51,7 @@ dependencies = [ "python-Xlib==0.33;platform_system=='Linux' or platform_system=='FreeBSD' or platform_system=='OpenBSD' or platform_system=='NetBSD'", "mss==10.2.0", "defusedxml==0.7.1", - "cryptography>=48.0.1" + "cryptography>=48.0.1; sys_platform != 'win32' or platform_machine != 'ARM64'" ] classifiers = [ "Programming Language :: Python :: 3.10", diff --git a/test/unit_test/headless/test_arm64_dependency_markers.py b/test/unit_test/headless/test_arm64_dependency_markers.py new file mode 100644 index 00000000..641992a6 --- /dev/null +++ b/test/unit_test/headless/test_arm64_dependency_markers.py @@ -0,0 +1,142 @@ +"""The Windows arm64 dependency markers must stay exactly as measured. No Qt. + +`opencv-python` and `cryptography` publish no `win_arm64` wheel, so +`pip install` used to fail on Windows arm64 before a single line of this +package ran — pip built OpenCV from source and CMake could not configure for +ARM64. Nothing in the package imports either one at import time, so the whole +blocker lived in `pyproject.toml`'s dependency list, and a PEP 508 marker is +the entire fix. + +Three things can silently undo it, and each has a test here: + +1. Dropping the marker from one of the three requirements. `je_open_cv` is the + easy one to miss — it is pure Python, but it depends on `opencv-python`, so + an unmarked `je_open_cv` drags OpenCV back in through the side door. +2. Writing a marker that does not mean what it looks like. PEP 508 has no + boolean `not`, so the condition is spelled `A or B`; getting that wrong + silently excludes far more than one platform. +3. Marking a package that was never a blocker. Pillow was named as one for a + while and is not: it has always shipped `win_arm64` wheels. Marking it off + would cost Windows arm64 screenshots for no reason at all. + +pip evaluates markers against the *running* interpreter — `--platform` only +changes wheel-compatibility tags — so a local dry-run cannot prove any of +this. Evaluating the marker directly can, and does not need the runner. +""" +import pathlib +import re +from typing import Dict, List + +import pytest + +ROOT = pathlib.Path(__file__).resolve().parents[3] + +# The one marker, spelled exactly as pyproject.toml spells it. +ARM64_MARKER = "sys_platform != 'win32' or platform_machine != 'ARM64'" + +# Absent from Windows arm64 because upstream publishes no wheel there. +MARKED = ("je_open_cv", "opencv-python", "cryptography") + +# Present everywhere, including Windows arm64. Pillow is on this list on +# purpose: it was once named a blocker and never was one. +UNMARKED = ("pillow", "mss", "defusedxml") + +ENVIRONMENTS: Dict[str, Dict[str, str]] = { + "windows arm64": { + "sys_platform": "win32", "platform_machine": "ARM64", + "platform_system": "Windows", + }, + "windows x86-64": { + "sys_platform": "win32", "platform_machine": "AMD64", + "platform_system": "Windows", + }, + "linux arm64": { + "sys_platform": "linux", "platform_machine": "aarch64", + "platform_system": "Linux", + }, + "macos arm64": { + "sys_platform": "darwin", "platform_machine": "arm64", + "platform_system": "Darwin", + }, + "freebsd x86-64": { + "sys_platform": "freebsd14", "platform_machine": "amd64", + "platform_system": "FreeBSD", + }, +} + + +def _dependencies() -> List[str]: + """Return the project's required dependencies, verbatim. + + Read with a small regex rather than `tomllib`, which is 3.11+ while this + project supports 3.10 back to the floor. A guard that cannot run on the + oldest supported interpreter is not a guard, and using `tomllib` here + took the whole 3.10 collection down with it. + """ + text = (ROOT / "pyproject.toml").read_text(encoding="utf-8") + block = re.search(r"^dependencies = \[(.*?)^\]", text, re.S | re.M) + assert block is not None, "pyproject.toml has no top-level dependencies array" + found: List[str] = [] + for line in block.group(1).splitlines(): + quoted = re.match(r'"([^"]+)"', line.strip()) + if quoted is not None: + found.append(quoted.group(1)) + assert found, "parsed no dependencies; the array format must have changed" + return found + + +def _requirement(name: str) -> str: + """Return the single requirement string whose distribution is ``name``.""" + prefix = name.lower() + matches = [ + text for text in _dependencies() + if text.lower().split(";")[0].strip() + .replace("_", "-").startswith(prefix.replace("_", "-")) + ] + assert len(matches) == 1, f"{name}: expected one requirement, got {matches}" + return matches[0] + + +@pytest.mark.parametrize("name", MARKED) +def test_the_unavailable_wheels_are_marked_off_windows_arm64(name: str) -> None: + """Each of the three carries the arm64 marker, spelled the one way.""" + requirement = _requirement(name) + assert ";" in requirement, ( + f"{name} lost its environment marker. Without it, pip resolves " + f"{name} on Windows arm64, where no wheel exists, and the install " + f"fails before any of this package runs." + ) + marker = requirement.split(";", 1)[1].strip() + assert marker == ARM64_MARKER, ( + f"{name} carries {marker!r}, not the shared arm64 marker " + f"{ARM64_MARKER!r}. Keep one spelling so all three move together." + ) + + +@pytest.mark.parametrize("name", UNMARKED) +def test_the_available_wheels_are_not_marked_off_anything(name: str) -> None: + """Pillow, mss and defusedxml ship for arm64 and must stay unconditional.""" + requirement = _requirement(name) + assert ARM64_MARKER not in requirement, ( + f"{name} was marked off Windows arm64, but it publishes a win_arm64 " + f"wheel and was never a blocker. Marking it costs that platform a " + f"working feature for nothing." + ) + + +@pytest.mark.parametrize("label,environment", sorted(ENVIRONMENTS.items())) +def test_the_marker_excludes_exactly_one_platform( + label: str, environment: Dict[str, str]) -> None: + """Windows arm64 loses the three; every other platform keeps them.""" + markers = pytest.importorskip( + "packaging.markers", + reason="packaging is not installed; the spelling test still guards this", + ) + wanted = label != "windows arm64" + for name in MARKED: + requirement = _requirement(name) + marker = markers.Marker(requirement.split(";", 1)[1].strip()) + assert marker.evaluate(environment) is wanted, ( + f"on {label}, {name} evaluates to {not wanted} — expected " + f"{wanted}. The marker excludes the wrong set of platforms." + ) diff --git a/test/unit_test/headless/test_missing_wheel_messages.py b/test/unit_test/headless/test_missing_wheel_messages.py new file mode 100644 index 00000000..52e9f721 --- /dev/null +++ b/test/unit_test/headless/test_missing_wheel_messages.py @@ -0,0 +1,144 @@ +"""A missing arm64 wheel must say so, not raise ModuleNotFoundError. No Qt. + +`cryptography`, `opencv-python` and `je_open_cv` publish no `win_arm64` +wheel, so `pyproject.toml` marks them off that one platform and everything +else in the package keeps working there. That trade is only honest if the +features that *do* need them fail legibly: on Windows arm64 the package is +installed and correct, so a bare `ModuleNotFoundError: No module named 'cv2'` +reads like a broken install and sends the user to reinstall something that +cannot exist. + +These tests make the missing import fail on a machine that has the wheels, +by putting `None` in `sys.modules` — the import system raises `ImportError` +for that, which is the branch the accessors catch. +""" +import importlib +import os +import pathlib +import subprocess +import sys +from typing import Callable + +import pytest + +REPO_ROOT = pathlib.Path(__file__).resolve().parents[3] + + +def _blocked(monkeypatch: pytest.MonkeyPatch, *names: str) -> None: + """Make importing each of ``names`` raise ImportError.""" + for name in names: + monkeypatch.setitem(sys.modules, name, None) + + +def test_the_secret_vault_names_the_wheel_it_needs(monkeypatch) -> None: + """SecretManager.initialize explains cryptography rather than failing raw.""" + from je_auto_control.utils.secrets import secret_store + + _blocked(monkeypatch, "cryptography.fernet") + with pytest.raises(RuntimeError) as caught: + secret_store._fernet_types() + + message = str(caught.value) + assert "cryptography" in message + assert "Windows arm64" in message + assert isinstance(caught.value.__cause__, ImportError) + + +def test_action_encryption_names_the_wheel_it_needs(monkeypatch) -> None: + """Action-file encryption explains cryptography rather than failing raw.""" + from je_auto_control.utils.action_signing import cipher + + _blocked(monkeypatch, "cryptography.fernet") + with pytest.raises(RuntimeError) as caught: + cipher._fernet_types() + + message = str(caught.value) + assert "cryptography" in message + assert "Windows arm64" in message + assert isinstance(caught.value.__cause__, ImportError) + + +def test_encrypting_an_action_file_surfaces_the_same_message( + monkeypatch, tmp_path) -> None: + """The public entry point carries the message, not just the accessor.""" + from je_auto_control.utils.action_signing import cipher + + script = tmp_path / "script.json" + script.write_text('[["AC_noop"]]', encoding="utf-8") + + _blocked(monkeypatch, "cryptography.fernet") + with pytest.raises(RuntimeError, match="Windows arm64"): + cipher.encrypt_action_file(script, b"unit-test-key") + + +@pytest.mark.parametrize("accessor_name,blocked,needle", [ + ("require_cv2", "cv2", "opencv-python"), + ("require_je_open_cv", "je_open_cv", "je_open_cv"), +]) +def test_the_image_stack_doors_name_their_wheel( + monkeypatch, accessor_name: str, blocked: str, needle: str) -> None: + """Both image-stack accessors explain the platform, not just the module.""" + from je_auto_control.utils.cv2_utils import optional + + accessor: Callable = getattr(optional, accessor_name) + _blocked(monkeypatch, blocked) + with pytest.raises(RuntimeError) as caught: + accessor() + + message = str(caught.value) + assert needle in message + assert "Windows arm64" in message + assert isinstance(caught.value.__cause__, ImportError) + + +def test_the_accessors_return_the_real_modules_when_the_wheels_are_there() -> None: + """A lazy import placed in the wrong branch must not pass as a win.""" + pytest.importorskip("cv2") + from je_auto_control.utils.cv2_utils.optional import require_cv2 + + assert require_cv2() is importlib.import_module("cv2") + + +# The four modules that import cryptography at module scope are not on the +# facade's import path, so they keep a module-scope import — what they must +# not do is fail with a bare ModuleNotFoundError. Checked in a subprocess +# because the import has to happen with the module genuinely absent. +_PROBE = """ +import sys + + +class Blocker: + def find_module(self, name, path=None): + return self.find_spec(name, path) + + def find_spec(self, name, path=None, target=None): + if name == "cryptography" or name.startswith("cryptography."): + raise ImportError("blocked for probe: " + name) + return None + + +sys.meta_path.insert(0, Blocker()) +for name in [m for m in sys.modules if m.split(".")[0] == "cryptography"]: + del sys.modules[name] + +try: + import je_auto_control.utils.tls_acme.keys # noqa: F401 +except ImportError as error: + print(str(error)) +else: + raise SystemExit("expected ImportError, module imported cleanly") +""" + + +def test_a_module_scope_importer_explains_itself() -> None: + """tls_acme.keys re-raises with the reason, following webrtc_transport.""" + env = dict(os.environ, PYTHONPATH=str(REPO_ROOT)) + # argv is this interpreter plus a module-level literal probe, with no + # interpolation at all. No shell. + finished = subprocess.run( # nosec B603 # nosemgrep # reason: literal argv, no shell + [sys.executable, "-c", _PROBE], + capture_output=True, text=True, timeout=120, env=env, check=False, + ) + assert finished.returncode == 0, finished.stderr + assert "Windows arm64" in finished.stdout, finished.stdout + assert "cryptography" in finished.stdout, finished.stdout