Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .github/workflows/release.yml
Original file line number Diff line number Diff line change
Expand Up @@ -44,6 +44,7 @@ jobs:
if: steps.release.outputs.released == 'true'
run: |
uv build
python scripts/verify_release_artifacts.py
python scripts/check_source_boundary.py --require-dist

# v1.14.0 bundles twine 6.1.0 and packaging 25.0, which reject the
Expand Down
1 change: 1 addition & 0 deletions .github/workflows/test.yml
Original file line number Diff line number Diff line change
Expand Up @@ -34,6 +34,7 @@ jobs:
if: matrix.python-version == '3.12'
run: |
uv build
python scripts/verify_release_artifacts.py
python scripts/check_source_boundary.py --require-dist

test-presidio:
Expand Down
23 changes: 12 additions & 11 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,19 +1,19 @@
# openadapt-privacy

> [!IMPORTANT]
> **Status: Experimental.** The API is published on the 1.x version line, but
> the PHI/PII detector is backed by synthetic regression evidence rather than
> clinical validation. Scrubbing is one control in a reviewed egress process,
> not a guarantee that an artifact is free of protected data.
> **Lifecycle: Support.** `openadapt-privacy` is the current public privacy
> dependency for OpenAdapt recording and artifact pipelines. Support identifies
> its role in the stack. It does not create an additional OpenAdapt product
> target or a separate Production claim.
>
> The OpenAdapt product is the demonstration compiler,
> [`openadapt-flow`](https://github.com/OpenAdaptAI/openadapt-flow), installed
> via the [`OpenAdapt`](https://github.com/OpenAdaptAI/OpenAdapt) launcher
> (`pip install openadapt`): it compiles a demonstrated GUI workflow into a
> deterministic, locally executable program. Healthy runs make no model calls,
> and it halts instead of guessing when verification fails. Lifecycle labels for
> every repository are in the
> [repository lifecycle registry](https://github.com/OpenAdaptAI/.github/blob/main/REPOSITORY_LIFECYCLE.md).
> and it halts instead of guessing when verification fails. The live admission
> result for the seven OpenAdapt product targets is available from
> [`openadapt.ai/status.json`](https://openadapt.ai/status.json).

[![Build Status](https://github.com/OpenAdaptAI/openadapt-privacy/actions/workflows/test.yml/badge.svg?branch=main)](https://github.com/OpenAdaptAI/openadapt-privacy/actions)
[![PyPI version](https://img.shields.io/pypi/v/openadapt-privacy.svg)](https://pypi.org/project/openadapt-privacy/)
Expand All @@ -38,14 +38,15 @@ OpenAdapt is a governed demonstration compiler: record a workflow once, compile
the recording into a deterministic program, and replay that program with zero
model calls on the healthy path. When the live screen does not match what was
demonstrated it halts instead of guessing, using identity gates and independent
effect verification. Every substrate is first-class: web and desktop recording
are validated, RDP and Windows replay are early, and Citrix is exploratory.
effect verification. Product status is derived from signed, expiring, and
revocable release admissions. A current product release can execute only an
exact workflow version that has its own active admission.

| Package | Role |
| --- | --- |
| [`openadapt`](https://github.com/OpenAdaptAI/OpenAdapt) | Launcher and installer (`pip install openadapt`) |
| [`openadapt-flow`](https://github.com/OpenAdaptAI/openadapt-flow) | Records, compiles, verifies, and replays workflows |
| [`openadapt-capture`](https://github.com/OpenAdaptAI/openadapt-capture) | Cross-platform local desktop recording |
| [`openadapt-flow`](https://github.com/OpenAdaptAI/openadapt-flow) | Normalizes demonstrations, then compiles, verifies, and replays workflows |
| [`openadapt-capture`](https://github.com/OpenAdaptAI/openadapt-capture) | Canonical native screen, mouse, keyboard, timing, window, and media capture |
| [`openadapt-types`](https://github.com/OpenAdaptAI/openadapt-types) | Canonical action and UI-state schema |
| [`openadapt-grounding`](https://github.com/OpenAdaptAI/openadapt-grounding) | Local OCR text-anchoring plus optional model grounding |
| **`openadapt-privacy`** | PHI/PII detection and redaction (this package) |
Expand Down
2 changes: 1 addition & 1 deletion pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,6 @@ authors = [
]
keywords = ["privacy", "pii", "phi", "scrubbing", "redaction", "gui", "automation"]
classifiers = [
"Development Status :: 3 - Alpha",
"Intended Audience :: Developers",
"License :: OSI Approved :: MIT License",
"Programming Language :: Python :: 3",
Expand Down Expand Up @@ -39,6 +38,7 @@ dev = [
"pytest>=7.0.0",
"pytest-cov>=4.0.0",
"ruff>=0.1.0",
"tomli>=2.0.0; python_version < '3.11'",
]

[project.urls]
Expand Down
103 changes: 103 additions & 0 deletions scripts/verify_release_artifacts.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,103 @@
"""Verify the exact wheel and source distribution before publication."""

from __future__ import annotations

import argparse
import email
import re
import tarfile
import zipfile
from email.message import Message
from pathlib import Path

try:
import tomllib
except ModuleNotFoundError: # pragma: no cover - exercised in the Python 3.10 CI job
import tomli as tomllib

ROOT = Path(__file__).resolve().parents[1]


class ArtifactError(RuntimeError):
"""A release artifact does not match the reviewed project metadata."""


def _canonical_name(value: str) -> str:
return re.sub(r"[-_.]+", "-", value).lower()


def _project_identity(root: Path) -> tuple[str, str]:
project = tomllib.loads((root / "pyproject.toml").read_text(encoding="utf-8"))[
"project"
]
name = project.get("name")
version = project.get("version")
if not isinstance(name, str) or not name or not isinstance(version, str) or not version:
raise ArtifactError("pyproject.toml must declare a project name and version")
classifiers = project.get("classifiers", [])
if any(str(item).startswith("Development Status ::") for item in classifiers):
raise ArtifactError("project metadata must not publish a static maturity classifier")
return name, version


def _wheel_metadata(path: Path) -> bytes:
with zipfile.ZipFile(path) as archive:
names = [name for name in archive.namelist() if name.endswith(".dist-info/METADATA")]
if len(names) != 1:
raise ArtifactError(f"{path.name} must contain exactly one METADATA file")
return archive.read(names[0])


def _sdist_metadata(path: Path) -> bytes:
with tarfile.open(path, "r:gz") as archive:
members = [
member
for member in archive.getmembers()
if member.isfile() and member.name.count("/") == 1 and member.name.endswith("/PKG-INFO")
]
if len(members) != 1:
raise ArtifactError(f"{path.name} must contain exactly one root PKG-INFO file")
stream = archive.extractfile(members[0])
if stream is None:
raise ArtifactError(f"{path.name} PKG-INFO cannot be read")
return stream.read()


def _verify_metadata(path: Path, payload: bytes, name: str, version: str) -> None:
metadata: Message = email.message_from_bytes(payload)
if _canonical_name(metadata.get("Name", "")) != _canonical_name(name):
raise ArtifactError(f"{path.name} has the wrong package name")
if metadata.get("Version") != version:
raise ArtifactError(f"{path.name} has the wrong package version")
classifiers = metadata.get_all("Classifier", [])
if any(value.startswith("Development Status ::") for value in classifiers):
raise ArtifactError(f"{path.name} publishes a static maturity classifier")


def verify_distributions(root: Path = ROOT) -> tuple[Path, Path]:
"""Verify one wheel and one source archive against ``pyproject.toml``."""
name, version = _project_identity(root)
dist = root / "dist"
wheels = sorted(dist.glob("*.whl"))
sdists = sorted(dist.glob("*.tar.gz"))
if len(wheels) != 1 or len(sdists) != 1:
raise ArtifactError("dist must contain exactly one wheel and one source distribution")
_verify_metadata(wheels[0], _wheel_metadata(wheels[0]), name, version)
_verify_metadata(sdists[0], _sdist_metadata(sdists[0]), name, version)
return wheels[0], sdists[0]


def main() -> int:
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument("--root", type=Path, default=ROOT)
args = parser.parse_args()
try:
wheel, sdist = verify_distributions(args.root.resolve())
except (ArtifactError, OSError, KeyError, tarfile.TarError, zipfile.BadZipFile) as exc:
parser.exit(1, f"release artifact verification failed: {exc}\n")
print(f"verified {wheel.name} and {sdist.name}")
return 0


if __name__ == "__main__":
raise SystemExit(main())
63 changes: 63 additions & 0 deletions tests/test_release_artifacts.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,63 @@
from __future__ import annotations

import importlib.util
import io
import tarfile
import zipfile
from pathlib import Path

import pytest

ROOT = Path(__file__).resolve().parents[1]
SCRIPT = ROOT / "scripts" / "verify_release_artifacts.py"
SPEC = importlib.util.spec_from_file_location("verify_release_artifacts", SCRIPT)
assert SPEC and SPEC.loader
artifacts = importlib.util.module_from_spec(SPEC)
SPEC.loader.exec_module(artifacts)


def _metadata(*, classifier: str | None = None) -> bytes:
lines = [
"Metadata-Version: 2.4",
"Name: example-package",
"Version: 1.2.3",
]
if classifier is not None:
lines.append(f"Classifier: {classifier}")
return ("\n".join(lines) + "\n\n").encode()


def _write_release(root: Path, *, artifact_classifier: str | None = None) -> None:
(root / "pyproject.toml").write_text(
'[project]\nname = "example-package"\nversion = "1.2.3"\nclassifiers = []\n',
encoding="utf-8",
)
dist = root / "dist"
dist.mkdir()
payload = _metadata(classifier=artifact_classifier)
with zipfile.ZipFile(dist / "example_package-1.2.3-py3-none-any.whl", "w") as archive:
archive.writestr("example_package-1.2.3.dist-info/METADATA", payload)
with tarfile.open(dist / "example_package-1.2.3.tar.gz", "w:gz") as archive:
member = tarfile.TarInfo("example_package-1.2.3/PKG-INFO")
member.size = len(payload)
archive.addfile(member, io.BytesIO(payload))


def test_matching_wheel_and_source_distribution_pass(tmp_path: Path) -> None:
_write_release(tmp_path)
wheel, sdist = artifacts.verify_distributions(tmp_path)
assert wheel.suffix == ".whl"
assert sdist.name.endswith(".tar.gz")


def test_static_maturity_classifier_in_archive_fails(tmp_path: Path) -> None:
_write_release(tmp_path, artifact_classifier="Development Status :: 3 - Alpha")
with pytest.raises(artifacts.ArtifactError, match="static maturity classifier"):
artifacts.verify_distributions(tmp_path)


def test_extra_release_archive_fails(tmp_path: Path) -> None:
_write_release(tmp_path)
(tmp_path / "dist" / "unexpected.whl").touch()
with pytest.raises(artifacts.ArtifactError, match="exactly one wheel"):
artifacts.verify_distributions(tmp_path)
27 changes: 27 additions & 0 deletions tests/test_release_config.py
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,8 @@
from __future__ import annotations

import re
import subprocess
import sys
from pathlib import Path

ROOT = Path(__file__).resolve().parents[1]
Expand All @@ -19,3 +21,28 @@ def test_release_uses_protected_branch_credential_everywhere() -> None:
assert "token: ${{ secrets.ADMIN_TOKEN }}" in workflow
assert workflow.count("github_token: ${{ secrets.ADMIN_TOKEN }}") == 2
assert "secrets.GITHUB_TOKEN" not in workflow


def test_release_checks_the_built_archives_before_publication() -> None:
test_workflow = (ROOT / ".github/workflows/test.yml").read_text(encoding="utf-8")
release_workflow = (ROOT / ".github/workflows/release.yml").read_text(
encoding="utf-8"
)
command = "python scripts/verify_release_artifacts.py"
assert command in test_workflow
assert command in release_workflow


def test_project_has_no_static_maturity_classifier() -> None:
pyproject = (ROOT / "pyproject.toml").read_text(encoding="utf-8")
assert "Development Status ::" not in pyproject


def test_current_built_archives_match_reviewed_metadata() -> None:
if not (ROOT / "dist").is_dir():
return
subprocess.run(
[sys.executable, "scripts/verify_release_artifacts.py"],
cwd=ROOT,
check=True,
)
Loading