Skip to content

Replace Python eggs with a universal wheel - #6341

Draft
harvey0100 wants to merge 11 commits into
avocado-framework:masterfrom
harvey0100:feat/podman-wheel-bootstrap
Draft

Replace Python eggs with a universal wheel#6341
harvey0100 wants to merge 11 commits into
avocado-framework:masterfrom
harvey0100:feat/podman-wheel-bootstrap

Conversation

@harvey0100

Copy link
Copy Markdown
Contributor

Fixes #6108
Fixes #6115

nrunner --spawner=podman deployed Avocado with per-CPython eggs plus setuptools 59.2 on PYTHONPATH. That fails on Fedora 39+ / Python 3.12+ (hang / ModuleNotFoundError: avocado). Eggs are a discontinued format; GitHub 113.0 still attached eggs and no wheel.

This switches isolated spawners to one avocado_framework-{ver}-py3-none-any.whl: unpack on the host, bind-mount /opt/avocado-wheel:ro,z, keep python -m avocado.plugins.runners… task-run. Not a venv inside the container. Not avocado run as the entrypoint.

Resolver: explicit URL → pip wheel from the git tree → GitHub release wheel → PyPI. CI/release stop bdist_egg and attach the wheel to GitHub releases. Docs drop the setuptools<82 pin. --spawner-podman-avocado-egg stays as a deprecated alias for one release.

The container image must still provide python3. Fedora 41+ default images do not; selftests use fedora:40. fedora-toolbox still has an interpreter.

Not in this PR: setup.pypyproject.toml / CI setuptools<80 (#5962), remaining distutils (#5159), changing the default Podman image to fedora-toolbox.

Test plan

  • selftests.unit.plugin.wheel_bootstrap
  • Podman passtest.py + /bin/true on fedora:40 (Python 3.12)
  • Same on fedora-toolbox:44 (Python 3.14)
  • Develop-from-source pip wheel when the tree is on a read-only mount
  • GitHub Actions CI on this PR
  • Reviewer: avocado run --spawner=podman --spawner-podman-image=fedora:40 examples/tests/passtest.py

Made with Cursor

@coderabbitai

coderabbitai Bot commented Aug 21, 2026

Copy link
Copy Markdown

Important

Review skipped

Draft detected.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: b888731c-71df-4222-b8db-e274344f8906

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@mr-avocado mr-avocado Bot moved this to Review Requested in Default project Aug 21, 2026
@harvey0100 harvey0100 changed the title podman: bootstrap Avocado from a universal wheel instead of eggs Replace Python eggs with a universal wheel Aug 21, 2026
@harvey0100
harvey0100 force-pushed the feat/podman-wheel-bootstrap branch from 7f569fd to 4eade76 Compare August 21, 2026 14:54
@codecov

codecov Bot commented Aug 21, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 75.00000% with 44 lines in your changes missing coverage. Please review.
✅ Project coverage is 70.70%. Comparing base (4e00d7f) to head (fc67554).

Files with missing lines Patch % Lines
avocado/core/spawners/wheel_bootstrap.py 79.61% 32 Missing ⚠️
avocado/plugins/spawners/podman.py 36.84% 12 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##           master    #6341      +/-   ##
==========================================
+ Coverage   70.60%   70.70%   +0.10%     
==========================================
  Files         207      208       +1     
  Lines       23637    23782     +145     
==========================================
+ Hits        16688    16816     +128     
- Misses       6949     6966      +17     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@harvey0100
harvey0100 force-pushed the feat/podman-wheel-bootstrap branch from 229ada7 to d320180 Compare August 21, 2026 15:16
Python eggs are CPython-minor specific, need pkg_resources, and
already fail inside Fedora 39+ / Python 3.12 containers.  Avocado
is pure Python, so one py3-none-any wheel is enough: unpack it on
the host and put that directory on PYTHONPATH.

When running from a source tree the helper builds the wheel with
pip; otherwise it fetches the GitHub release asset.  Eggs remain
as a deprecated zipimport fallback.

Reference: avocado-framework#6108
Reference: avocado-framework#6115
Signed-off-by: Harvey Lynden <harveylynden@gmail.com>
Stop copying per-CPython eggs plus setuptools 59.2 into every
container.  Unpack one wheel on the host and bind-mount it at
/opt/avocado-wheel so nrunner can still start with
python -m avocado.plugins.runners... task-run.

--spawner-podman-avocado-wheel selects the package;
--spawner-podman-avocado-egg is kept as a deprecated alias.

Reference: avocado-framework#6108
Reference: avocado-framework#6115
Signed-off-by: Harvey Lynden <harveylynden@gmail.com>
Unit tests build a py3-none-any wheel, unpack it, and prove
import avocado plus python -m avocado.plugins.runners.exec_test
work from PYTHONPATH.  Functional Podman jobs now target
fedora:latest so they keep exercising current CPython.

Reference: avocado-framework#6115
Signed-off-by: Harvey Lynden <harveylynden@gmail.com>
Record the egg-to-wheel switch for the next release notes, the
nrunner reference, and the man page.  Point users at
--spawner-podman-avocado-wheel.

Reference: avocado-framework#6108
Signed-off-by: Harvey Lynden <harveylynden@gmail.com>
bdist_egg plus fedora:40 cannot prove the bootstrap still works
on current images.  Build a universal wheel and pass it with
--spawner-podman-avocado-wheel against fedora:latest.

Reference: avocado-framework#6115
Signed-off-by: Harvey Lynden <harveylynden@gmail.com>
avocado-fetch-eggs.py pulled one egg per CPython minor plus
setuptools 59.2.  Add avocado-fetch-wheels.py for the single
py3-none-any wheel, and warn on the old script.

Reference: avocado-framework#6108
Signed-off-by: Harvey Lynden <harveylynden@gmail.com>
Gate 3 on a Fedora 44 VM showed fedora:41+ default container
images no longer include python3, so nrunner cannot start.
fedora:40 still has Python 3.12, the first interpreter where
eggs already failed.

Reference: avocado-framework#6115
Signed-off-by: Harvey Lynden <harveylynden@gmail.com>
Avocado 113.0 GitHub releases only attached eggs.  pip-installed
Avocado therefore could not bootstrap Podman until we also try
PyPI.  Snapshot the source tree before pip wheel so read-only
mounts and parallel selftests still build.  Copy README.rst as a
regular file because it is a symlink into docs/.

Reference: avocado-framework#6108
Signed-off-by: Harvey Lynden <harveylynden@gmail.com>
Drop the per-interpreter egg-build matrix and Makefile.gh
bdist_egg.  The release pipeline uploads py3-none-any wheels to
the GitHub release instead of eggs.

Reference: avocado-framework#6108
Signed-off-by: Harvey Lynden <harveylynden@gmail.com>
Avocado core no longer imports pkg_resources.  Document that
releases publish wheels, not eggs, and that docs can use current
setuptools.

Reference: avocado-framework#6108
Signed-off-by: Harvey Lynden <harveylynden@gmail.com>
Stop downloading per-CPython eggs and setuptools 59.2.  The old
script warns and uses the same resolver as avocado-fetch-wheels.

Reference: avocado-framework#6108
Signed-off-by: Harvey Lynden <harveylynden@gmail.com>
@harvey0100
harvey0100 force-pushed the feat/podman-wheel-bootstrap branch from d320180 to fc67554 Compare August 21, 2026 15:43
@pevogam

pevogam commented Aug 24, 2026

Copy link
Copy Markdown
Collaborator

Hi @harvey0100, I can see the various rpm-builds failing and the GH action with the podman spawner even errors entirely. I assume I should take a look at these once the CI passes and is marked as non-draft.

@PraveenPenguin PraveenPenguin left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@harvey0100 thanks for the patch .. have review comment see if those make sense

shutil.rmtree(dest)
dest.mkdir(parents=True)
with zipfile.ZipFile(wheel_path) as archive:
archive.extractall(dest)

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

zipfile.ZipFile.extractall(dest) is called with no path-traversal guard. can we have like

with zipfile.ZipFile(wheel_path) as archive:
    dest_real = os.path.realpath(dest)
    for member in archive.namelist():
        member_path = os.path.realpath(
            os.path.join(dest_real, member)
        )
        if not member_path.startswith(dest_real + os.sep):
            raise RuntimeError(
                f"Wheel contains path-traversal entry: {member!r}"
            )
    archive.extractall(dest)

if os.path.isdir(package):
host_dir = os.path.abspath(package)
else:
cache_root = (

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

prepare_bootstrap() constructs its own cache_root expression here using cache_dirs[0] if cache_dirs else Path.cwd(). This duplicates the same conditional already resolved inside resolve_wheel_file().
when cache_dirs is None (the default), the guard if cache_dirs else treats None the same as an empty list — but resolve_wheel_file normalises None to [] first, so the two computations are not equivalent if callers pass None. The extracted site directory therefore ends up in a different location than the cached wheel, breaking the extraction marker cache across calls.

thinking can we do this way

    if os.path.isdir(package):
        host_dir = os.path.abspath(package)
    else:
        _cache_dirs = cache_dirs or []
        _cache_root = (
            Path(_cache_dirs[0] if _cache_dirs else Path.cwd())
            / "wheel-bootstrap"
        )
        host_dir = _extract_wheel(package, _cache_root / "site")

def _bootstrap_url(self):
return self.config.get(
"spawner.podman.avocado_spawner_wheel"
) or self.config.get("spawner.podman.avocado_spawner_egg")

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can we deprecation path as well something like

    def _bootstrap_url(self):
        wheel = self.config.get("spawner.podman.avocado_spawner_wheel")
        if wheel:
            return wheel
        egg = self.config.get("spawner.podman.avocado_spawner_egg")
        if egg:
            LOG.warning(
                "--spawner-podman-avocado-egg is deprecated; "
                "use --spawner-podman-avocado-wheel instead."
            )
            return egg
        return None

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: Review Requested

Development

Successfully merging this pull request may close these issues.

Avocado is stuck with podman container > fedora:38 Avocado venv bootstrap - deprecation/removal of eggs

3 participants