From a228448183ecb85831809c22454a6c4dd84e10b7 Mon Sep 17 00:00:00 2001 From: Xavier Roynard Date: Sat, 18 Apr 2026 00:58:19 +0200 Subject: [PATCH 1/8] docs: add structured AGENTS.md with nested module guides Add a comprehensive root AGENTS.md following modeles_d_agents best practices, plus nested AGENTS.md files for the containers and storage modules. --- AGENTS.md | 182 +++++++++++++++++++++++++++++++++ src/plaid/containers/AGENTS.md | 28 +++++ src/plaid/storage/AGENTS.md | 46 +++++++++ 3 files changed, 256 insertions(+) create mode 100644 AGENTS.md create mode 100644 src/plaid/containers/AGENTS.md create mode 100644 src/plaid/storage/AGENTS.md diff --git a/AGENTS.md b/AGENTS.md new file mode 100644 index 00000000..adcc4d99 --- /dev/null +++ b/AGENTS.md @@ -0,0 +1,182 @@ +# AGENTS.md -- plaid (pyplaid) + +## Project identity + +**plaid** is the foundational data model library of the [PLAID ecosystem](https://github.com/PLAID-lib). +Published on PyPI as `pyplaid`, it provides a structured format for representing physics +simulation data (meshes, fields, boundary conditions) and abstracts storage backends +(zarr, HuggingFace datasets, CGNS). + +Every other library in the ecosystem depends on plaid. + +## Expected agent behavior + +### Role + +You are a senior Python developer with experience in scientific computing, data modeling, +and open-source library design. You prioritize backward compatibility and clean abstractions. + +### Decision priorities + +1. **Backward compatibility** > new features -- this is a foundational library, breaking downstream users is costly +2. **Correctness** > performance -- data integrity in scientific computing is non-negotiable +3. **Readability** > cleverness -- contributors come from diverse scientific backgrounds + +### When in doubt + +- Do not change public API signatures without explicit approval +- Prefer adding new optional parameters with sensible defaults +- Check if the change impacts `scimm` or `maestro` (downstream consumers) +- Run the full test suite before proposing changes + +## Tech stack + +- **Language**: Python 3.11--3.13 +- **Package manager**: uv (with `pyproject.toml`) +- **Build backend**: setuptools with setuptools-scm (dynamic versioning) +- **Linter/formatter**: ruff +- **Test framework**: pytest +- **Documentation**: Sphinx (ReadTheDocs) +- **CI/CD**: GitHub Actions + +## Project structure + +``` +. +├── AGENTS.md <- This file +├── pyproject.toml <- Dependencies and project metadata +├── ruff.toml <- Ruff linter/formatter configuration +├── CHANGELOG.md <- Version history +├── CONTRIBUTING.md <- Contribution guidelines +├── src/plaid/ <- Source code +│ ├── __init__.py +│ ├── constants.py <- Global constants +│ ├── problem_definition.py <- ProblemDefinition (core concept) +│ ├── containers/ <- Dataset, Sample, Features (see nested AGENTS.md) +│ ├── storage/ <- Storage backends: zarr, hf_datasets, cgns (see nested AGENTS.md) +│ ├── bridges/ <- HuggingFace bridge utilities +│ ├── pipelines/ <- sklearn-compatible processing blocks +│ ├── post/ <- Post-processing (metrics, bisection) +│ └── examples/ <- Built-in example datasets +├── tests/ <- Test suite +├── docs/ <- Sphinx documentation source +├── examples/ <- Usage examples +└── benchmarks/ <- Performance benchmarks +``` + +## Architecture and key concepts + +### Core abstractions + +| Concept | Module | Description | +|---------|--------|-------------| +| `ProblemDefinition` | `problem_definition.py` | Declares fields, meshes, and their roles (input/output/context) for a physics problem | +| `Sample` | `containers/sample.py` | One simulation snapshot: mesh + field values | +| `Dataset` | `containers/dataset.py` | Ordered collection of Samples with shared ProblemDefinition | +| `Features` | `containers/features.py` | Named tensor-like data with metadata | +| `FeatureIdentifier` | `containers/feature_identifier.py` | Unique key to identify a feature across samples | + +### Storage pattern + +Storage uses a **Registry pattern** (`storage/registry.py`) to dispatch read/write +operations to the correct backend (zarr, hf_datasets, cgns). Each backend implements +a `reader.py` and `writer.py` following a common interface defined in `storage/common/`. + +### Dependency graph (ecosystem) + +``` +plaid (pyplaid) scimm + ^ ^ + | pyplaid>=0.1.13 | scimm>=0.2.0 + | | + +----------+-------------+ + | + maestro (glue layer) +``` + +plaid has **no dependency** on scimm or maestro. Changes here propagate downstream. + +## Code conventions + +### Formatting and linting + +Ruff is configured in `ruff.toml`: +- **Line length**: 88 characters +- **Lint rules**: `D` (docstrings), `E`/`W` (pycodestyle), `F` (pyflakes), `ARG` (unused arguments), `I` (import sorting) +- **Docstring convention**: Google style +- **Excluded directories**: `examples/`, `docs/`, `benchmarks/` +- **Test files**: docstring rules (`D`) and `S101` (assert) are ignored + +```bash +# Check linting +uv run ruff check . + +# Auto-fix +uv run ruff check --fix . + +# Format +uv run ruff format . +``` + +### Type hints + +- Required on all public functions and methods +- Use modern syntax: `list[str]`, `dict[str, int]`, `X | None` (not `Optional[X]`) +- Never use deprecated `typing.List`, `typing.Dict`, `typing.Optional` + +### Docstrings + +- Google style (enforced by ruff rule `D` with `convention = "google"`) +- Required on all public modules, classes, functions, and methods +- Update docstrings whenever you modify code behavior + +## Testing + +- **Framework**: pytest +- **Location**: `tests/` +- **Run all**: `uv run pytest` +- **Run specific**: `uv run pytest tests/path/to/test_file.py` +- **With coverage**: `uv run pytest --cov=src` + +Guidelines: +- Write tests for new public functions, classes, and methods +- Test edge cases and error conditions +- Use descriptive test names that explain the scenario +- Mock external dependencies (file I/O, network) to keep tests fast +- Do not test trivial code or third-party libraries + +## Commands + +```bash +# Install dependencies +uv sync + +# Run tests +uv run pytest + +# Check linting +uv run ruff check . + +# Auto-fix linting issues +uv run ruff check --fix . + +# Format code +uv run ruff format . + +# Build documentation +cd docs && make html +``` + +## Contribution workflow + +When making changes: + +1. Read and understand existing code before modifying +2. Write or update code with type hints +3. Write unit tests for new functionality +4. Update docstrings (Google style) +5. Update Sphinx documentation if functionality changed +6. Run formatter: `uv run ruff format .` +7. Run linter: `uv run ruff check --fix .` +8. Run tests: `uv run pytest` +9. Check if changes are breaking and inform the reviewer if a major version bump is needed diff --git a/src/plaid/containers/AGENTS.md b/src/plaid/containers/AGENTS.md new file mode 100644 index 00000000..675503f8 --- /dev/null +++ b/src/plaid/containers/AGENTS.md @@ -0,0 +1,28 @@ +# AGENTS.md -- plaid/containers + +This module defines the core data containers of the PLAID data model. + +## Key classes + +| Class | File | Description | +|-------|------|-------------| +| `Dataset` | `dataset.py` | Ordered collection of `Sample` objects sharing a common `ProblemDefinition`. Main entry point for loading and manipulating simulation data. | +| `Sample` | `sample.py` | Single simulation snapshot containing mesh coordinates and field values as `Features`. | +| `Features` | `features.py` | Named tensor-like container with shape and dtype metadata. Wraps numpy arrays. | +| `FeatureIdentifier` | `feature_identifier.py` | Immutable key (name + location) used to uniquely identify a feature across samples. | +| `DefaultManager` | `managers/default_manager.py` | Manages default values and missing data for features within a dataset. | + +## Design constraints + +- `Dataset` is a **large class** (~1800 lines). Avoid adding new responsibilities to it. Prefer extracting logic into helper functions or dedicated modules. +- `Sample` and `Features` are **value objects** -- they should remain simple, with minimal business logic. +- `FeatureIdentifier` is **immutable and hashable** -- it is used as dictionary keys throughout the codebase. Do not add mutable state. +- All containers must support **serialization** through the storage backends (zarr, hf_datasets, cgns). + +## Downstream impact + +These classes are the public API surface consumed by `maestro` and end users. Any signature change is a **breaking change** that requires a major version bump. + +## Testing + +Tests are in `tests/`. When modifying a container class, verify that storage round-trips (write then read) still produce identical data. diff --git a/src/plaid/storage/AGENTS.md b/src/plaid/storage/AGENTS.md new file mode 100644 index 00000000..74805e4d --- /dev/null +++ b/src/plaid/storage/AGENTS.md @@ -0,0 +1,46 @@ +# AGENTS.md -- plaid/storage + +This module implements the multi-backend storage layer for reading and writing PLAID datasets. + +## Architecture + +Storage follows a **Registry pattern**: + +``` +storage/ +├── registry.py <- Dispatches to the correct backend based on format +├── reader.py <- Public read API (delegates to backend readers) +├── writer.py <- Public write API (delegates to backend writers) +├── common/ <- Abstract interfaces and shared utilities +│ ├── reader.py <- Base reader interface +│ ├── writer.py <- Base writer interface +│ ├── bridge.py <- Format conversion helpers +│ └── preprocessor.py +├── zarr/ <- Zarr backend (reader.py, writer.py, bridge.py) +├── hf_datasets/ <- HuggingFace datasets backend (reader.py, writer.py, bridge.py) +└── cgns/ <- CGNS backend (reader.py, writer.py) +``` + +## How it works + +1. The **registry** (`registry.py`) maps format identifiers to backend modules. +2. The public `reader.py` and `writer.py` at the top level accept a format parameter and delegate to the appropriate backend. +3. Each backend implements the interfaces defined in `common/reader.py` and `common/writer.py`. + +## Adding a new backend + +1. Create a new subdirectory under `storage/` (e.g., `storage/my_format/`). +2. Implement `reader.py` and `writer.py` following the interfaces in `common/`. +3. Register the new backend in `registry.py`. +4. Add round-trip tests (write then read) to verify data integrity. + +## Design constraints + +- Backends must be **stateless** -- all configuration is passed through function parameters. +- Read/write operations must preserve **data integrity** exactly (no lossy conversions without explicit user consent). +- The `common/` interfaces are the **contract** -- do not add backend-specific parameters to the public API without updating the contract first. +- `zarr` is the primary backend and the most feature-complete. Use it as the reference when implementing others. + +## Testing + +Each backend should have round-trip tests that write a dataset and read it back, asserting equality. Tests are in `tests/`. From b2ba8229459a20a438986e62345686d305faad61 Mon Sep 17 00:00:00 2001 From: Xavier Roynard Date: Sat, 18 Apr 2026 17:55:34 +0200 Subject: [PATCH 2/8] docs: remove private repo names from public AGENTS.md, add PR rules and confidentiality section - Remove all mentions of scimm and maestro (private repos) from public-facing content - Add confidentiality section warning agents not to mention private repos - Add PR title emoji rules from .github/pull_request_template.md - Add PR checklist reference --- AGENTS.md | 45 ++++++++++++++++++++++------------ src/plaid/containers/AGENTS.md | 2 +- 2 files changed, 30 insertions(+), 17 deletions(-) diff --git a/AGENTS.md b/AGENTS.md index adcc4d99..93ac12be 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -7,7 +7,7 @@ Published on PyPI as `pyplaid`, it provides a structured format for representing simulation data (meshes, fields, boundary conditions) and abstracts storage backends (zarr, HuggingFace datasets, CGNS). -Every other library in the ecosystem depends on plaid. +Other libraries in the ecosystem depend on plaid. ## Expected agent behavior @@ -26,9 +26,16 @@ and open-source library design. You prioritize backward compatibility and clean - Do not change public API signatures without explicit approval - Prefer adding new optional parameters with sensible defaults -- Check if the change impacts `scimm` or `maestro` (downstream consumers) +- Check if the change impacts downstream consumers of plaid - Run the full test suite before proposing changes +### Confidentiality + +plaid is a **public** repository. Some downstream libraries in the PLAID ecosystem are private. +Never mention private repository names, internal project names, or confidential details +in any public-facing content (code comments, docstrings, commit messages, PR descriptions, +issues, or documentation). + ## Tech stack - **Language**: Python 3.11--3.13 @@ -82,20 +89,6 @@ Storage uses a **Registry pattern** (`storage/registry.py`) to dispatch read/wri operations to the correct backend (zarr, hf_datasets, cgns). Each backend implements a `reader.py` and `writer.py` following a common interface defined in `storage/common/`. -### Dependency graph (ecosystem) - -``` -plaid (pyplaid) scimm - ^ ^ - | pyplaid>=0.1.13 | scimm>=0.2.0 - | | - +----------+-------------+ - | - maestro (glue layer) -``` - -plaid has **no dependency** on scimm or maestro. Changes here propagate downstream. - ## Code conventions ### Formatting and linting @@ -145,6 +138,26 @@ Guidelines: - Mock external dependencies (file I/O, network) to keep tests fast - Do not test trivial code or third-party libraries +## Pull request rules + +PR titles **must start with one of the following emojis** to indicate the type of change: + +| Emoji | Type | +|-------|------| +| 🐛 | Bug fix | +| 📄 | Documentation | +| 🎉 | New feature or initial commit | +| 🚀 | Performance or deployment | +| ♻️ | Refactor or cleanup | +| 📦 | Packaging or dependency management | + +PR checklist (from `.github/pull_request_template.md`): +- Typing enforced +- Documentation updated +- Changelog updated +- Tests and example updates +- Coverage should be 100% + ## Commands ```bash diff --git a/src/plaid/containers/AGENTS.md b/src/plaid/containers/AGENTS.md index 675503f8..21bcb61c 100644 --- a/src/plaid/containers/AGENTS.md +++ b/src/plaid/containers/AGENTS.md @@ -21,7 +21,7 @@ This module defines the core data containers of the PLAID data model. ## Downstream impact -These classes are the public API surface consumed by `maestro` and end users. Any signature change is a **breaking change** that requires a major version bump. +These classes are the public API surface consumed by downstream libraries and end users. Any signature change is a **breaking change** that requires a major version bump. ## Testing From cd4b3b213e92fca6dc20a7cf4393c739920ecc4c Mon Sep 17 00:00:00 2001 From: Xavier Roynard Date: Sat, 18 Apr 2026 18:43:55 +0200 Subject: [PATCH 3/8] docs: add communication rules (English, direct tone) --- AGENTS.md | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/AGENTS.md b/AGENTS.md index 93ac12be..333c9920 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -36,6 +36,11 @@ Never mention private repository names, internal project names, or confidential in any public-facing content (code comments, docstrings, commit messages, PR descriptions, issues, or documentation). +### Communication rules + +- All interactions on this repository (issues, PRs, reviews, comments) must be in **English**. +- Be direct and concise. Avoid compliments, flattery, or filler sentences. + ## Tech stack - **Language**: Python 3.11--3.13 From 0e8d4c56f2b3520ac71bae50c69af7145e324ca6 Mon Sep 17 00:00:00 2001 From: Xavier Roynard Date: Fri, 5 Jun 2026 20:43:26 +0000 Subject: [PATCH 4/8] =?UTF-8?q?=F0=9F=93=84=20docs(agents):=20update=20AGE?= =?UTF-8?q?NTS.md=20files=20for=20v1.0.0=20structure?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Sync the root, containers and storage AGENTS.md with the post-V1 repo: - root: fix project tree (drop removed bridges/pipelines/post/examples, add cli/types/viewer/downloadable_examples), fix Core abstractions table (Dataset/Features/FeatureIdentifier removed -> Sample/Infos/ProblemDefinition), doc tooling Sphinx -> Zensical, build command -> docs/generate_doc.sh - containers: Dataset/Features/FeatureIdentifier no longer exist; document Sample (pydantic BaseModel), DefaultManager and utils helpers - storage: add backend_api.py (BackendModule Protocol) and the BACKENDS registry --- AGENTS.md | 39 +++++++++++++++++++++------------- src/plaid/containers/AGENTS.md | 33 +++++++++++++++++----------- src/plaid/storage/AGENTS.md | 14 ++++++++---- 3 files changed, 55 insertions(+), 31 deletions(-) diff --git a/AGENTS.md b/AGENTS.md index 333c9920..de108f61 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -48,7 +48,7 @@ issues, or documentation). - **Build backend**: setuptools with setuptools-scm (dynamic versioning) - **Linter/formatter**: ruff - **Test framework**: pytest -- **Documentation**: Sphinx (ReadTheDocs) +- **Documentation**: Zensical (with mkdocstrings for the API reference), published on ReadTheDocs - **CI/CD**: GitHub Actions ## Project structure @@ -61,21 +61,25 @@ issues, or documentation). ├── CHANGELOG.md <- Version history ├── CONTRIBUTING.md <- Contribution guidelines ├── src/plaid/ <- Source code -│ ├── __init__.py +│ ├── __init__.py <- Public API: Sample, Infos, ProblemDefinition │ ├── constants.py <- Global constants │ ├── problem_definition.py <- ProblemDefinition (core concept) -│ ├── containers/ <- Dataset, Sample, Features (see nested AGENTS.md) +│ ├── infos.py <- Infos (dataset/problem metadata) +│ ├── containers/ <- Sample container + helpers (see nested AGENTS.md) │ ├── storage/ <- Storage backends: zarr, hf_datasets, cgns (see nested AGENTS.md) -│ ├── bridges/ <- HuggingFace bridge utilities -│ ├── pipelines/ <- sklearn-compatible processing blocks -│ ├── post/ <- Post-processing (metrics, bisection) -│ └── examples/ <- Built-in example datasets +│ ├── types/ <- Shared type aliases and definitions +│ ├── cli/ <- Command-line entry points (e.g. plaidcheck) +│ ├── viewer/ <- Dataset visualization services +│ └── downloadable_examples/ <- Built-in downloadable example datasets ├── tests/ <- Test suite ├── docs/ <- Sphinx documentation source -├── examples/ <- Usage examples -└── benchmarks/ <- Performance benchmarks +└── examples/ <- Usage examples ``` +> Note: the v1.0.0 reorganization removed the top-level `Dataset` re-export and the +> `bridges/`, `pipelines/`, `post/` and `examples/` source packages. Data is now handled +> through `Sample` objects and the `storage` layer. See `docs/source/upgrade_guide.md`. + ## Architecture and key concepts ### Core abstractions @@ -83,16 +87,21 @@ issues, or documentation). | Concept | Module | Description | |---------|--------|-------------| | `ProblemDefinition` | `problem_definition.py` | Declares fields, meshes, and their roles (input/output/context) for a physics problem | -| `Sample` | `containers/sample.py` | One simulation snapshot: mesh + field values | -| `Dataset` | `containers/dataset.py` | Ordered collection of Samples with shared ProblemDefinition | -| `Features` | `containers/features.py` | Named tensor-like data with metadata | -| `FeatureIdentifier` | `containers/feature_identifier.py` | Unique key to identify a feature across samples | +| `Infos` | `infos.py` | Metadata describing a dataset/problem (legal, data production, etc.) | +| `Sample` | `containers/sample.py` | One simulation snapshot: mesh + field values (a pydantic `BaseModel`) | + +`Sample`, `Infos` and `ProblemDefinition` are re-exported at the top level of the +`plaid` package, together with the helpers `get_number_of_samples` and `get_sample_ids` +from `containers/utils.py`. ### Storage pattern Storage uses a **Registry pattern** (`storage/registry.py`) to dispatch read/write operations to the correct backend (zarr, hf_datasets, cgns). Each backend implements -a `reader.py` and `writer.py` following a common interface defined in `storage/common/`. +a `reader.py` and `writer.py` following the backend contract defined in +`storage/backend_api.py` and the shared interfaces in `storage/common/`. +Reading/writing a collection of samples is done through this storage layer rather +than through a dedicated `Dataset` class. ## Code conventions @@ -182,7 +191,7 @@ uv run ruff check --fix . uv run ruff format . # Build documentation -cd docs && make html +bash docs/generate_doc.sh ``` ## Contribution workflow diff --git a/src/plaid/containers/AGENTS.md b/src/plaid/containers/AGENTS.md index 21bcb61c..144fd2f3 100644 --- a/src/plaid/containers/AGENTS.md +++ b/src/plaid/containers/AGENTS.md @@ -1,28 +1,37 @@ # AGENTS.md -- plaid/containers -This module defines the core data containers of the PLAID data model. +This module defines the core data container of the PLAID data model. ## Key classes | Class | File | Description | |-------|------|-------------| -| `Dataset` | `dataset.py` | Ordered collection of `Sample` objects sharing a common `ProblemDefinition`. Main entry point for loading and manipulating simulation data. | -| `Sample` | `sample.py` | Single simulation snapshot containing mesh coordinates and field values as `Features`. | -| `Features` | `features.py` | Named tensor-like container with shape and dtype metadata. Wraps numpy arrays. | -| `FeatureIdentifier` | `feature_identifier.py` | Immutable key (name + location) used to uniquely identify a feature across samples. | -| `DefaultManager` | `managers/default_manager.py` | Manages default values and missing data for features within a dataset. | +| `Sample` | `sample.py` | Single simulation snapshot containing mesh coordinates and field values. Implemented as a pydantic `BaseModel`. This is the main data container exposed by plaid. | +| `DefaultManager` | `managers/default_manager.py` | Manages default values and missing data for features within a sample. | + +Helper functions live in `utils.py` (e.g. `get_number_of_samples`, `get_sample_ids`) +and are re-exported at the top level of the `plaid` package. + +> Note: the v1.0.0 reorganization removed the `Dataset`, `Features` and +> `FeatureIdentifier` classes. A collection of samples is now read/written through the +> `storage` layer rather than a dedicated `Dataset` class. See `docs/source/upgrade_guide.md`. ## Design constraints -- `Dataset` is a **large class** (~1800 lines). Avoid adding new responsibilities to it. Prefer extracting logic into helper functions or dedicated modules. -- `Sample` and `Features` are **value objects** -- they should remain simple, with minimal business logic. -- `FeatureIdentifier` is **immutable and hashable** -- it is used as dictionary keys throughout the codebase. Do not add mutable state. -- All containers must support **serialization** through the storage backends (zarr, hf_datasets, cgns). +- `Sample` is a **value object** built on pydantic -- keep it focused on holding mesh + and field data, with minimal business logic. Prefer extracting heavy logic into + helper functions or dedicated modules. +- `DefaultManager` centralizes default/missing-data handling -- do not duplicate this + logic inside `Sample`. +- All containers must support **serialization** through the storage backends + (zarr, hf_datasets, cgns). ## Downstream impact -These classes are the public API surface consumed by downstream libraries and end users. Any signature change is a **breaking change** that requires a major version bump. +`Sample` is part of the public API surface consumed by downstream libraries and end +users. Any signature change is a **breaking change** that requires a major version bump. ## Testing -Tests are in `tests/`. When modifying a container class, verify that storage round-trips (write then read) still produce identical data. +Tests are in `tests/`. When modifying a container class, verify that storage round-trips +(write then read) still produce identical data. diff --git a/src/plaid/storage/AGENTS.md b/src/plaid/storage/AGENTS.md index 74805e4d..b2744e5d 100644 --- a/src/plaid/storage/AGENTS.md +++ b/src/plaid/storage/AGENTS.md @@ -9,6 +9,7 @@ Storage follows a **Registry pattern**: ``` storage/ ├── registry.py <- Dispatches to the correct backend based on format +├── backend_api.py <- Backend contract (BackendModule Protocol) ├── reader.py <- Public read API (delegates to backend readers) ├── writer.py <- Public write API (delegates to backend writers) ├── common/ <- Abstract interfaces and shared utilities @@ -23,15 +24,20 @@ storage/ ## How it works -1. The **registry** (`registry.py`) maps format identifiers to backend modules. +1. The **registry** (`registry.py`) holds a `BACKENDS` dict mapping each format name + (`"cgns"`, `"hf_datasets"`, `"zarr"`) to its backend class, exposed through + `get_backend(name)` and `available_backends()`. 2. The public `reader.py` and `writer.py` at the top level accept a format parameter and delegate to the appropriate backend. -3. Each backend implements the interfaces defined in `common/reader.py` and `common/writer.py`. +3. Each backend exposes a backend class (e.g. `ZarrBackend`, `HFBackend`, `CgnsBackend`) + that conforms to the `BackendModule` Protocol in `backend_api.py`, and implements the + read/write logic in its `reader.py` and `writer.py`. ## Adding a new backend 1. Create a new subdirectory under `storage/` (e.g., `storage/my_format/`). -2. Implement `reader.py` and `writer.py` following the interfaces in `common/`. -3. Register the new backend in `registry.py`. +2. Implement a backend class conforming to the `BackendModule` Protocol + (`backend_api.py`), with its `reader.py` and `writer.py` following the interfaces in `common/`. +3. Register the new backend by adding it to the `BACKENDS` dict in `registry.py`. 4. Add round-trip tests (write then read) to verify data integrity. ## Design constraints From 7748d739998620d46c3143deb31690fe6ffc3229 Mon Sep 17 00:00:00 2001 From: Xavier Roynard Date: Thu, 18 Jun 2026 14:35:49 +0200 Subject: [PATCH 5/8] =?UTF-8?q?=F0=9F=93=84=20docs:=20add=20"Efficiency=20?= =?UTF-8?q?and=20minimalism"=20section=20to=20AGENTS.md?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Distill the ponytail (minimal code, YAGNI ladder, deletion over addition) and caveman (terse communication, why over what) agent conventions into a PLAID-adapted ruleset. Includes explicit "never simplify away" guards for backward compatibility, API stability, validation, and data integrity. --- AGENTS.md | 30 ++++++++++++++++++++++++++++++ 1 file changed, 30 insertions(+) diff --git a/AGENTS.md b/AGENTS.md index de108f61..41750497 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -41,6 +41,36 @@ issues, or documentation). - All interactions on this repository (issues, PRs, reviews, comments) must be in **English**. - Be direct and concise. Avoid compliments, flattery, or filler sentences. +### Efficiency and minimalism + +These rules cut token usage, latency, and the volume of generated code without +sacrificing correctness. They are distilled from the `ponytail` (minimal code) and +`caveman` (terse communication) agent conventions. + +**Write the least code that works.** Climb this ladder, stop at the first rung that holds: + +1. Does this need to exist at all? Speculative need = skip it, say so in one line (YAGNI). +2. Does the standard library do it? Use it. +3. Does an already-installed dependency solve it? Use it -- never add a new dependency for what a few lines cover. +4. Can it be one line? Make it one line. +5. Only then: the minimum code that works. + +- No unrequested abstractions: no interface with a single implementation, no factory for one product, no config for a value that never changes. +- Deletion over addition. The shortest working diff wins. Fewest files possible. +- Mark a deliberate shortcut with a `# NOTE(shortcut):` comment naming its ceiling and upgrade trigger, e.g. `# NOTE(shortcut): O(n) scan, index it if the sample count grows`. +- Non-trivial logic leaves one runnable check behind (an `assert`-based self-check or a small `test_*.py`) -- no heavy fixtures unless asked. + +**Communicate tersely.** Why over what; the diff already says what. + +- Drop filler, pleasantries, and hedging. Code first, then at most a few short lines: what was skipped and when to add it. +- No tool-call narration, no decorative tables or emoji in explanations, no dumping long raw logs -- quote the shortest decisive line. +- If the explanation is longer than the code, delete the explanation. + +**Never simplify away** (these override the ladder): backward compatibility, public API +stability, input validation at trust boundaries, error handling that prevents data loss +or corruption, security, or anything explicitly requested. In scientific computing, data +integrity is non-negotiable -- see Decision priorities above. + ## Tech stack - **Language**: Python 3.11--3.13 From 78ef2e5618c1e59dc1d675e2d969ac0c4924ee8f Mon Sep 17 00:00:00 2001 From: Xavier Roynard Date: Wed, 29 Jul 2026 08:21:30 +0000 Subject: [PATCH 6/8] docs: fix factual inaccuracies in AGENTS.md (Zensical not Sphinx, plaid-check, utils/, ruff ignores) --- AGENTS.md | 12 +++++++----- 1 file changed, 7 insertions(+), 5 deletions(-) diff --git a/AGENTS.md b/AGENTS.md index 41750497..202a8074 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -98,11 +98,12 @@ integrity is non-negotiable -- see Decision priorities above. │ ├── containers/ <- Sample container + helpers (see nested AGENTS.md) │ ├── storage/ <- Storage backends: zarr, hf_datasets, cgns (see nested AGENTS.md) │ ├── types/ <- Shared type aliases and definitions -│ ├── cli/ <- Command-line entry points (e.g. plaidcheck) +│ ├── utils/ <- Internal helpers: base.py, cgns_helper.py, cgns_worker.py +│ ├── cli/ <- Command-line entry points (e.g. plaid-check) │ ├── viewer/ <- Dataset visualization services │ └── downloadable_examples/ <- Built-in downloadable example datasets ├── tests/ <- Test suite -├── docs/ <- Sphinx documentation source +├── docs/ <- Zensical (Markdown) documentation source └── examples/ <- Usage examples ``` @@ -138,11 +139,12 @@ than through a dedicated `Dataset` class. ### Formatting and linting Ruff is configured in `ruff.toml`: -- **Line length**: 88 characters +- **Line length**: 88 characters -- but `E501` (line-too-long) is in the `ignore` list, so this limit is enforced by the **formatter** (`ruff format`), not checked by the linter - **Lint rules**: `D` (docstrings), `E`/`W` (pycodestyle), `F` (pyflakes), `ARG` (unused arguments), `I` (import sorting) +- **Ignored rules**: `E501` (line too long) and `D107` (missing docstring in `__init__`) - **Docstring convention**: Google style - **Excluded directories**: `examples/`, `docs/`, `benchmarks/` -- **Test files**: docstring rules (`D`) and `S101` (assert) are ignored +- **Per-file ignores**: test files ignore docstring rules (`D`) and `S101` (assert); `__init__.py` files ignore `F401` (unused imports, to allow re-exports) ```bash # Check linting @@ -232,7 +234,7 @@ When making changes: 2. Write or update code with type hints 3. Write unit tests for new functionality 4. Update docstrings (Google style) -5. Update Sphinx documentation if functionality changed +5. Update the Zensical documentation if functionality changed 6. Run formatter: `uv run ruff format .` 7. Run linter: `uv run ruff check --fix .` 8. Run tests: `uv run pytest` From 9c5be0f3e311d507dfd1d255e62608aa6e9857a8 Mon Sep 17 00:00:00 2001 From: probe Date: Tue, 4 Aug 2026 14:26:03 +0000 Subject: [PATCH 7/8] docs: align AGENTS.md with current main (examples/, storage callbacks, plaid-serve/viewer CLIs) --- AGENTS.md | 8 +++++--- src/plaid/storage/AGENTS.md | 1 + 2 files changed, 6 insertions(+), 3 deletions(-) diff --git a/AGENTS.md b/AGENTS.md index 202a8074..06c07869 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -99,7 +99,7 @@ integrity is non-negotiable -- see Decision priorities above. │ ├── storage/ <- Storage backends: zarr, hf_datasets, cgns (see nested AGENTS.md) │ ├── types/ <- Shared type aliases and definitions │ ├── utils/ <- Internal helpers: base.py, cgns_helper.py, cgns_worker.py -│ ├── cli/ <- Command-line entry points (e.g. plaid-check) +│ ├── cli/ <- Command-line entry points (plaid-check, plaid-serve, plaid-viewer) │ ├── viewer/ <- Dataset visualization services │ └── downloadable_examples/ <- Built-in downloadable example datasets ├── tests/ <- Test suite @@ -108,8 +108,10 @@ integrity is non-negotiable -- see Decision priorities above. ``` > Note: the v1.0.0 reorganization removed the top-level `Dataset` re-export and the -> `bridges/`, `pipelines/`, `post/` and `examples/` source packages. Data is now handled -> through `Sample` objects and the `storage` layer. See `docs/source/upgrade_guide.md`. +> `bridges/`, `pipelines/` and `post/` source packages under `src/plaid/`. Data is now +> handled through `Sample` objects and the `storage` layer. (The top-level `examples/` +> directory still exists — it holds usage scripts, not an importable package.) +> See `docs/source/upgrade_guide.md`. ## Architecture and key concepts diff --git a/src/plaid/storage/AGENTS.md b/src/plaid/storage/AGENTS.md index b2744e5d..1fad64a8 100644 --- a/src/plaid/storage/AGENTS.md +++ b/src/plaid/storage/AGENTS.md @@ -10,6 +10,7 @@ Storage follows a **Registry pattern**: storage/ ├── registry.py <- Dispatches to the correct backend based on format ├── backend_api.py <- Backend contract (BackendModule Protocol) +├── callbacks.py <- Callback contracts (SampleCallbackContext, post-write hooks) ├── reader.py <- Public read API (delegates to backend readers) ├── writer.py <- Public write API (delegates to backend writers) ├── common/ <- Abstract interfaces and shared utilities From f987cc934a5eea3f3fa7e5885245893457f4e8f2 Mon Sep 17 00:00:00 2001 From: probe Date: Tue, 4 Aug 2026 16:15:29 +0000 Subject: [PATCH 8/8] docs(changelog): note AGENTS.md contributor guides under Unreleased/Added --- CHANGELOG.md | 1 + 1 file changed, 1 insertion(+) diff --git a/CHANGELOG.md b/CHANGELOG.md index 3c1a6772..b484afa4 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -10,6 +10,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ### Added - (storage/writer) add optional `sample_callback` to `save_to_disk`, invoked once per sample right after it is written to disk as `sample_callback(split_name, index, sample_path)`. This lets callers process samples one by one once written instead of waiting for the whole dataset. Currently supported for the `cgns` backend, including parallel writing (`num_proc > 1`), where the callback runs inside the worker processes and must be picklable and process-safe. +- (docs) add `AGENTS.md` contributor guides at the repository root and for the `containers/` and `storage/` subpackages, following the [AGENTS.md](https://agents.md/) convention. ### Fixed