Conversation
Uses RGI6, requiring heavy pre-processing to align with RGI7
Print help message.
We now seperate model configuration (toml file) and run / HPC configuration (j2) file.
- removed unused templates
Let's try without.
…aging checks
Runners (glacier, ismip7/greenland, kitp):
- Collapse seven individual CLI override params (resolution, nodes,
ntasks, tasks, queue, walltime, stress_balance) on run_glacier /
run_greenland / run_kitp into a single ``config_cli: dict | None``.
Bodies use ``config_cli = config_cli or {}`` and ``config_cli.get(...)``;
run_single/run_ensemble build the dict from argparse.
- Add ``--stress-balance`` flag that swaps ``[stress_balance].model`` at
run time. The previously-selected model's keys are popped from the run
dict before the new model's options are applied so leftover flags
(e.g. blatter.*) don't leak into e.g. a sia run.
Stage scripts (kitp, ismip7/greenland):
- Parallelize ``check_xr_lazy`` validation via
``ThreadPoolExecutor(max_workers=8)`` + ``tqdm``. Only invalid paths
print; the per-file ✓ chatter is silenced.
- ismip7 also deduplicates the per-GCM file lookups: surface_input_file
aliases climate, frontal_melt_file aliases ocean, so each path is
validated once instead of twice.
Configs: minor tweaks to kitp_greenland*.toml and setup_ismip7_greenland.toml
to match the new flow.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
- New ``pism_terra/validate.py`` (registered as ``pism-validate`` entrypoint): recursively validates every NetCDF under a directory via ``check_xr_lazy`` with a parallel worker pool and a tqdm progress bar. Only invalid paths print; exit code is non-zero when anything fails. - ``kitp/stage.py`` and ``ismip7/greenland/stage.py``: swap the parallel ``check_xr_lazy`` batches from threads to ``ProcessPoolExecutor``. HDF5 isn't reliably thread-safe across all builds (Chinook's HPC build segfaults under concurrent thread opens); each worker now gets its own interpreter and HDF5 state. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…aaschwanden/ismip7
Aaschwanden/ismip7
The glacier/ismip7/kitp prepare modules import add_include_argument and select_datasets from pism_terra.prepare_select, but the module file was never tracked — so CI (which runs on the committed tree) failed with pylint E0611 no-name-in-module. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Implements the section-8 conventions: when output.ISMIP6 is set, the
forward run writes spatial/scalar outputs into the
<domain>/<source>/<ism>/<set>/<set_counter>/ tree with conforming
filenames (<variable_id>_<domain>_<source>_<ism>_<ISM_member>_<ESM>_
<forcing_member>_<experiment>_<set_counter>_<time_range>.nc). PISM's
{var} placeholder yields one conforming file per spatial variable; the
scalar split and CF/attribute conformance are deferred to postprocess.
- config.py: InfoConfig now declares the ISMIP7 run_info metadata
(group, model, domain, set, ism, experiment, contact_*), which were
previously dropped by extra="ignore". as_params() keeps them out of
the PISM command (only institution/title are forwarded).
- ismip7/naming.py: ISMIP7Names (directory/stem/filename) and
set-type-aware member_ids (CORE fixes m001/f001, PPE varies mNNN,
ESM varies fNNN; set_counter always increments).
- ismip7/greenland/run.py: _render_forward_run routes outputs into the
ISMIP7 tree when output.ISMIP6 is truthy; ESM_id from the GCM,
set_counter ordered by gcms position, time_range from the run years
(Jan-1 end -> last full year). Errors if a required run_info field
is missing. Also removes the stale getattr(reporting, "output.ISMIP6")
debug block that crashed the run.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
ismip7/greenland/run.py imports ISMIP7Names and member_ids from pism_terra.ismip7.naming, but the module file was never tracked, so CI (linting the committed tree) failed with pylint E0611 no-name-in-module. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Aaschwanden/prepare refactor
data_<i>.nc per variable. _cds_finish_year was returning only matching[0], silently dropping the rest — which broke CARRA2 radiation staging (asks for ssrd + ssr together, downstream prepare_carra2 then bailed with "Expected exactly two shortwave variables"). Open every extracted .nc and xr.merge them into the canonical nc_path. Single-file responses are still returned unchanged.
…into aaschwanden/ismip7-gris
Aaschwanden/ismip7 gris
Split into historical and projection run. Hardwired This also currently hard wires 2300 as the end year. Needs to be adjusted by experiment
The Core Experiment identity (experiment, ESM/gcm, forcing pathway, and end year) was duplicated across run_info.experiment and campaign.pathway/ gcms. Since the ISMIP7 Core Experiment Overview table maps each counter_id (C001-C008) uniquely to those values, encode that mapping once and let a single run_info.counter field drive both staging and running. - Add pism_terra/ismip7/experiments.py: CoreExperiment dataclass, CORE_EXPERIMENTS mapping (C001-C008), and resolve_counter(). - config.py: add run_info.counter (naming-only) and a PismConfig validator that expands it into experiment/pathway/gcms/time.end. Make time.end optional (filled by the counter) with a guard for the no-counter case. Legacy non-counter configs are unaffected. - ismip7/greenland/run.py: use the protocol counter as the ISMIP7 set_counter (was derived from GCM list index), derive the projection end from time.end (was hardcoded 2300-01-01), and emit ISMIP7 names only for the counter's product leg (the other leg gets flat names). C001/C002 keep the historical leg as the product and continue on ssp585 to 2100; C003-C008 keep the historical spin-up but submit the projection leg. - Add ismip7_greenland_c001.toml..c008.toml (standalone; only run_info.counter differs, derived fields removed). - Add tests/test_ismip7_counter.py. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014DGQGDhsstkb1faoVUwh4M
Aaschwanden/ismip7 gris
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.