Mark UniProt dat.gz files as temp to free ~200GB disk space - #249
Merged
Conversation
Copilot
AI
changed the title
[WIP] Clean up uniprot dat.gz files
Mark UniProt dat.gz files as temp to free ~200GB disk space
Aug 24, 2026
Mark uniprot_sprot.dat.gz and uniprot_trembl.dat.gz as temp so Snakemake removes them after use, freeing ~200GB of disk space. Closes #248 Co-authored-by: jonasfreimuth <68993915+jonasfreimuth@users.noreply.github.com>
jonasfreimuth
force-pushed
the
copilot/clean-up-uniprot-dat-gz-files
branch
from
August 24, 2026 15:48
26ac53a to
c91b1ad
Compare
Collaborator
|
Anticipates #250, needs to be merged after that. |
jonasfreimuth
marked this pull request as ready for review
August 24, 2026 15:50
jonasfreimuth
approved these changes
Aug 24, 2026
jonasfreimuth
left a comment
jonasfreimuth
left a comment
Collaborator
There was a problem hiding this comment.
Good from my side.
Contributor
There was a problem hiding this comment.
Pull request overview
This PR aims to reduce disk usage in the UniProt resource pipeline by allowing Snakemake to clean up large extracted UniProt *.dat.gz files once downstream rules are complete, and also includes a small robustness change in UniProt release parsing plus release metadata updates.
Changes:
- Mark
uniprot_sprot.dat.gzanduniprot_trembl.dat.gzastemp()outputs so they can be deleted after downstream consumption. - Adjust gzip parsing in
extract_uniprot_release.pyto avoid hard UTF-8 decode failures. - Bump release metadata to
2.0.1and add a2.0.1changelog entry.
Reviewed changes
Copilot reviewed 5 out of 5 changed files in this pull request and generated 3 comments.
Show a summary per file
| File | Description |
|---|---|
| workflow/rules/pull_resources.smk | Marks extracted UniProt snapshot *.dat.gz outputs as temporary to enable cleanup. |
| workflow/scripts/extract_uniprot_release.py | Changes gzip reading to handle decode errors more robustly. |
| pixi.toml | Bumps project version to 2.0.1. |
| CHANGELOG.md | Adds a 2.0.1 release entry (but currently doesn’t mention the UniProt temp-file cleanup). |
| .release-please-manifest.json | Updates release-please manifest version to 2.0.1. |
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
jonasfreimuth
temporarily deployed
to
github-pages
August 26, 2026 08:46 — with 
GitHub Actions
Inactive
GitHub Actions
Inactive
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The
uniprot_sprot.dat.gzanduniprot_trembl.dat.gzfiles extracted from the UniProt snapshot are not cleaned up after use, with TrEMBL alone consuming ~200GB that is of little direct value to users.Changes
workflow/rules/pull_resources.smk: Wrap both outputs ofextract_uniprot_snapshotintemp()so Snakemake deletes them once all downstream consumers have completeddat.gzfiles #248