Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
28 commits
Select commit Hold shift + click to select a range
880a882
Add GPU hamming improvements
felixpetschko Jul 1, 2026
3d40391
Change cutoff guard to <=125
felixpetschko Jul 1, 2026
d053bee
Add parameter guards for block size parameters
felixpetschko Jul 1, 2026
5200588
Add tests for different block size parameters
felixpetschko Jul 1, 2026
5b89ab2
Adapt GPUHammingDistanceCalculator doc string
felixpetschko Jul 1, 2026
7718e2c
Change sequence length array data type to int32
felixpetschko Jul 1, 2026
9d3252d
Avoid unnecessary dtype copies in GPU hamming transfers
felixpetschko Jul 1, 2026
80ee4c5
Replace asserts with exceptions
felixpetschko Jul 1, 2026
dada164
Transform guarding assert into error
felixpetschko Jul 7, 2026
6887bf3
Remove benchmarking option
felixpetschko Jul 7, 2026
86391e7
Skip calculation of blocks under diagonal
felixpetschko Jul 7, 2026
1ab2ffb
Update block skipping
felixpetschko Jul 7, 2026
701b860
Update block skipping to work with tqdm
felixpetschko Jul 7, 2026
c82add0
Remove logic to only compute upper triangular half for symmetric resu…
felixpetschko Jul 7, 2026
4884d50
Extend test cases to cover block skipping
felixpetschko Jul 7, 2026
efa31eb
Update changelog
felixpetschko Jul 7, 2026
a2bba48
Add GPU hamming integration test
felixpetschko Jul 7, 2026
940d3a0
Prevent out-of-bounds writes in GPU Hamming kernel
felixpetschko Jul 22, 2026
d5eef6e
Handle excess blocks in GPU Hamming calculator
felixpetschko Jul 22, 2026
0530c30
Support joblib blocks in GPU Hamming calculator
felixpetschko Jul 22, 2026
4c9f30e
Balance symmetric distance blocks and use fixed GPU tile sizes
felixpetschko Jul 23, 2026
6d22b8b
Improve GPU Hamming block sizing and buffer handling
felixpetschko Jul 23, 2026
c5938ab
Change parameter names
felixpetschko Jul 23, 2026
7f4b593
Log tile retry info
felixpetschko Jul 23, 2026
14e0bcf
Test maximum GPU Hamming cutoff
felixpetschko Jul 23, 2026
9fea04d
Extend large dataset tutorial
felixpetschko Jul 23, 2026
717974b
Reformat
felixpetschko Jul 23, 2026
7feb2b4
Adapt input array memory management
felixpetschko Jul 26, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,10 @@ and this project adheres to [Semantic Versioning][].
### Performance improvements

- Speed up identity distance metric computation for comparisons between two different sequence arrays ([#701](https://github.com/scverse/scirpy/pull/701)).
- Improve the GPU implementation of the Hamming distance metric with row and column tiling, tile skipping for
symmetric distance matrices, adaptive result buffers, and faster sparse tile assembly. GPU tiles can be configured
with `gpu_tile_rows` and `gpu_tile_cols`; `gpu_n_blocks` has been replaced by `gpu_tile_cols`, and `gpu_block_width`
has been replaced by `gpu_tile_buffer_cols`.

### Chore

Expand Down
43 changes: 40 additions & 3 deletions docs/tutorials/large-datasets.md
Original file line number Diff line number Diff line change
Expand Up @@ -59,10 +59,47 @@ First, install the optional `cupy` dependency:
!pip install scirpy[cupy]
```

Then simply run
Then run

```
```python
ir.pp.ir_dist(mdata, metric="gpu_hamming")
```

to take advantage of GPU acceleration.
to use the first available GPU. The calculation is split into tiles. Their size can be adjusted with
`gpu_tile_rows` and `gpu_tile_cols`: smaller tiles use less GPU memory but increase processing overhead.
`gpu_tile_buffer_cols` controls the initially reserved space for retained distances and is enlarged automatically
if necessary.

### Using multiple GPUs

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do you think linking to the rapids-singlecell docs for advanced usage would make sense here?


Multiple GPUs can be used through the joblib dask backend and
[dask-cuda](https://docs.rapids.ai/api/dask-cuda/stable/):

```
!pip install "dask[distributed]" dask-cuda
```

The following example creates one dask worker per GPU and divides the distance calculation into two outer
partitions:

```python
import joblib
from dask.distributed import Client
from dask_cuda import LocalCUDACluster

with LocalCUDACluster(
CUDA_VISIBLE_DEVICES="0,1",
n_workers=2,
threads_per_worker=1,
) as cluster, Client(cluster):
with joblib.parallel_config(backend="dask", n_jobs=2):
ir.pp.ir_dist(
mdata,
metric="gpu_hamming",
n_blocks=2,
)
```

Set `CUDA_VISIBLE_DEVICES`, `n_workers`, `n_jobs`, and `n_blocks` to the number of GPUs to use. `n_blocks` controls
the outer partitions distributed between workers, whereas `gpu_tile_rows` and `gpu_tile_cols` control the smaller
tiles computed within each worker.
Loading
Loading