This code repo corresponds to the "Reduction (Sum)" blog post series:
mkdir build && cd build
cmake ..
cmake --build .
The threads in Kernel 0 to 3 only process one element each, i.e., num_elements_per_batch and NUM_THREADS_PER_BATCH must be identical. If the values differ, it would cause incorrect sum error since each thread only processes one element.
Similarly, num_elements_per_batch for running Kernel 7 has to be NUM_THREADS_PER_BATCH * <multiple of 4>.
A next step for cleaner code and easier usage is to automate or add additional checks to prevent the issues above.
- Optimizing Parallel Reduction in CUDA by Mark Harris (2007)
- CUDA Reduction by Lei Mao (2024)
- Programming Massively Parallel Processors, 4th Edition by Hwu, Kirk, and Hajj (2023)
- Faster Parallel Reductions on Kepler by Justin Luitjens (2014)
- CUDA Pro Tip: Do The Kepler Shuffler by Mark Harris (2014)
- Using CUDA Warp-Level Primitives by Yuan Lin and Vinod Grover (2018)