perf: Pipelined parallel extract - #72

Closed

Pr0methean wants to merge 16 commits into

Commits on Sep 18, 2023

and we got a benchmark
cosmicexplorer
committed
47d2c39
significantly faster pipelined extraction!!!
cosmicexplorer
committed
009c6dd

Commits on Sep 20, 2023

using a secondary rayon threadpool is SLOWER than manual threading!!
cosmicexplorer
committed
5d939b8
convert Vec<u8> to Bytes/BytesMut
cosmicexplorer
committed
ddd7cf5
add once_cell
cosmicexplorer
committed
e0afdd4
conver to rayon::join
cosmicexplorer
committed
ebb9148
reuse the file handle to set permissions
cosmicexplorer
committed
321ffca
clarify randomness in bench (means incompressibility)
cosmicexplorer
committed
24e76c7
move dir creation out of the hot path
cosmicexplorer
committed
1922204
avoid the need to reverse by hand
cosmicexplorer
committed
e4dbf6e

Commits on Sep 21, 2023

ok the pipelining works perfectly and lucidrously fast now
cosmicexplorer
committed
f4c11f1

Commits on May 1, 2024

Merge remote-tracking branch 'cosmicex/pipelined-extract' into oldpr407
Pr0methean
committed
a9a182b

Commits on May 2, 2024

Merge branch 'master' into oldpr407
Pr0methean
authored
64fadca

Commits on May 6, 2024

Merge branch 'master' into oldpr407
Pr0methean
authored
fdc7b2b

Commits on May 10, 2024

Merge branch 'master' into oldpr407
Pr0methean
authored
e0bf245
chore: Partial fix for build errors
Pr0methean
authored
ff8a59a