-
Notifications
You must be signed in to change notification settings - Fork 4.3k
[C++] GZipCodec::Init does deflateInit and immediately throws it away #51308
Description
Describe the bug, including details regarding any error messages, version, and platform.
Performance issue in GZipCodec, no crashes/wrong results/..., but I still think it's a bug.
arrow/cpp/src/arrow/util/compression_zlib.cc
Lines 497 to 501 in cd304b2
This is wasteful: InitCompressor on line 497 calls deflateInit2, which at windowBits=15 and memLevel=8 allocates roughly 256 KB. InitDecompressor on line 501 calls EndCompressor which does deflateEnd and frees it again. It's pure waste. The only thing it gives us is some validation on the compression level.
The current code leaves the GZipCodec with a valid decompressor (which also allocates, but less) which is fine for the case where the caller wants to use the codec to do decompression, but is extra-wasteful for the case where the caller wants to compress: it allocates a compressor, frees it, allocates a decompressor, frees it, then allocates a compressor again.
I think the correct fix is firebolt-db#47, I'm happy to file it as a PR here as well.
Component(s)
C++