116 questions
- Bountied 0
- Unanswered
- Frequent
- Score
- Trending
- Week
- Month
- Unanswered (my tags)
2
votes
1
answer
120
views
Zstandard in Python 3.14: compress a file in streaming and multi-threaded mode
It is very nice that Python 3.14 added Zstandard to the standard library. Please see here for its package documentation. I can use it with a rudimentary syntax
from compression import zstd
from ...
0
votes
0
answers
107
views
Decompress .zstd file using typescript
I'm trying to programmatically decompress a .zst file using typescript. This is the code I am using:
import { createReadStream, createWriteStream } from 'node:fs';
import { createZstdDecompress } from ...
0
votes
1
answer
124
views
KafkaJS consumer fails to decompress ZSTD messages (decompress returns null for specific topic)
I'm working with KafkaJS and trying to consume messages compressed with ZSTD.
I registered the Zstd decompression handler like this:
ZstdCodec.run((zstd) => {
const Simple = new zstd.Simple();
...
0
votes
2
answers
241
views
How to compile zstd library with Visual Studio 2022?
I need to compile zstd library with Visual Studio 2022. But I found that zstd official github page does not support the Visual Studio 2022 version; the latest supported version is Visual Studio 2017.
...
1
vote
0
answers
125
views
Unable to import the compressed ZSTD GeoTiff to PostGIS in Python
I would like to import the compressed ZSTD GeoTiff to PostGIS with the following commands (below) but I get this error: RASTER_fromGDALRaster: Could not open bytea with GDAL. Check that the bytea is ...
3
votes
1
answer
162
views
Streaming the reading of a very large compressed JSON file in Python
I have a very large (too large to hold in RAM) .json.zstd file that I built iteratively with a generator of texts data_chunks.
A completely toy example of such generator is:
[{"text": "...
0
votes
1
answer
240
views
zstd: Unknown frame descriptor
I have a binary file that appears to be originally generated by a Java hashmap serialization. The majority of the file seems to be a zstandard-compressed archive:
00000000 28 B5 2F FD:00 58 9D 21|00 ...
4
votes
1
answer
269
views
Pandas zstd compression level 10 better than Apache Spark's
For the same compression level of 10 using zstd with Parquet, I get significantly better performance in Pandas than from Apache Spark. The following files, for example, were first generated using ...
0
votes
0
answers
50
views
How to further decrease financial data size?
I’ve been working on compressing tick data and have made some progress, but I’m looking for ways to further optimize file sizes. Currently, I use delta encoding followed by saving the data in Parquet ...
1
vote
0
answers
116
views
Unsupported OS/arch, cannot find /darwin/aarch64/libzstd-jni-1.5.5-1.dylib or load zstd-jni-1.5.5-1 from system libraries
Graalvm is not able to load libzstd-jni, getting error while running executable generated from graalvm. Able to bring server up, however getting below error at run time.
commend used for generating ...
-1
votes
2
answers
527
views
Python Requests Disable zstd Encoding [closed]
My Synology DS418play recently updated to the latest version of DSM7 that is available. I noticed that a lot of the python scripts that I had have started returning weirdly encoded data. Here is an ...
0
votes
0
answers
115
views
use fallback for zstd with mod_deflate
I just realized the (shared hosting) server I'm using has zstd encoding enabled (Accept-Encoding: gzip, deflate, br, zstd). But since zstd compression is not yet supported in Safari I would like to ...
0
votes
0
answers
82
views
Using airlift java library to uncompress a file
I am using the https://github.com/airlift/aircompressor library to uncompress a byte array of data using the zstd compression algorithm. The documentation of the project (which is actively maintained) ...
1
vote
2
answers
171
views
AttributeError: 'zstd.ZstdDecompressionReader' object has no attribute 'fileno'
I need to run a subprocess pipeline that uses zstandard files (too large to fit in memory) both as their input and output. Consider the following example:
import subprocess
import zstandard
with ...
0
votes
0
answers
593
views
How to link C++ project statically to zstd.lib?
I have a VS2022 C++ project that uses zstd. Previously, it used zstd v1.5.2; I changed this to v1.5.6 (the current version). Now when I attempt to run the project, I get an error "The code ...