177 questions
- Bountied 0
- Unanswered
- Frequent
- Score
- Trending
- Week
- Month
- Unanswered (my tags)
0
votes
0
answers
86
views
python-lz4: how to configure the number of threads?
I have Python script using Python binding lz4 of the original lz4:
import lz4.frame
import shutil
input_file = r"E:\Personal Projects\tmp\chunk_0.ndjson"
output_file = r"E:\Personal ...
Akira's user avatar
- 2,918
0
votes
2
answers
124
views
Scala Spark decompress lz4 csv
Is there any way to decompress csv lz4 files by Spark?
I tried following approaches:
First:
sparkSession.read
.option("delimiter", ",")
.option("compression", "...
0
votes
0
answers
286
views
libarchive & iOS issue
I've got an issue with libarchive extraction on iOS system, it crashes while unpacking it, happens not for all users, only a few ones.
int copy_data(struct archive* ar, struct archive* aw)
{
const ...
1
vote
0
answers
290
views
K4os.Compression.LZ4: Could not load file or assembly 'System.Memory, Version=4.0.1.2, ...'
I'm at a loss.
I am on .NET Framework 4.8.
I installed K4os.Compression.LZ4 1.3.8 via Nuget and use LZ4Pickler.Pickle().
My required dependencies according to https://www.nuget.org/packages/K4os....
0
votes
0
answers
383
views
java.lang.NoSuchFieldError: LZ4 : spark.sql()
I am trying to run one of my methods which has -
spark.sql("DROP TABLE IF EXISTS " + dbNameString + "." + tableNameString)
When I am running the method, the code breaks on the ...
4
votes
1
answer
2k
views
Setting the column COMPRESSION to LZ4 does not seem to work?
Working with Postgres 16.2 for Windows, I am attempting to set LZ4 compression on a column within one of our tables.
message text COMPRESSION lz4,
When I perform the following query:
SELECT attname,...
0
votes
1
answer
140
views
How to create a Buffer in JS to decompress an lz4 file?
I have a file in s3 that is compressed using lz4 (I tried bz2 too) compression and served from an API endpoint as an application/octet-stream. The API fetches the compressed file from S3 and serves ...
0
votes
1
answer
67
views
Is calculating the max bounds in a compression algorithm necessary?
I've been using a few compression algorithms and before you compress you're apparently supposed to get the maximum bound of the possible compression result with calls such as:
ZSTD_compressBound(...
0
votes
0
answers
152
views
LZ4 deprecated function being called within a deprecated function inside implementation
In LZ4 compression library in a header file we have a function definition which is deprecated:
/*! Obsolete decompression functions (since v1.8.0) */
LZ4_DEPRECATED("use LZ4_decompress_fast() ...
0
votes
1
answer
210
views
how to decompress and write to disk lz4 compressed OS image
I am trying to fetch an OS image with pycurl and write the decompressed data to disk. With gzip it is straight forward, only with lz4 formats I face issues, it seems the write_lz4(buf) decompresses ...
0
votes
1
answer
130
views
"corrupted size vs. prev_size" when using LZ4
I'm using JSON to pass messages between a server and a client, both written in C++. To get a performance boost, I'm evaluating compression, as well as binary formats (like MessagePack). I started with ...
2
votes
1
answer
2k
views
Export Clickhouse table to s3 without using LIMIT
I'm trying to export Clickhouse table to s3 following official docs and using command:
INSERT INTO FUNCTION
s3(
'https://my-bucket.s3.amazonaws.com/files/table.csv.lz4',
'aws_key',
...
0
votes
1
answer
219
views
Why is my InputStream not reading all bytes of a specified file? (CRC32 archive validation)
I'm attempting to use a CheckedOutputStream/CheckedInputStream to calculate a CRC32 checksum for a tar archive to ensure data integrity in a volatile system with frequent total electronic power loss. ...
0
votes
1
answer
56
views
When Lz4 compression,process short length data,iOS system approach compression_encode_buffer,Compression failure
The official iOS method,compression_encode_buffer,unable to process short data.
For example
data {length = 17, bytes = 0x00000000020000000100000000ffe2ff61}
Use compression_encode_buffer to ...
0
votes
1
answer
936
views
Having trouble satisfying a dependency during compilation
I apologize in advance, I am very new to using linux. I am attempting to install the proxmark3 client on Pop!OS and everything goes well up until I try to make all in the github repo. This is what I ...