508 questions
- Bountied 0
- Unanswered
- Frequent
- Score
- Trending
- Week
- Month
- Unanswered (my tags)
Advice
0
votes
0
replies
31
views
Sorted Runs vs SST Files in RocksDB Universal Compaction
This documentation here states that in Universal Compaction with num_levels=1 then the entire database can be written into a single SST file.
I understand that the entire database will be written into ...
0
votes
0
answers
77
views
Kafka Streams main consumer fetch rate stays low after GlobalKTable (RocksDB) restore
I’m running a Kafka Streams application (tested with versions 3.9.1 and 4.1.0) that uses a GlobalKTable backed by RocksDB. There are multiple instances of the application, each with 4 stream threads.
...
0
votes
0
answers
48
views
Is the "tiered storage" feature ready for production?
I'm looking to implement a tiered storage solution with RocksDB, where hot data resides on fast storage (like SSDs) and cold data is moved to slower, more cost-effective storage (like HDDs).
I'm aware ...
0
votes
1
answer
41
views
Can a basic database be opened by a optimistictransactiondb or transactiondb in rocksdb?
I had created a base database with rocksdb::Open() and written some data to it.
Can I close it and open it with OptimisticTransactionDB::Open() for the transactional support? Will the existing data ...
0
votes
2
answers
338
views
RocksDB for efficient storage and retrieval of keys only (no values)
I have a use case requiring the reliable, durable storage, and efficient retrieval, of a large number of index entries (for which I've an application-specific serialization to bytes that preserves ...
0
votes
0
answers
107
views
Flink pipeline latency increases over time despite low processing time per task
I’m currently testing a Flink pipeline with the following architecture:
3 TaskManagers
Each TaskManager has 4 slots
The pipeline structure is:
Source → Map → KeyBy → KeyedProcessFunction → Sink
...
0
votes
1
answer
97
views
Efficiency of schema design for rocksdb - normalization and key compression considerations
I have a use case, for which I'm considering RocksDB. Part of the problem involves persistently storing values (with a meaningful ordering) associated with hierarchically structured labels. ...
0
votes
2
answers
81
views
infinispan 15.1.0, rocksdb jar not loaded at runtime
I am trying to explore rocksdb persistance for infinispan remote cache.
Below is my cache config.
<distributed-cache statistics="true" owners="1">\
<encoding media-type="application/x-...
0
votes
0
answers
30
views
Why is writePrepared faster than writeCommitted?
The writePrepared policy in RocksDB writes the batch to the memtable in prepare phase, while the writeCommitted policy writes the batch to the memtable in commit phase. It seems the total amount of ...
0
votes
0
answers
68
views
Why Flink States in rocksdb are not retaining from checkpoints after a runtime failure?
I am running a Flink job for a use case of joining Adwords info and user events. I need to store those ad specific events in state to join with Adwords data that arrives late.
After a failure , the ...
1
vote
0
answers
62
views
rocksdb 7.6.x get operation vs seek operation
I'm working with RocksDB 6.7.3, I don't have a chance to upgrade it.
Here is my logic:
GET by key
return value if value is null or value.length > 0
if value.length == 0, then iterate and try to ...
1
vote
0
answers
74
views
Flink RocksDB state backend taking too long to read
I have a flink 1.19 application that has KeyedProcessFunction with ValueState (Boolean) with TTL of 5 minutes, it is processing a stream of 35000 rps
I have setup RocksDB as flink's state backend:
...
0
votes
0
answers
45
views
learning rocksdb with librocksdb.a
I am learning rocksdb and would like to change rocksdb lib codes. I am running ubuntu on windows. librocksdb.a is linked in Makefile. If I make some changes in rocksdb lib codes, do I have to rebuild ...
0
votes
1
answer
347
views
why flink checkpoint size always growing with rocksdb
I have a Flink processing program that receives data from Kafka.
Then, after performing keyby, it sinks the data to Doris and Kafka.
Inside the keyby operation, I only use ValueState<T> to store ...
1
vote
0
answers
128
views
Spark Streaming app with RocksDB gets OOKilled on K8S
I'm running a Spark Structured Streaming application on K8S.
The application is deployed with spark-submit --master k8s://...
Using RocksDB state backend, configuring spark.kubernetes....