All Questions
Tagged with kafka-cluster or apache-kafka
33,420 questions
- Bountied 0
- Unanswered
- Frequent
- Score
- Trending
- Week
- Month
- Unanswered (my tags)
-3
votes
0
answers
42
views
Corrupted Kafka state after rsync: Record for partition offset 1202515169851212184 is invalid, Invalid magic found in record: 53 [closed]
I have three Kafka instances running on three hosts. I needed to move these Kafka instances to three new larger hosts, so I rsynced the data to the new hosts (while Kafka was down), then started up ...
0
votes
0
answers
19
views
How can I make Schema Registry connect successfully to a Strimzi Kafka cluster using PLAINTEXT? [closed]
Why is Schema Registry unable to connect to the Strimzi Kafka cluster on port 9092 (PLAINTEXT)?
Is there something special about how Strimzi advertises its broker listeners that prevents Schema ...
1
vote
1
answer
104
views
Possible Spark and Kafka version incompatibility in Docker?
I'm running a Spark Streaming notebook from my Data Engineering course and ran into an error which I suspect is related to Spark and Kafka version incompatibility in Docker.
game_monitoring.ipynb
...
-5
votes
0
answers
43
views
How to produce and consume messages with Apache Kafka using Java? [closed]
I’m learning Apache Kafka and I want to create a simple example in Java where:
A producer
sends a message to a Kafka topic
A consumer reads it from the same topic I’ve already installed
Kafka and ...
0
votes
0
answers
26
views
How to add a broker after a very long downtime back to kafka cluster?
I have a kafka cluster running v2.3.0 with 27 brokers. The max retention period for our topics is 7 days. Now, 2 of our brokers went down on seperate occasions due to disk failure.
I tried adding the ...
0
votes
1
answer
31
views
how to verify Confluent Kafka JsonSchema Serialized message?
I’m trying to verify whether messages produced using Confluent’s KafkaJsonSchemaSerializer are correctly serialized in the expected Confluent wire format (i.e. a "magic byte" followed by a 4-byte ...
0
votes
2
answers
29
views
Spring Cloud Stream: Application not consuming from second topic with multiple input bindings
I'm trying to configure a single consumer function to handle messages from two different Kafka topics using Spring Cloud Stream. The application only consumes from the first topic (consumeMessage-in-0)...
0
votes
0
answers
52
views
springboot integration Kafka @RetryableTopic is not effective
describe
springboot integration Kafka @RetryableTopic is not effective,I cannot control retry count; it is retrying continuously.There are two types of error logs continuously printed in the console: ...
0
votes
0
answers
26
views
Clients unable to authenticate against Apache/Kafka container with existing user accounts
I inherited a working Docker Compose tech stack that used Bitnami/Kafka and a recompiled VerneMQ broker. Since Bitnami began deprecating some of its catalogue, we made a decision to switch to Apache's ...
0
votes
0
answers
31
views
about EMQX Enterprise 6 Action Sink to Kafka cluster
I am a newbie in EMQX architecture. Recently, for test purposes, I deployed an EMQX enterprise 6 single node, with Kafka producer connector integration (Action Sink).
The compatition matrix of the ...
0
votes
0
answers
21
views
Kafka cluster with `default.replication.factor=3` and `min.insync.replicas=2`. A producer with `acks=all` and 1 replica fails. What's the effect?
A Kafka cluster is configured with default.replication.factor=3 and min.insync.replicas=2. A producer with acks=all sends a message to a topic. The leader writes the message successfully, but one of ...
0
votes
0
answers
70
views
Kafka Streams main consumer fetch rate stays low after GlobalKTable (RocksDB) restore
I’m running a Kafka Streams application (tested with versions 3.9.1 and 4.1.0) that uses a GlobalKTable backed by RocksDB. There are multiple instances of the application, each with 4 stream threads.
...
0
votes
1
answer
39
views
Capture Oracle metadata via Debezium connector
I'm sourcing Oracle data into Kafka topics using Debezium Oracle connector.
I have a source Oracle table, as below:
CREATE TABLE MYDATA.MY_USER
(
USER_ID RAW(16) DEFAULT SYS_GUID() NOT ...
1
vote
0
answers
44
views
Assertion failed: (self->u.Consumer.rkqu) for confluent python kafka
Try the confluent kafka base example
from confluent_kafka import Consumer
c = Consumer({
'bootstrap.servers': 'localhost:9092',
'group.id': None,
'auto.offset.reset': 'earliest'
})
c....
0
votes
0
answers
19
views
WARN Exception while shutting down acceptor. (org.apache.zookeeper.server.quorum.Leader) java.net.SocketException: Socket closed
I have a strimzi kafka installation deployed in Kubernetes using the below command:
helm upgrade --install kafka --set replicaCount=1 --set persistence.storageClass=kafka-storage .
I get exception in ...