Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly
Sign up
Appearance settings

Commit 9af4bbb

Browse files
author
rokh-conduktor
authored
add platform (#118)
1 parent 9a95e0f commit 9af4bbb

File tree

4 files changed

+144
-26
lines changed

4 files changed

+144
-26
lines changed

‎README.md‎

Lines changed: 24 additions & 19 deletions
Original file line numberDiff line numberDiff line change
@@ -15,6 +15,7 @@ This replicates as well as possible real deployment configurations, where you ha
1515

1616
## Stack version
1717

18+
- Conduktor Platform: 1.0.2
1819
- Zookeeper version: 3.6.3 (Confluent 7.2.1)
1920
- Kafka version: 3.2.0 (Confluent 7.2.1)
2021
- Kafka Schema Registry: Confluent 7.2.1
@@ -46,6 +47,29 @@ If you want to downgrade confluent platform version, there are two ways:
4647
1. Add `platform: linux/amd64`. It will work as docker is able to emulate AMD64 instructions.
4748
2. Previous versions have been [built](https://github.com/arm64-compat/confluent-platform) for ARM64 by the community. If you want to use it, just change the image in the corresponding yml. Since it is a not an official image, use it at your own risks.
4849

50+
## Full stack
51+
52+
To ease you journey with kafka just connect to [localhost:8080](http://localhost:8080/)
53+
54+
login: `login@admin.io`
55+
password: `admin`
56+
57+
- Conduktor-platform: `$DOCKER_HOST_IP:8080`
58+
- Single Zookeeper: `$DOCKER_HOST_IP:2181`
59+
- Single Kafka: `$DOCKER_HOST_IP:9092`
60+
- Kafka Schema Registry: `$DOCKER_HOST_IP:8081`
61+
- Kafka Rest Proxy: `$DOCKER_HOST_IP:8082`
62+
- Kafka Connect: `$DOCKER_HOST_IP:8083`
63+
- KSQL Server: `$DOCKER_HOST_IP:8088`
64+
- (experimental) JMX port at `$DOCKER_HOST_IP:9001`
65+
66+
Run with:
67+
```
68+
docker-compose -f full-stack.yml up
69+
docker-compose -f full-stack.yml down
70+
```
71+
72+
4973
## Single Zookeeper / Single Kafka
5074

5175
This configuration fits most development requirements.
@@ -102,25 +126,6 @@ docker-compose -f zk-multiple-kafka-multiple.yml up
102126
docker-compose -f zk-multiple-kafka-multiple.yml down
103127
```
104128

105-
## Full stack
106-
107-
Need a UI? We recommend using [Conduktor](https://conduktor.io) as your tool to bring a unified UI to all these components
108-
109-
- Single Zookeeper: `$DOCKER_HOST_IP:2181`
110-
- Single Kafka: `$DOCKER_HOST_IP:9092`
111-
- Kafka Schema Registry: `$DOCKER_HOST_IP:8081`
112-
- Kafka Rest Proxy: `$DOCKER_HOST_IP:8082`
113-
- Kafka Connect: `$DOCKER_HOST_IP:8083`
114-
- KSQL Server: `$DOCKER_HOST_IP:8088`
115-
- Zoonavigator Web: `$DOCKER_HOST_IP:8004`
116-
- (experimental) JMX port at `$DOCKER_HOST_IP:9999`
117-
118-
Run with:
119-
```
120-
docker-compose -f full-stack.yml up
121-
docker-compose -f full-stack.yml down
122-
```
123-
124129
# FAQ
125130

126131
## Kafka

‎full-stack.yml‎

Lines changed: 25 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -30,7 +30,7 @@ services:
3030
KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR: 1
3131
KAFKA_TRANSACTION_STATE_LOG_REPLICATION_FACTOR: 1
3232
KAFKA_TRANSACTION_STATE_LOG_MIN_ISR: 1
33-
KAFKA_JMX_PORT: 9999
33+
KAFKA_JMX_PORT: 9001
3434
KAFKA_JMX_HOSTNAME: ${DOCKER_HOST_IP:-127.0.0.1}
3535
KAFKA_AUTHORIZER_CLASS_NAME: kafka.security.authorizer.AclAuthorizer
3636
KAFKA_ALLOW_EVERYONE_IF_NO_ACL_FOUND: "true"
@@ -126,11 +126,29 @@ services:
126126
- zoo1
127127
- kafka1
128128

129-
zoonavigator:
130-
image: elkozmon/zoonavigator:1.1.1
131-
container_name: zoonavigator
129+
conduktor-platform:
130+
image: conduktor/conduktor-platform:1.0.2
132131
ports:
133-
- "8004:8000"
132+
- 8080:80
133+
volumes:
134+
- conduktor_data:/var/conduktor
135+
- type: bind
136+
source: "./platform-config.yaml"
137+
target: /opt/conduktor/platform-config.yaml
138+
read_only: true
139+
depends_on:
140+
- kafka1
141+
- kafka-schema-registry
142+
- kafka-connect
134143
environment:
135-
HTTP_PORT: 8000
136-
AUTO_CONNECT_CONNECTION_STRING: zoo1:2181
144+
CDK_IN_CONF_FILE: /opt/conduktor/platform-config.yaml
145+
KAFKA_BOOTSTRAP_SERVER: PLAINTEXT://kafka1:19092
146+
SR_SERVER: "http://kafka-schema-registry:8081"
147+
ORGANISATION_NAME: "default"
148+
ADMIN_EMAIL: "admin@admin.io"
149+
ADMIN_PSW: "admin"
150+
151+
152+
volumes:
153+
conduktor_data: {}
154+

‎jmx-exporter.yml‎

Lines changed: 70 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,70 @@
1+
lowercaseOutputName: true
2+
lowercaseOutputLabelNames: true
3+
rules:
4+
# Special cases and very specific rules
5+
- pattern : kafka.server<type=(.+), name=(.+), clientId=(.+), topic=(.+), partition=(.*)><>Value
6+
name: kafka_server_1ドル_2ドル
7+
type: GAUGE
8+
labels:
9+
clientId: "3ドル"
10+
topic: "4ドル"
11+
partition: "5ドル"
12+
- pattern : kafka.server<type=(.+), name=(.+), clientId=(.+), brokerHost=(.+), brokerPort=(.+)><>Value
13+
name: kafka_server_1ドル_2ドル
14+
type: GAUGE
15+
labels:
16+
clientId: "3ドル"
17+
broker: "4ドル:5ドル"
18+
19+
- pattern : kafka.server<type=KafkaRequestHandlerPool, name=RequestHandlerAvgIdlePercent><>OneMinuteRate
20+
name: kafka_server_kafkarequesthandlerpool_requesthandleravgidlepercent_total
21+
type: GAUGE
22+
23+
- pattern : kafka.server<type=socket-server-metrics, clientSoftwareName=(.+), clientSoftwareVersion=(.+), listener=(.+), networkProcessor=(.+)><>connections
24+
name: kafka_server_socketservermetrics_connections
25+
type: GAUGE
26+
labels:
27+
client_software_name: "1ドル"
28+
client_software_version: "2ドル"
29+
listener: "3ドル"
30+
network_processor: "4ドル"
31+
32+
- pattern : 'kafka.server<type=socket-server-metrics, listener=(.+), networkProcessor=(.+)><>(.+):'
33+
name: kafka_server_socketservermetrics_3ドル
34+
type: GAUGE
35+
labels:
36+
listener: "1ドル"
37+
network_processor: "2ドル"
38+
39+
# Count and Value
40+
- pattern: kafka.(.+)<type=(.+), name=(.+), (.+)=(.+), (.+)=(.+)><>(Count|Value)
41+
name: kafka_1ドル_2ドル_3ドル
42+
labels:
43+
"4ドル": "5ドル"
44+
"6ドル": "7ドル"
45+
- pattern: kafka.(.+)<type=(.+), name=(.+), (.+)=(.+)><>(Count|Value)
46+
name: kafka_1ドル_2ドル_3ドル
47+
labels:
48+
"4ドル": "5ドル"
49+
- pattern: kafka.(.+)<type=(.+), name=(.+)><>(Count|Value)
50+
name: kafka_1ドル_2ドル_3ドル
51+
52+
# Percentile
53+
- pattern: kafka.(.+)<type=(.+), name=(.+), (.+)=(.*), (.+)=(.+)><>(\d+)thPercentile
54+
name: kafka_1ドル_2ドル_3ドル
55+
type: GAUGE
56+
labels:
57+
"4ドル": "5ドル"
58+
"6ドル": "7ドル"
59+
quantile: "0.8ドル"
60+
- pattern: kafka.(.+)<type=(.+), name=(.+), (.+)=(.*)><>(\d+)thPercentile
61+
name: kafka_1ドル_2ドル_3ドル
62+
type: GAUGE
63+
labels:
64+
"4ドル": "5ドル"
65+
quantile: "0.6ドル"
66+
- pattern: kafka.(.+)<type=(.+), name=(.+)><>(\d+)thPercentile
67+
name: kafka_1ドル_2ドル_3ドル
68+
type: GAUGE
69+
labels:
70+
quantile: "0.4ドル"

‎platform-config.yaml‎

Lines changed: 25 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,25 @@
1+
organization:
2+
name: "${ORGANISATION_NAME}"
3+
4+
clusters:
5+
- id: default
6+
name: My Local Kafka Cluster
7+
color: "#0013E7"
8+
ignoreUntrustedCertificate: false
9+
bootstrapServers: "$KAFKA_BOOTSTRAP_SERVER"
10+
properties:
11+
schemaRegistry:
12+
url: "$SR_SERVER"
13+
ignoreUntrustedCertificate: false
14+
properties:
15+
labels: {}
16+
kafkaConnects:
17+
- url: http://kafka-connect:8083
18+
name: full stack kafka connect
19+
20+
auth:
21+
demo-users:
22+
- email: "${ADMIN_EMAIL}"
23+
password: "${ADMIN_PSW}"
24+
groups:
25+
- ADMIN

0 commit comments

Comments
(0)

AltStyle によって変換されたページ (->オリジナル) /