Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly
Sign up
Appearance settings

Commit e3cdc8b

Browse files
committed
Switch Quickwit bootstrap to REST and add tooling
Removed the bootstrap container and provide REST-based bootstrap/validation tools and docs. Added scripts/bootstrap_quickwit_manual.sh, scripts/validate_quickwit.sh and updated scripts/quickwit_bootstrap.sh. Added docs/QUICKWIT_COMMANDS.md Also tweaked docker-compose (platform linux/amd64, expose 7282, use /health/readyz healthcheck), updated Quickwit config (data_dir, peer_seeds, Jaeger endpoint) and adjusted logs index default search fields.
1 parent cadeea0 commit e3cdc8b

File tree

7 files changed

+572
-53
lines changed

7 files changed

+572
-53
lines changed

‎docker-compose.observability.yml‎

Lines changed: 2 additions & 29 deletions
Original file line numberDiff line numberDiff line change
@@ -8,6 +8,7 @@ services:
88
ports:
99
- "7280:7280"
1010
- "7281:7281"
11+
- "7282:7282"
1112
environment:
1213
QUICKWIT_CONFIG: /quickwit/config/quickwit.yaml
1314
QW_ENABLE_OPENTELEMETRY_OTLP_EXPORTER: "true"
@@ -21,40 +22,12 @@ services:
2122
depends_on:
2223
- minio
2324
healthcheck:
24-
test: ["CMD", "curl", "-f", "http://localhost:7280/api/v1/health"]
25+
test: ["CMD", "curl", "-f", "http://localhost:7280/health/readyz"]
2526
interval: 30s
2627
timeout: 10s
2728
retries: 3
2829
start_period: 40s
2930

30-
quickwit-bootstrap:
31-
image: quickwit/quickwit:0.8.2
32-
container_name: ruxlog-quickwit-bootstrap
33-
volumes:
34-
- ./observability/quickwit/config:/quickwit/config
35-
networks:
36-
- network
37-
depends_on:
38-
quickwit:
39-
condition: service_healthy
40-
command: >
41-
sh -c "
42-
echo 'Waiting for Quickwit to be ready...';
43-
sleep 5;
44-
45-
for index_file in /quickwit/config/indexes/*.yaml; do
46-
if [ -f \"$$index_file\" ]; then
47-
echo \"Creating index from $$index_file\";
48-
quickwit --config /quickwit/config/quickwit.yaml index create \
49-
--endpoint http://quickwit:7280 \
50-
--index-config \"$$index_file\" 2>&1 | grep -v 'already exists' || true;
51-
fi;
52-
done;
53-
54-
echo 'Quickwit indexes bootstrap completed.';
55-
"
56-
restart: "no"
57-
5831
minio:
5932
image: minio/minio:RELEASE.2024年10月29日T16-01-48Z
6033
container_name: ruxlog-minio

‎docs/QUICKWIT_COMMANDS.md‎

Lines changed: 316 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,316 @@
1+
# Quickwit Setup Commands
2+
3+
## Quick Start
4+
5+
### 1. Start Quickwit (without bootstrap container)
6+
```bash
7+
docker compose -f docker-compose.observability.yml up -d quickwit minio
8+
```
9+
10+
### 2. Validate Setup
11+
```bash
12+
./scripts/validate_quickwit.sh
13+
```
14+
15+
### 3. Bootstrap Indexes Manually
16+
```bash
17+
./scripts/bootstrap_quickwit_manual.sh
18+
```
19+
20+
---
21+
22+
## Detailed Commands
23+
24+
### Validation Checks
25+
26+
**Run full validation suite:**
27+
```bash
28+
./scripts/validate_quickwit.sh
29+
```
30+
31+
This checks:
32+
- ✓ Container is running
33+
- ✓ Quickwit binary exists
34+
- ✓ Config files are present
35+
- ✓ Index configs are available
36+
- ✓ API is healthy
37+
- ✓ Data directory exists
38+
- ✓ Indexes are created
39+
- ✓ MinIO connectivity
40+
- ✓ Bootstrap script exists
41+
42+
**Manual checks:**
43+
44+
```bash
45+
# Check if container is running
46+
docker ps | grep ruxlog-quickwit
47+
48+
# Check Quickwit version inside container
49+
docker exec ruxlog-quickwit quickwit --version
50+
51+
# Check if config file exists
52+
docker exec ruxlog-quickwit test -f /quickwit/config/quickwit.yaml && echo "Config exists" || echo "Config missing"
53+
54+
# List index config files
55+
docker exec ruxlog-quickwit ls -la /quickwit/config/indexes/
56+
57+
# Check API health
58+
docker exec ruxlog-quickwit curl -f http://localhost:7280/api/v1/health
59+
60+
# Check MinIO connectivity
61+
docker exec ruxlog-quickwit curl -f http://minio:9000/minio/health/live
62+
63+
# View container logs
64+
docker logs ruxlog-quickwit --tail 50
65+
```
66+
67+
---
68+
69+
## Bootstrap Commands
70+
71+
### Automatic Bootstrap (Recommended)
72+
```bash
73+
./scripts/bootstrap_quickwit_manual.sh
74+
```
75+
76+
### Manual Bootstrap via docker exec
77+
```bash
78+
# Execute bootstrap script inside container
79+
docker exec ruxlog-quickwit sh -c '
80+
for index_file in /quickwit/config/indexes/*.yaml; do
81+
if [ -f "$index_file" ]; then
82+
echo "Creating index from $index_file"
83+
quickwit --config /quickwit/config/quickwit.yaml index create \
84+
--index-config "$index_file" 2>&1 | grep -v "already exists" || true
85+
fi
86+
done
87+
'
88+
```
89+
90+
### Create Individual Index
91+
```bash
92+
# Create logs index
93+
docker exec ruxlog-quickwit quickwit --config /quickwit/config/quickwit.yaml \
94+
index create --index-config /quickwit/config/indexes/logs.yaml
95+
96+
# Create traces index
97+
docker exec ruxlog-quickwit quickwit --config /quickwit/config/quickwit.yaml \
98+
index create --index-config /quickwit/config/indexes/traces.yaml
99+
100+
# Create metrics index
101+
docker exec ruxlog-quickwit quickwit --config /quickwit/config/quickwit.yaml \
102+
index create --index-config /quickwit/config/indexes/metrics.yaml
103+
```
104+
105+
---
106+
107+
## Index Management Commands
108+
109+
### List All Indexes
110+
```bash
111+
docker exec ruxlog-quickwit quickwit --config /quickwit/config/quickwit.yaml index list
112+
```
113+
114+
### Describe Specific Index
115+
```bash
116+
# Logs index
117+
docker exec ruxlog-quickwit quickwit --config /quickwit/config/quickwit.yaml \
118+
index describe --index ruxlog-logs
119+
120+
# Traces index
121+
docker exec ruxlog-quickwit quickwit --config /quickwit/config/quickwit.yaml \
122+
index describe --index ruxlog-traces
123+
124+
# Metrics index
125+
docker exec ruxlog-quickwit quickwit --config /quickwit/config/quickwit.yaml \
126+
index describe --index ruxlog-metrics
127+
```
128+
129+
### Delete Index (if needed)
130+
```bash
131+
docker exec ruxlog-quickwit quickwit --config /quickwit/config/quickwit.yaml \
132+
index delete --index ruxlog-logs --yes
133+
```
134+
135+
---
136+
137+
## Query & Search Commands
138+
139+
### Search via CLI
140+
```bash
141+
# Search logs
142+
docker exec ruxlog-quickwit quickwit --config /quickwit/config/quickwit.yaml \
143+
index search --index ruxlog-logs --query "*"
144+
145+
# Search with filter
146+
docker exec ruxlog-quickwit quickwit --config /quickwit/config/quickwit.yaml \
147+
index search --index ruxlog-logs --query "level:error"
148+
```
149+
150+
### Search via API
151+
```bash
152+
# Search logs API
153+
curl 'http://localhost:7280/api/v1/ruxlog-logs/search?query=*'
154+
155+
# Search with filter
156+
curl 'http://localhost:7280/api/v1/ruxlog-logs/search?query=level:error'
157+
158+
# Search traces
159+
curl 'http://localhost:7280/api/v1/ruxlog-traces/search?query=*'
160+
```
161+
162+
---
163+
164+
## Container Management
165+
166+
### Start Services
167+
```bash
168+
# Start everything
169+
docker compose -f docker-compose.observability.yml up -d
170+
171+
# Start only Quickwit and MinIO
172+
docker compose -f docker-compose.observability.yml up -d quickwit minio
173+
```
174+
175+
### Stop Services
176+
```bash
177+
# Stop all
178+
docker compose -f docker-compose.observability.yml down
179+
180+
# Stop but keep volumes
181+
docker compose -f docker-compose.observability.yml stop
182+
183+
# Stop and remove volumes (CAUTION: deletes all data)
184+
docker compose -f docker-compose.observability.yml down -v
185+
```
186+
187+
### Restart Quickwit
188+
```bash
189+
docker restart ruxlog-quickwit
190+
```
191+
192+
### View Logs
193+
```bash
194+
# Follow logs
195+
docker logs -f ruxlog-quickwit
196+
197+
# Last 50 lines
198+
docker logs ruxlog-quickwit --tail 50
199+
200+
# Logs since 10 minutes ago
201+
docker logs ruxlog-quickwit --since 10m
202+
```
203+
204+
### Interactive Shell
205+
```bash
206+
# Enter container shell
207+
docker exec -it ruxlog-quickwit sh
208+
209+
# Run commands interactively
210+
docker exec -it ruxlog-quickwit quickwit --help
211+
```
212+
213+
---
214+
215+
## Troubleshooting
216+
217+
### Check if all binaries are available
218+
```bash
219+
# Quickwit binary
220+
docker exec ruxlog-quickwit which quickwit
221+
222+
# Curl (for health checks)
223+
docker exec ruxlog-quickwit which curl
224+
225+
# Test basic commands
226+
docker exec ruxlog-quickwit sh -c 'ls -la /quickwit/config && echo "Config dir OK"'
227+
```
228+
229+
### Verify index files are mounted correctly
230+
```bash
231+
docker exec ruxlog-quickwit sh -c 'cat /quickwit/config/indexes/logs.yaml | head -20'
232+
```
233+
234+
### Check network connectivity
235+
```bash
236+
# From Quickwit to MinIO
237+
docker exec ruxlog-quickwit ping -c 3 minio
238+
239+
# Check if MinIO port is accessible
240+
docker exec ruxlog-quickwit nc -zv minio 9000
241+
```
242+
243+
### Reset everything (clean slate)
244+
```bash
245+
# Stop all services
246+
docker compose -f docker-compose.observability.yml down -v
247+
248+
# Remove any orphaned containers
249+
docker compose -f docker-compose.observability.yml rm -f
250+
251+
# Start fresh
252+
docker compose -f docker-compose.observability.yml up -d quickwit minio
253+
254+
# Wait for startup
255+
sleep 10
256+
257+
# Validate
258+
./scripts/validate_quickwit.sh
259+
260+
# Bootstrap
261+
./scripts/bootstrap_quickwit_manual.sh
262+
```
263+
264+
---
265+
266+
## Web Interfaces & Endpoints
267+
268+
- **Quickwit UI**: http://localhost:7280/ui
269+
- **API Playground**: http://localhost:7280/ui/api-playground
270+
- **Quickwit API**: http://localhost:7280/api/v1
271+
- **Liveness Check**: http://localhost:7280/health/livez
272+
- **Readiness Check**: http://localhost:7280/health/readyz
273+
- **Version Info**: http://localhost:7280/api/v1/version
274+
- **MinIO Console**: http://localhost:9001 (credentials: quickwit/quickwit-secret)
275+
276+
---
277+
278+
## Common Issues
279+
280+
### "Config file not found"
281+
```bash
282+
# Check if volume is mounted
283+
docker inspect ruxlog-quickwit | grep -A 10 Mounts
284+
285+
# Verify files exist on host
286+
ls -la observability/quickwit/config/
287+
```
288+
289+
### "Index already exists"
290+
This is normal on re-runs. The script handles this gracefully.
291+
292+
### "MinIO connection failed"
293+
```bash
294+
# Ensure MinIO is running
295+
docker ps | grep minio
296+
297+
# Check if bucket exists (optional, Quickwit creates it automatically)
298+
docker exec ruxlog-quickwit curl -I http://minio:9000/quickwit
299+
```
300+
301+
### "API not responding"
302+
```bash
303+
# Check if service is running
304+
docker exec ruxlog-quickwit ps aux | grep quickwit
305+
306+
# Check logs for errors
307+
docker logs ruxlog-quickwit --tail 100
308+
```
309+
310+
---
311+
312+
## Configuration Files Location
313+
314+
- Main config: `observability/quickwit/config/quickwit.yaml`
315+
- Index configs: `observability/quickwit/config/indexes/*.yaml`
316+
- Scripts: `scripts/validate_quickwit.sh`, `scripts/bootstrap_quickwit_manual.sh`

‎observability/quickwit/config/indexes/logs.yaml‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -58,7 +58,7 @@ doc_mapping:
5858
tokenizer: default
5959

6060
search_settings:
61-
default_search_fields: [message, level, service_name, span_name, event_type]
61+
default_search_fields: [message, level, service_name, http_route, scope]
6262

6363
indexing_settings:
6464
commit_timeout_secs: 30

0 commit comments

Comments
(0)

AltStyle によって変換されたページ (->オリジナル) /