-
Couldn't load subscription status.
- Fork 244
fix: simplify event buffer #218
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
WalkthroughThis refactor simplifies the event buffering system by replacing a complex Lua-script-driven Redis implementation with a single-list approach, introduces centralized duration calculation as a helper function, updates ClickHouse client configuration for improved performance, and refactors service-layer queries to use inline SQL builders for consistency and maintainability. Changes
Sequence Diagram(s)sequenceDiagram
participant App
participant EventBuffer
participant Redis
participant ClickHouse
rect rgb(220, 240, 255)
Note over App,ClickHouse: Event Processing Flow (New)
App->>EventBuffer: addEvent(event)
EventBuffer->>Redis: push to single list
EventBuffer->>Redis: increment counter
App->>EventBuffer: processBuffer()
EventBuffer->>Redis: read all events from list
EventBuffer->>EventBuffer: parse & sort by creation_time
loop chunk by EVENT_BUFFER_CHUNK_SIZE
EventBuffer->>ClickHouse: insert batch with retry
ClickHouse-->>EventBuffer: success
end
EventBuffer->>App: emit "saved" events
EventBuffer->>Redis: trim/clear buffer list
end
Estimated code review effort🎯 4 (Complex) | ⏱️ ~45 minutes Multiple files with heterogeneous changes spanning buffering logic, configuration, SQL query construction, and API endpoints. Event buffer rewrite eliminates complex Lua scripts but introduces new ordering/chunking logic. Overview service refactor replaces previous query patterns with inline SQL builders. Changes are interdependent and require understanding how duration calculations, filtering, and data sources interact across layers. Poem
Pre-merge checks and finishing touches❌ Failed checks (1 warning)
✅ Passed checks (2 passed)
✨ Finishing touches
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Actionable comments posted: 5
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (2)
packages/db/src/buffers/event-buffer.ts (2)
48-96: Verify event publication timing and Redis operation success.The
publishEventcall at line 85 occurs aftermulti.exec()but doesn't verify the Redis operations succeeded. If the multi execution fails (e.g., Redis connection issue), the event is still published as "received", creating inconsistency between what's buffered and what's announced.Additionally, in bulk-add scenarios (when
_multiis provided),getBufferSize()at line 88 reads the buffer counter before the bulk multi is executed, returning stale data. This could prevent auto-flush when the buffer actually exceedsbatchSize.Consider checking the multi.exec() result before publishing:
if (!_multi) { - await multi.exec(); + const result = await multi.exec(); + if (!result || result.some(([err]) => err)) { + this.logger.error('Failed to execute Redis operations', { result }); + return; + } } await publishEvent('events', 'received', transformEvent(event));And reconsider buffer size checking in bulk scenarios:
- // Check buffer length using counter - const bufferLength = await this.getBufferSize(); - - if (bufferLength >= this.batchSize) { - await this.tryFlush(); - } + // Only check buffer size if not in bulk mode (counter would be stale) + if (!_multi) { + const bufferLength = await this.getBufferSize(); + if (bufferLength >= this.batchSize) { + await this.tryFlush(); + } + }
153-212: ClickHouse events table uses MergeTree (no deduplication)—atomicity issue is real.The concern is valid: ClickHouse
eventstable is defined withMergeTree(append-only, no deduplication), notReplacingMergeTree. WithORDER BY: [project_id, toDate(created_at), profile_id, name], theevent.idis not part of any deduplication key.If ClickHouse insertion succeeds but Redis trimming (line 200-204) fails, the same events will be reprocessed on the next flush and inserted as true duplicates. API-level deduplication (100ms lock) only protects against immediate duplicate HTTP requests—not buffer reprocessing after failures.
Consider marking events as "processing" before insertion, or wrap ClickHouse insert + Redis trim in a retry loop with exponential backoff to ensure atomicity.
🧹 Nitpick comments (8)
packages/db/src/services/event.service.ts (1)
573-574: Don’t log raw SQL in production.Gate behind an env flag or remove to avoid noisy logs and potential PII leakage.
- console.log('getSql() ----> ', getSql()); + if (process.env.DEBUG_SQL === '1') { + // eslint-disable-next-line no-console + console.log('getSql() ----> ', getSql()); + }packages/db/src/services/profile.service.ts (1)
43-45: Prefer counting sessions from the sessions table for consistency.Counting
session_startevents can drift; usesessionswithsign = 1.- sessionsCount AS ( - SELECT count(*) as sessions FROM ${TABLE_NAMES.events} WHERE name = 'session_start' AND profile_id = ${sqlstring.escape(profileId)} AND project_id = ${sqlstring.escape(projectId)} - ), + sessionsCount AS ( + SELECT countIf(sign = 1) as sessions + FROM ${TABLE_NAMES.sessions} + WHERE profile_id = ${sqlstring.escape(profileId)} + AND project_id = ${sqlstring.escape(projectId)} + ),packages/db/src/clickhouse/client.ts (1)
91-129: Broaden retryable error detection.Consider common network errors to reduce transient failures.
- if ( - error.message.includes('Connect') || - error.message.includes('socket hang up') || - error.message.includes('Timeout error') - ) { + const msg = String(error?.message || ''); + if ( + /Connect|socket hang up|Timeout error|ETIMEDOUT|ECONNRESET|EPIPE|ECONNREFUSED|ENOTFOUND/i.test( + msg, + ) + ) {packages/db/src/services/overview.service.ts (4)
99-125: Support ‘minute’ in toStartOfInterval.Add
toStartOfMinuteto avoid defaulting to day for minute bins.switch (interval) { + case 'minute': + return `toStartOfMinute(${field}${tzPart})`; case 'hour': return `toStartOfHour(${field}${tzPart})`;
555-570: Apply the same fix for the non-page-filter branch.Aligns logic and avoids including tombstones.
SELECT ${mode}_origin AS origin, ${mode}_path AS path, - round(avg(duration * sign) / 1000, 2) AS avg_duration, - round(sum(sign * is_bounce) * 100.0 / sum(sign), 2) AS bounce_rate, - sum(sign) AS sessions + round(avgIf(duration, duration > 0 AND sign > 0) / 1000, 2) AS avg_duration, + round(sumIf(is_bounce, sign = 1) * 100.0 / nullIf(sumIf(sign, sign = 1), 0), 2) AS bounce_rate, + sumIf(sign, sign = 1) AS sessions
480-481: Remove or gate SQL debug log.Avoid logging full queries in production.
- console.log('sql', sql); + if (process.env.DEBUG_SQL === '1') { + // eslint-disable-next-line no-console + console.log('sql', sql); + }
304-309: ROLLUP totals row: don’t assume first position.Find the rollup row via
WHERE date IS NULL(orisNull(date)), notres[0], to be robust across CH versions/orderings.If helpful, I can provide a small post-processing helper to split totals vs. series reliably.
packages/db/src/buffers/event-buffer.ts (1)
109-141: Reconsider the sessionId overload if it's not supported.The method provides an overload accepting
sessionId(lines 109-115), but the implementation always returnsnullfor session-based lookups with a comment stating it's not supported (lines 138-140). This creates a misleading API surface—callers might expect it to work based on the type signature.Consider either:
- Removing the
sessionIdoverload entirely if it's not planned for implementation- If it's a temporary limitation, add
@deprecatedJSDoc or mark it clearly in documentation- public async getLastScreenView({ - projectId, - sessionId, - }: { - projectId: string; - sessionId: string; - }): Promise<IServiceEvent | null>; public async getLastScreenView({ projectId, profileId, - sessionId, }: { projectId: string; profileId?: string; - sessionId?: string; }): Promise<IServiceEvent | null> {Note: Line 126 uses
getRedisCache()instead ofthis.redis—verify this is intentional or consolidate tothis.redisfor consistency.
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (8)
packages/db/src/buffers/event-buffer.test.ts(3 hunks)packages/db/src/buffers/event-buffer.ts(3 hunks)packages/db/src/clickhouse/client.ts(2 hunks)packages/db/src/services/event.service.ts(3 hunks)packages/db/src/services/overview.service.ts(7 hunks)packages/db/src/services/profile.service.ts(3 hunks)packages/trpc/src/routers/chart.ts(0 hunks)packages/trpc/src/routers/realtime.ts(4 hunks)
💤 Files with no reviewable changes (1)
- packages/trpc/src/routers/chart.ts
🧰 Additional context used
🧬 Code graph analysis (5)
packages/db/src/buffers/event-buffer.test.ts (2)
packages/db/src/clickhouse/client.ts (1)
ch(131-161)packages/db/src/buffers/event-buffer.ts (1)
EventBuffer(16-249)
packages/trpc/src/routers/realtime.ts (1)
packages/db/src/services/event.service.ts (1)
getDurationSql(1020-1021)
packages/db/src/buffers/event-buffer.ts (4)
packages/redis/redis.ts (3)
Redis(9-9)getRedisCache(66-72)getRedisPub(84-90)packages/db/src/services/event.service.ts (3)
IClickhouseEvent(68-102)IServiceEvent(145-182)transformEvent(104-138)packages/json/index.ts (1)
getSafeJson(3-9)packages/redis/publisher.ts (1)
publishEvent(28-36)
packages/db/src/services/profile.service.ts (1)
packages/db/src/clickhouse/client.ts (1)
TABLE_NAMES(48-60)
packages/db/src/services/overview.service.ts (3)
packages/validation/src/types.validation.ts (1)
IChartEventFilter(28-28)packages/db/src/services/chart.service.ts (1)
getEventFiltersWhereClause(219-470)packages/db/src/clickhouse/client.ts (3)
TABLE_NAMES(48-60)formatClickhouseDate(208-219)chQuery(201-206)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (3)
- GitHub Check: build-and-push-dashboard
- GitHub Check: build-and-push-worker
- GitHub Check: build-and-push-api
🔇 Additional comments (2)
packages/db/src/services/event.service.ts (1)
1020-1021: leadInFrame and lead are not semantically equivalent; proposed changes require careful testing.The proposed diff has both improvements and semantic changes:
Valid improvements:
- Null-safety via
coalesce(..., 0)andgreatest(..., 0)prevents NULL results and negative durationsSemantic concern:
leadInFrame respects the frame bounds specified in the window clause, while lead always uses an unbounded following frame regardless of explicit frame specification. The current code usesROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWINGwith leadInFrame; the proposed code removes this frame and uses lead instead. While functionally similar for typical forward-offset cases, this IS a behavioral change that could affect edge cases or NULL handling.GROUP BY warning is valid: The note about avoiding window functions inside aggregate arguments is correct ClickHouse constraint, though getDurationSql is not currently used inside aggregates—only in SELECT clauses.
Recommendation: Verify that replacing
leadInFrame(...) OVER (... ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING)withlead(...) OVER (...)produces identical results in your test cases before deploying. The null-safety improvements are solid; the function substitution needs validation.packages/db/src/buffers/event-buffer.test.ts (1)
1-363: Excellent test coverage for the refactored buffer implementation.The test suite comprehensively validates the new single-list buffering approach:
- Buffer operations (add, bulkAdd, size tracking)
- Processing flow (ordering, chunking, batch sizing)
- Edge cases (empty buffer, environment configuration)
- Integration points (ClickHouse insertion, active visitors)
- Data integrity (full event fields, serialization)
The tests properly isolate state with
beforeEachcleanup, restore environment variables, and use appropriate mocks. This gives good confidence in the refactored implementation.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Duration can be NULL/negative; coalesce and clamp to zero.
Lead-based diff returns NULL for session tail and can go negative with clock skew. That breaks consumers expecting a number.
Apply this by hardening the helper (below); the call here can stay unchanged once the helper is fixed.
🤖 Prompt for AI Agents
In packages/db/src/services/event.service.ts around lines 512 to 514, the
duration expression can be NULL or negative (lead-based diff for session tail
and clock skew), so update the helper used by getDurationSql() to return SQL
that clamps negatives to zero and coalesces NULLs to 0 (e.g., wrap the computed
duration in GREATEST(..., 0) and COALESCE(..., 0) or equivalent for the SQL
dialect). Keep the call here unchanged — just harden the helper so
sb.select.duration receives a non-null, non-negative numeric duration.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
bounce_stats CTE not joined; wrong alias used.
any(ss.bounce_rate) references a non-existent column; join bounce_stats and use its alias.
- SELECT - ${startOfInterval} AS date, - any(ss.bounce_rate) AS bounce_rate, + SELECT + ${startOfInterval} AS date, + any(bs.bounce_rate) AS bounce_rate, uniq(e.profile_id) AS unique_visitors, uniq(e.session_id) AS total_sessions, any(ss.avg_session_duration) AS avg_session_duration, count(*) AS total_screen_views, round(count(*) * 1.0 / nullIf(uniq(e.session_id), 0), 2) AS views_per_session FROM ${TABLE_NAMES.events} e - LEFT JOIN session_stats ss ON ${startOfInterval} = ss.date + LEFT JOIN session_stats ss ON ${startOfInterval} = ss.date + LEFT JOIN bounce_stats bs ON ${startOfInterval} = bs.date
📝 Committable suggestion
!!️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
🤖 Prompt for AI Agents
In packages/db/src/services/overview.service.ts around lines 262 to 279, the
query is referencing any(ss.bounce_rate) but the bounce_stats CTE is never
joined and uses the wrong alias; fix by adding a LEFT JOIN bounce_stats bs ON
${startOfInterval} = bs.date (matching how session_stats is joined) and change
any(ss.bounce_rate) to any(bs.bounce_rate) so the bounce_rate column comes from
the correct CTE while keeping session_stats (ss) for avg_session_duration.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Correct session aggregations with Collapsing tables.
Use avgIf(duration, duration > 0 AND sign > 0) and sumIf-based bounce/sessions.
SELECT
${mode}_origin AS origin,
${mode}_path AS path,
- round(avg(duration * sign) / 1000, 2) AS avg_duration,
- round(sum(sign * is_bounce) * 100.0 / sum(sign), 2) AS bounce_rate,
- sum(sign) AS sessions
+ round(avgIf(duration, duration > 0 AND sign > 0) / 1000, 2) AS avg_duration,
+ round(sumIf(is_bounce, sign = 1) * 100.0 / nullIf(sumIf(sign, sign = 1), 0), 2) AS bounce_rate,
+ sumIf(sign, sign = 1) AS sessions📝 Committable suggestion
!!️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
🤖 Prompt for AI Agents
In packages/db/src/services/overview.service.ts around lines 526 to 543, the
current aggregates use plain avg/sum which miscounts when using
CollapsingMergeTree semantics; replace avg(duration * sign) / 1000 with
avgIf(duration, duration > 0 AND sign > 0) / 1000 (or round(avgIf(duration,
duration > 0 AND sign > 0) / 1000, 2)), replace round(sum(sign * is_bounce) *
100.0 / sum(sign), 2) with round(sumIf(is_bounce, duration > 0 AND sign > 0) *
100.0 / sumIf(sign, duration > 0 AND sign > 0), 2), and replace sum(sign) AS
sessions with sumIf(sign, duration > 0 AND sign > 0) AS sessions; keep same
GROUP BY, HAVING and ORDER clauses but ensure HAVING uses sumIf(sign, duration >
0 AND sign > 0) > 0.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Filter sessions by sign=1 for duration stats.
Without sign = 1, collapsed rows (or tombstones) skew avg and P90.
- sessionDuration AS ( + sessionDuration AS ( SELECT - avg(duration) / 1000 as durationAvg, - quantilesExactInclusive(0.9)(duration)[1] / 1000 as durationP90 - FROM ${TABLE_NAMES.sessions} - WHERE profile_id = ${sqlstring.escape(profileId)} AND project_id = ${sqlstring.escape(projectId)} + avgIf(duration, sign > 0) / 1000 as durationAvg, + quantilesExactInclusive(0.9)(multiIf(sign > 0, duration, NULL)) [1] / 1000 as durationP90 + FROM ${TABLE_NAMES.sessions} + WHERE profile_id = ${sqlstring.escape(profileId)} + AND project_id = ${sqlstring.escape(projectId)} + AND sign = 1 ),
📝 Committable suggestion
!!️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
🤖 Prompt for AI Agents
In packages/db/src/services/profile.service.ts around lines 46 to 52, the
sessionDuration CTE’s WHERE clause is missing a filter on sign=1 which allows
collapsed/tombstone rows to skew duration stats; update the WHERE clause to
include AND sign = 1 (keeping existing escaped profileId and projectId) so
avg(duration) and quantilesExactInclusive(0.9)(duration) only consider
positive-sign sessions.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
🧩 Analysis chain
Window function inside avg will fail; precompute duration in a subquery.
ClickHouse doesn’t allow window funcs inside aggregate args with GROUP BY. Compute duration first, then aggregate.
Apply these diffs:
Paths
- .from(TABLE_NAMES.events) - .where('project_id', '=', input.projectId) - .where('path', '!=', '') - .where('created_at', '>=', formatClickhouseDate(subMinutes(new Date(), 30))) - .groupBy(['path', 'origin']) + .from( + clix.exp(`( + SELECT + origin, + path, + session_id, + ${getDurationSql()} AS duration + FROM ${TABLE_NAMES.events} + WHERE project_id = ${sqlstring.escape(input.projectId)} + AND path != '' + AND created_at >= '${formatClickhouseDate(subMinutes(new Date(), 30))}' + )`) + ) + .groupBy(['path', 'origin']) .orderBy('count', 'DESC') .limit(100) .execute();
Referrals
- .from(TABLE_NAMES.events) - .where('project_id', '=', input.projectId) - .where('referrer_name', 'IS NOT NULL') - .where('created_at', '>=', formatClickhouseDate(subMinutes(new Date(), 30))) - .groupBy(['referrer_name']) + .from( + clix.exp(`( + SELECT + referrer_name, + session_id, + ${getDurationSql()} AS duration + FROM ${TABLE_NAMES.events} + WHERE project_id = ${sqlstring.escape(input.projectId)} + AND referrer_name IS NOT NULL + AND created_at >= '${formatClickhouseDate(subMinutes(new Date(), 30))}' + )`) + ) + .groupBy(['referrer_name']) .orderBy('count', 'DESC') .limit(100) .execute();
Geo
- .from(TABLE_NAMES.events) - .where('project_id', '=', input.projectId) - .where('created_at', '>=', formatClickhouseDate(subMinutes(new Date(), 30))) - .groupBy(['country', 'city']) + .from( + clix.exp(`( + SELECT + country, + city, + session_id, + ${getDurationSql()} AS duration + FROM ${TABLE_NAMES.events} + WHERE project_id = ${sqlstring.escape(input.projectId)} + AND created_at >= '${formatClickhouseDate(subMinutes(new Date(), 30))}' + )`) + ) + .groupBy(['country', 'city']) .orderBy('count', 'DESC') .limit(100) .execute();
Optional: use round(avgIf(duration, duration > 0)/1000, 2) to ignore zero/NULL durations.
Confirm whether these should be restricted to name = 'screen_view'. If yes, add it inside the subqueries.
Also applies to: 96-100, 124-130
I'll verify the technical claim and code locations for this review comment. Let me gather the necessary information.
🏁 Script executed:
#!/bin/bash # Find getDurationSql implementation and check what it returns rg -n "getDurationSql" --type=ts -B2 -A5 | head -50
Length of output: 3252
🏁 Script executed:
#!/bin/bash # View the actual code at the specified line ranges in realtime.ts fd "realtime.ts" --type=f | xargs -I {} sh -c 'echo "=== File: {} ===" && sed -n "60,75p" {} && echo "" && sed -n "90,105p" {} && echo "" && sed -n "118,135p" {}'
Length of output: 1651
🏁 Script executed:
#!/bin/bash # Check what clix is and how exp() works - verify pattern used rg -n "clix\.exp|from.*clix\.exp" --type=ts -B2 -A2 | head -30
Length of output: 2250
🌐 Web query:
ClickHouse window functions inside aggregate functions GROUP BY
💡 Result:
Short answer: Yes — ClickHouse supports using aggregate functions as window functions (e.g. sum(...) OVER (...)), but you cannot directly nest window expressions inside other aggregate/GROUP BY clauses. Compute aggregates either as window functions with an OVER() clause, or do the GROUP BY aggregation first and then apply window functions in an outer query (or vice‐versa depending on intent).
Key points and examples:
- ClickHouse supports aggregate_function(column) OVER (PARTITION BY ... ORDER BY ... [ROWS|RANGE ...]). Use this when you want an aggregate computed per row across a frame/partition. [1]
Example:
SELECT
id,
value,
sum(value) OVER (PARTITION BY id ORDER BY ts ROWS BETWEEN UNBOUNDED PRECEDING AND CURRENT ROW) AS running_sum
FROM events; [1] - You cannot mix a plain GROUP BY aggregate and a window expression in invalid contexts (e.g. putting a window call inside another aggregate or using aggregates incorrectly in WHERE/GROUP BY) — that produces illegal aggregation errors. If you need both, do one in a subquery and the other in an outer query. [2][3]
Example (aggregate then window):
SELECT id, value, avg_value, value - avg_value AS diff
FROM (
SELECT id, value, avg(value) OVER (PARTITION BY id) AS avg_value
FROM events
); [1][3]
References:
- ClickHouse window functions docs (syntax, aggregate functions as window functions). [1]
- Common ILLEGAL_AGGREGATION explanation / patterns and fixes (mixing aggregates and window usage). [2]
- ClickHouse aggregate functions reference. [3]
Would you like a concrete example using your table schema so I can show the subquery vs. OVER() patterns?
I need to find the actual implementation of getDurationSql() to confirm whether it returns a window function that cannot be nested in aggregate functions.
---
🏁 Script executed:
```shell
#!/bin/bash
# Find the getDurationSql implementation in the db package
fd "event.service.ts" --type=f | xargs rg -n "export.*getDurationSql|function getDurationSql|const getDurationSql" -A10
Length of output: 250
🏁 Script executed:
#!/bin/bash # Also search for any getDurationSql definition/export more broadly rg -n "getDurationSql\s*=" --type=ts -A5 | head -40
Length of output: 334
🏁 Script executed:
#!/bin/bash # Check the imports in realtime.ts to confirm the source rg -n "from '@openpanel/db'" packages/trpc/src/routers/realtime.ts -A10 -B2
Length of output: 435
Now I need to verify the specific implementation details of the proposed diffs and whether they match codebase patterns.
---
🏁 Script executed:
```shell
#!/bin/bash
# Check how sqlstring is used with project_id in codebase
rg -n "sqlstring.escape.*projectId|sqlstring.escape.*project_id" --type=ts -B2 -A2 | head -30
Length of output: 2369
🏁 Script executed:
#!/bin/bash # Check if paths/referrals/geo queries in realtime.ts filter by event name elsewhere rg -n "name.*=|name.*IN" packages/trpc/src/routers/realtime.ts -B5 -A5 | grep -A5 -B5 "paths\|referrals\|coordinates"
Length of output: 0
🏁 Script executed:
#!/bin/bash # Check overview.ts for similar queries to see if name filter is used rg -n "\.where.*'name'" packages/trpc/src/routers/overview.ts -B3 -A3 | head -40
Length of output: 1300
I found one critical issue with the proposed diffs. Let me verify how clix.exp() works with .from():
---
🏁 Script executed:
```shell
#!/bin/bash
# Check if clix.exp() is used in .from() anywhere in codebase
rg -n "\.from\s*\(" --type=ts -A1 packages/trpc/src/routers/ | grep -B1 "clix.exp\|exp\("
Length of output: 23
🏁 Script executed:
#!/bin/bash # Check how clix handles subqueries or raw SQL in from clause rg -n "clix.*\.from" --type=ts -B2 -A3 | head -50
Length of output: 959
🏁 Script executed:
#!/bin/bash # Look for any existing examples of using raw SQL as a table/subquery with clix rg -n "SELECT|from.*\(" packages/trpc/src/routers/realtime.ts -B1 -A1 | head -40
Length of output: 613
I found a critical issue: the proposed diffs use .from(clix.exp()) which doesn't appear in the codebase. Let me check the correct pattern:
---
🏁 Script executed:
```shell
#!/bin/bash
# Find how the coordinates query (which uses raw SQL) is implemented
sed -n "22,35p" packages/trpc/src/routers/realtime.ts
Length of output: 545
🏁 Script executed:
#!/bin/bash # Search for any clix subquery or CTE/WITH patterns rg -n "WITH\s|CTE|subquery" packages/db/src/clickhouse/ --type=ts -B2 -A2 | head -40
Length of output: 2447
🏁 Script executed:
#!/bin/bash # Check clix query builder for .from() signature - look at type definitions or examples rg -n "from.*\(.*\)" packages/db/src/clickhouse/query-builder.ts -B1 -A3 | head -60
Length of output: 639
Found it! The clix builder supports Expression objects in .from(). Let me verify the complete solution:
---
🏁 Script executed:
```shell
#!/bin/bash
# Find clix.exp definition to confirm it returns Expression
rg -n "exp\s*\(" packages/db/src/clickhouse/ --type=ts -B1 -A3 | grep -B1 -A3 "export.*exp\|function exp"
Length of output: 0
🏁 Script executed:
#!/bin/bash # Check if paths/referrals/geo are meant to track only screen_view events # Look for any documentation or related queries that might filter by event type rg -n "screen_view|session_start" packages/trpc/src/routers/realtime.ts -B3 -A3
Length of output: 0
🏁 Script executed:
#!/bin/bash # Verify the Expression type and clix.exp usage pattern rg -n "class Expression|interface Expression|export.*clix" packages/db/src/clickhouse/client.ts -A5 | head -30
Length of output: 0
Fix window function aggregation error by precomputing duration in subqueries before GROUP BY.
The critical issue is confirmed: getDurationSql() returns a ClickHouse window function (dateDiff(...) OVER (PARTITION BY session_id ...)), which cannot be nested inside avg() with GROUP BY. This causes an ILLEGAL_AGGREGATION error at runtime.
The proposed subquery approach is technically sound and follows codebase patterns (clix with Expression objects, sqlstring.escape parameterization), but the exact syntax should be verified during implementation. The three locations requiring fixes are at lines 70, 98, and 128 in packages/trpc/src/routers/realtime.ts.
Regarding the optional screen_view filter: verify whether paths, referrals, and geo queries should be constrained to name = 'screen_view' like similar queries in overview.ts. If yes, add the filter inside the subqueries.
🤖 Prompt for AI Agents
In packages/trpc/src/routers/realtime.ts around lines 67-72 (and also update
analogous blocks at ~98 and ~128), the avg(round(... getDurationSql() ...)) call
is illegal because getDurationSql() returns a ClickHouse window function that
cannot be aggregated directly; rewrite each query to compute the windowed
duration in a subquery (select the window function as duration along with
origin/path/etc), then in the outer query perform COUNT(*) and
round(avg(duration)/1000,2) as avg_duration; keep using the codebase pattern of
clix Expression objects and sqlstring.escape for parameters when building the
subquery and outer query; also verify whether the paths/referrals/geo queries
should include the filter name = 'screen_view' like overview.ts and add that
filter inside the subquery if required.
Uh oh!
There was an error while loading. Please reload this page.
Summary by CodeRabbit
Release Notes
Tests
Performance
Changes
durationfrom chart property list.