Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

Sign up
Appearance settings

[AI SDK] Removes input setting from v1 and adds test cases to v2 #46

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
SamstyleGhost wants to merge 1 commit into main
base: main
Choose a base branch
Loading
from 10-22-_ai_sdk_removes_input_setting_from_v1_and_adds_test_cases_to_v2

Conversation

@SamstyleGhost
Copy link
Contributor

@SamstyleGhost SamstyleGhost commented Oct 22, 2025

No description provided.

Copy link
Contributor Author

This stack of pull requests is managed by Graphite. Learn more about stacking.

Copy link

coderabbitai bot commented Oct 22, 2025
edited
Loading

Note

Other AI code review bot(s) detected

CodeRabbit has detected other AI code review bot(s) in this pull request and will avoid duplicating their findings in the review comments. This may lead to a less comprehensive review.

📝 Walkthrough

Summary by CodeRabbit

Release v6.19.0

  • Bug Fixes

    • Resolved trace input setting incompatibility with AI SDK v1
  • Tests

    • Added comprehensive test suite for AI SDK v2 integration covering streaming, multi-modal inputs, conversations, and tool calling
    • Added end-to-end City Prediction workflow test

Walkthrough

Removes trace.input recording for AI SDK v1, bumps package version to v6.19.0, adds a large AI SDK v2 test suite for Vercel/Maxim, and introduces an end-to-end City Prediction test script.

Changes

Cohort / File(s) Summary
Version & Changelog
README.md, package.json
Bump to v6.19.0 and add changelog entry: removal of trace input for AI SDK v1 and added AI SDK v2 tests.
Logging Flow
src/lib/logger/vercel/wrapper.ts
Removed logic that extracted user messages from promptMessages and invoked trace.input; setupLogging now returns metadata, trace, session, span, and promptMessages without recording user input.
AI SDK V2 Tests
src/lib/logger/vercel/vercelLoggerV2.test.ts
Add comprehensive V2 integration tests (text & streaming generation, object/schema generation, sessions/multi-turn, multi-modal inputs, tool calls, error paths, concurrency, custom trace/span IDs).
Integration Demo / Script
src/lib/tests/vercel/agent_testing.ts
New end-to-end City Prediction script using Maxim logger: init, generate text, parse structured data via Zod schema, format output, with error handling and cleanup.

Sequence Diagram(s)

sequenceDiagram
 participant Client
 participant setupLogging as wrapper.ts:setupLogging
 participant Logger as MaximLogger
 Note over Client,setupLogging: Before: setupLogging extracted\nuser message and called trace.input
 Client->>setupLogging: call setupLogging(promptMessages, providerOptions)
 note right of setupLogging: Extract user message removed\nNo trace.input() invocation
 setupLogging->>Logger: return { maximMetadata, trace, session, span, promptMessages }
 Logger-->>Client: metadata + trace objects (no recorded user input)
Loading

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~20 minutes

Poem

A rabbit hops into v6.19, hooray,
I left old traces quietly away,
V2 tests sparkle, streaming and bright,
City predictions take flight in the night,
Code trimmed and tested — a soft carrot bite. 🥕

Pre-merge checks and finishing touches

❌ Failed checks (1 warning)
Check name Status Explanation Resolution
Description Check ⚠️ Warning The pull request has no description provided by the author. While the check is lenient and aims to pass as long as the description is not completely off-topic, an entirely absent description fails to describe any part of the changeset whatsoever. This represents a complete absence of contextual information that would help reviewers understand the motivation, scope, or impact of the changes, even though the title itself is clear and informative. Add a pull request description that provides context for the changes. At minimum, explain why the trace input setting was removed from v1, what benefits the v2 test cases provide, and any testing recommendations or known limitations. Even a brief description would help reviewers and future maintainers understand the reasoning behind these modifications.
✅ Passed checks (2 passed)
Check name Status Explanation
Title Check ✅ Passed The pull request title "[AI SDK] Removes input setting from v1 and adds test cases to v2" directly and accurately summarizes the main changes in the changeset. The changes show removal of trace input logic in wrapper.ts (related to AI SDK v1) and addition of comprehensive test suites in vercelLoggerV2.test.ts and agent_testing.ts (for AI SDK v2), along with a corresponding version bump in package.json and changelog update in README.md. The title is concise, clear, and specific enough for a teammate to understand the primary objectives without being vague or misleading.
Docstring Coverage ✅ Passed Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%.
✨ Finishing touches
  • 📝 Generate docstrings
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Post copyable unit tests in a comment
  • Commit unit tests in branch 10-22-_ai_sdk_removes_input_setting_from_v1_and_adds_test_cases_to_v2

Comment @coderabbitai help to get the list of available commands and usage tips.

@SamstyleGhost SamstyleGhost force-pushed the 10-22-_ai_sdk_removes_input_setting_from_v1_and_adds_test_cases_to_v2 branch from af9bc12 to 77440b3 Compare October 22, 2025 13:11
@SamstyleGhost SamstyleGhost marked this pull request as ready for review October 22, 2025 13:12
@Copilot Copilot AI review requested due to automatic review settings October 22, 2025 13:12
Copy link

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR removes automatic input setting from AI SDK v1 wrapper and adds comprehensive test coverage for v2 specification. The changes prevent v1 wrapper from automatically extracting and logging user input from messages, deferring this responsibility to the caller, while ensuring v2 functionality is thoroughly tested.

  • Removed automatic trace input extraction logic from v1 wrapper
  • Added comprehensive test suite for AI SDK v2 specification covering multiple providers and scenarios
  • Updated version to 6.19.0 with changelog entry

Reviewed Changes

Copilot reviewed 5 out of 5 changed files in this pull request and generated 1 comment.

Show a summary per file
File Description
src/lib/logger/vercel/wrapper.ts Removes automatic input extraction from user messages in v1 wrapper
src/lib/tests/vercel/agent_testing.ts Adds manual testing script demonstrating proper trace input usage
src/lib/logger/vercel/vercelLoggerV2.test.ts Adds comprehensive test suite for v2 specification covering text generation, streaming, objects, tools, sessions, multi-modal, and error handling
package.json Bumps version to 6.19.0
README.md Documents the changes in changelog

Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.

Copy link

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)
package.json (1)

23-54: Pin all dependencies (no carets allowed).

Per coding guidelines, remove all leading ^ to avoid auto-updates. This ensures reproducible builds and CI stability.

Apply diffs (excerpted):

- "@ai-sdk/anthropic": "^2.0.9",
+ "@ai-sdk/anthropic": "2.0.9",
- "@ai-sdk/google": "^2.0.23",
+ "@ai-sdk/google": "2.0.23",
- "@ai-sdk/openai": "^2.0.23",
+ "@ai-sdk/openai": "2.0.23",
- "@langchain/anthropic": "^0.3.21",
+ "@langchain/anthropic": "0.3.21",
- "@langchain/community": "^0.3.46",
+ "@langchain/community": "0.3.46",
- "@langchain/core": "^0.3.0",
+ "@langchain/core": "0.3.0",
- "@langchain/langgraph": "^0.3.1",
+ "@langchain/langgraph": "0.3.1",
- "@langchain/openai": "^0.5.12",
+ "@langchain/openai": "0.5.12",
- "@openai/agents": "^0.1.9",
+ "@openai/agents": "0.1.9",
- "@types/jest": "^29.5.14",
+ "@types/jest": "29.5.14",
- "ai": "^5.0.72",
+ "ai": "5.0.72",
- "ai-sdk-provider-v1": "npm:@ai-sdk/provider@^1.0.0",
+ "ai-sdk-provider-v1": "npm:@ai-sdk/provider@1.0.0",
- "ai-sdk-provider-v2": "npm:@ai-sdk/provider@^2.0.0",
+ "ai-sdk-provider-v2": "npm:@ai-sdk/provider@2.0.0",
- "dotenv": "^16.5.0",
+ "dotenv": "16.5.0",
- "langchain": "^0.3.28",
+ "langchain": "0.3.28",
- "rimraf": "^6.0.1",
+ "rimraf": "6.0.1",
- "ts-jest": "^29.3.4",
+ "ts-jest": "29.3.4",
- "ts-node": "^10.9.2",
+ "ts-node": "10.9.2",
- "typedoc": "^0.28.5",
+ "typedoc": "0.28.5",
- "typedoc-plugin-frontmatter": "^1.3.0",
+ "typedoc-plugin-frontmatter": "1.3.0",
- "typedoc-plugin-markdown": "^4.6.4",
+ "typedoc-plugin-markdown": "4.6.4",
- "zod": "^3.25.76"
+ "zod": "3.25.76"
- "axios": "^1.11.0",
+ "axios": "1.11.0",
- "axios-retry": "^4.5.0"
+ "axios-retry": "4.5.0"
- "expo-crypto": "^13.0.0",
+ "expo-crypto": "13.0.0",

As per coding guidelines.

Also applies to: 55-59, 105-107

🧹 Nitpick comments (4)
README.md (1)

526-529: Changelog entry OK; add a short privacy note.

Clarify that v1 wrapper no longer auto-logs user prompts; apps must call trace.input() explicitly if they want inputs captured.

 ### v6.19.0
 - fix: Removes trace input setting for AI SDK v1
 - adds test cases for AI SDK v2 wrapper
+ - Note: AI SDK v1 models no longer auto-populate trace input. If you want to capture user prompts, call `trace.input(...)` explicitly.
src/lib/tests/vercel/agent_testing.ts (2)

15-33: Prefer raiseExceptions during tests.

Enable raiseExceptions for faster failure in CI.

- const maxim = new Maxim({ baseUrl, apiKey, debug: true });
+ const maxim = new Maxim({ baseUrl, apiKey, debug: true, raiseExceptions: true });

103-115: Also log the final output to the trace.

Optional: set trace.output(formattedOutput) before ending the trace to complete the I/O pair.

- const formattedOutput = await formatOutput(model, structuredData, trace);
+ const formattedOutput = await formatOutput(model, structuredData, trace);
+ trace.output(formattedOutput);
src/lib/logger/vercel/vercelLoggerV2.test.ts (1)

23-30: Env var message consistency.

Use the same name everywhere: MAXIM_LOG_REPO_ID.

- throw new Error("MAXIM_BASE_URL, MAXIM_API_KEY & LOG_REPO_ID environment variables are required");
+ throw new Error("MAXIM_BASE_URL, MAXIM_API_KEY & MAXIM_LOG_REPO_ID environment variables are required");
📜 Review details

Configuration used: CodeRabbit UI

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 49a67cb and 77440b3.

📒 Files selected for processing (5)
  • README.md (1 hunks)
  • package.json (1 hunks)
  • src/lib/logger/vercel/vercelLoggerV2.test.ts (1 hunks)
  • src/lib/logger/vercel/wrapper.ts (0 hunks)
  • src/lib/tests/vercel/agent_testing.ts (1 hunks)
💤 Files with no reviewable changes (1)
  • src/lib/logger/vercel/wrapper.ts
🧰 Additional context used
📓 Path-based instructions (1)
package.json

⚙️ CodeRabbit configuration file

do not allow any carats in package.json, we never want any auto updates for any patch versions of any packages in package.json

Files:

  • package.json
🧬 Code graph analysis (2)
src/lib/tests/vercel/agent_testing.ts (3)
src/lib/maxim.ts (2)
  • Maxim (149-1302)
  • logger (1183-1214)
src/lib/logger/components/trace.ts (1)
  • Trace (65-456)
src/lib/logger/vercel/wrapper.ts (1)
  • wrapMaximAISDKModel (28-37)
src/lib/logger/vercel/vercelLoggerV2.test.ts (2)
src/lib/maxim.ts (2)
  • Maxim (149-1302)
  • logger (1183-1214)
src/lib/logger/vercel/wrapper.ts (1)
  • wrapMaximAISDKModel (28-37)
🪛 Biome (2.1.2)
src/lib/logger/vercel/vercelLoggerV2.test.ts

[error] 694-694: Other switch clauses can erroneously access this declaration.
Wrap the declaration in a block to restrict its access to the switch clause.

The declaration is defined in this switch clause:

Safe fix: Wrap the declaration in a block.

(lint/correctness/noSwitchDeclarations)


[error] 695-695: Other switch clauses can erroneously access this declaration.
Wrap the declaration in a block to restrict its access to the switch clause.

The declaration is defined in this switch clause:

Safe fix: Wrap the declaration in a block.

(lint/correctness/noSwitchDeclarations)

🔇 Additional comments (2)
package.json (1)

4-4: Version bump looks good.

6.19.0 aligns with README changelog and new V2 tests.

src/lib/logger/vercel/vercelLoggerV2.test.ts (1)

36-56: Good V2 path smoke test setup.

Wrapping the model and tagging with providerOptions is correct and aligns with wrapper semantics.

Please ensure OPENAI_API_KEY/ANTHROPIC_API_KEY are present in CI for these tests to run against live providers.

@SamstyleGhost SamstyleGhost force-pushed the 10-22-_ai_sdk_removes_input_setting_from_v1_and_adds_test_cases_to_v2 branch from 77440b3 to 9e747d3 Compare October 22, 2025 13:18
Copy link

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

♻️ Duplicate comments (2)
src/lib/logger/vercel/vercelLoggerV2.test.ts (1)

685-701: Wrap case-local declarations in 'median' to satisfy noSwitchDeclarations.

Biome flags const declarations in switch without a block; scope them with braces.

- case "median":
- const sorted = [...data].sort((a, b) => a - b);
- const mid = Math.floor(sorted.length / 2);
- return { 
- result: sorted.length % 2 === 0 ? (sorted[mid - 1] + sorted[mid]) / 2 : sorted[mid],
- type: "median",
- dataPoints: data.length 
- };
+ case "median": {
+ const sorted = [...data].sort((a, b) => a - b);
+ const mid = Math.floor(sorted.length / 2);
+ return {
+ result: sorted.length % 2 === 0 ? (sorted[mid - 1] + sorted[mid]) / 2 : sorted[mid],
+ type: "median",
+ dataPoints: data.length,
+ };
+ }
src/lib/tests/vercel/agent_testing.ts (1)

95-123: Also cleanup the Maxim instance; destructure it from initializeMaxim.

Prevents agent/socket leaks and ensures flush. This was flagged earlier as well.

- const { logger } = await initializeMaxim();
+ const { maxim, logger } = await initializeMaxim();
 try {
 ...
 } catch (error) {
 console.error("Error in city prediction demo:", error);
 } finally {
 await logger.cleanup();
+ await maxim.cleanup();
 }
🧹 Nitpick comments (2)
src/lib/logger/vercel/vercelLoggerV2.test.ts (2)

682-711: Enum has "mode" but switch lacks a handler. Add a mode case.

Prevents default/error path for a declared option.

 switch (analysisType) {
 case "mean":
 return { 
 result: data.reduce((a, b) => a + b, 0) / data.length,
 type: "mean",
 dataPoints: data.length 
 };
- case "median": {
+ case "median": {
 const sorted = [...data].sort((a, b) => a - b);
 const mid = Math.floor(sorted.length / 2);
 return { 
 result: sorted.length % 2 === 0 ? (sorted[mid - 1] + sorted[mid]) / 2 : sorted[mid],
 type: "median",
 dataPoints: data.length 
 };
 }
+ case "mode": {
+ const counts = new Map<number, number>();
+ for (const n of data) counts.set(n, (counts.get(n) ?? 0) + 1);
+ let mode = data[0];
+ let freq = 0;
+ for (const [k, v] of counts) {
+ if (v > freq) {
+ mode = k;
+ freq = v;
+ }
+ }
+ return { result: mode, type: "mode", dataPoints: data.length };
+ }
 case "range":
 return { 
 result: Math.max(...data) - Math.min(...data),
 type: "range",
 min: Math.min(...data),
 max: Math.max(...data),
 dataPoints: data.length 
 };

492-516: Prefer rejects assertion over try/catch + fail in error test.

Cleaner and avoids false positives.

- try {
- // Intentionally cause an error with invalid parameters
- await generateText({
- model: model,
- maxOutputTokens: -1, // Invalid token count
- prompt: "This should cause an error.",
- providerOptions: {
- maxim: {
- traceName: "V2 Error Handling Test",
- generationName: "Invalid Parameters",
- generationTags: {
- test_type: "error_handling",
- specification: "v2",
- },
- } as MaximVercelProviderMetadata,
- },
- });
- 
- // If we reach here, the test should fail
- fail("Expected an error to be thrown");
- } catch (error) {
- console.log("Expected error caught in V2 error handling test:", error);
- expect(error).toBeDefined();
- }
+ await expect(
+ generateText({
+ model: model,
+ maxOutputTokens: -1,
+ prompt: "This should cause an error.",
+ providerOptions: {
+ maxim: {
+ traceName: "V2 Error Handling Test",
+ generationName: "Invalid Parameters",
+ generationTags: { test_type: "error_handling", specification: "v2" },
+ } as MaximVercelProviderMetadata,
+ },
+ })
+ ).rejects.toThrow();
📜 Review details

Configuration used: CodeRabbit UI

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 77440b3 and 9e747d3.

📒 Files selected for processing (5)
  • README.md (1 hunks)
  • package.json (1 hunks)
  • src/lib/logger/vercel/vercelLoggerV2.test.ts (1 hunks)
  • src/lib/logger/vercel/wrapper.ts (0 hunks)
  • src/lib/tests/vercel/agent_testing.ts (1 hunks)
💤 Files with no reviewable changes (1)
  • src/lib/logger/vercel/wrapper.ts
✅ Files skipped from review due to trivial changes (1)
  • README.md
🚧 Files skipped from review as they are similar to previous changes (1)
  • package.json
🧰 Additional context used
🧬 Code graph analysis (2)
src/lib/tests/vercel/agent_testing.ts (3)
src/lib/maxim.ts (2)
  • Maxim (149-1302)
  • logger (1183-1214)
src/lib/logger/components/trace.ts (1)
  • Trace (65-456)
src/lib/logger/vercel/wrapper.ts (1)
  • wrapMaximAISDKModel (28-37)
src/lib/logger/vercel/vercelLoggerV2.test.ts (2)
src/lib/maxim.ts (2)
  • Maxim (149-1302)
  • logger (1183-1214)
src/lib/logger/vercel/wrapper.ts (1)
  • wrapMaximAISDKModel (28-37)
🪛 Biome (2.1.2)
src/lib/logger/vercel/vercelLoggerV2.test.ts

[error] 693-693: Other switch clauses can erroneously access this declaration.
Wrap the declaration in a block to restrict its access to the switch clause.

The declaration is defined in this switch clause:

Safe fix: Wrap the declaration in a block.

(lint/correctness/noSwitchDeclarations)


[error] 694-694: Other switch clauses can erroneously access this declaration.
Wrap the declaration in a block to restrict its access to the switch clause.

The declaration is defined in this switch clause:

Safe fix: Wrap the declaration in a block.

(lint/correctness/noSwitchDeclarations)

🔇 Additional comments (1)
src/lib/logger/vercel/vercelLoggerV2.test.ts (1)

235-243: Verify trace existence when passing only a generated traceId.

You set traceId = uuid() but don’t create a corresponding trace here. Confirm the wrapper/server creates or upserts traces by ID; otherwise, create a trace via logger.trace(...) and reuse its id.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

Copilot code review Copilot Copilot left review comments

@coderabbitai coderabbitai[bot] coderabbitai[bot] approved these changes

At least 2 approving reviews are required to merge this pull request.

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

AltStyle によって変換されたページ (->オリジナル) /