Skip to content

Navigation Menu

Sign in
Sign up

feat(chat): add "use general knowledge" toggle for document chats - #2042

Open
manzke wants to merge 3 commits into
main from
claude/charming-fermat-2d707j
Open

feat(chat): add "use general knowledge" toggle for document chats #2042
manzke wants to merge 3 commits into
main from
claude/charming-fermat-2d707j

Conversation

@manzke

@manzke manzke commented Jul 14, 2026

Copy link
Copy Markdown
Contributor

Summary

Closes #1662.

BMAS reported that when chatting about an uploaded document, the model frequently blends in outside general knowledge, which users find undesirable and confusing. This adds a per-conversation toggle, gated on app.upload.enabled, that lets users restrict answers to the uploaded document(s).

This mirrors the existing websearchEnabled toggle end-to-end:

  • Server
    • server/services/chat/RequestBuilder.js: new appendDocumentOnlyNotice(llmMessages, app, documentOnlyEnabled), modeled directly on appendWebSearchDisabledNotice. No-ops unless app.upload.enabled and the toggle is on; idempotent (won't duplicate the notice); appends a directive telling the model to answer only from the uploaded document(s) and say so explicitly if the answer isn't there.
    • server/validators/index.js: new optional documentOnlyEnabled: z.boolean() on the chat POST schema.
    • server/routes/chat/sessionRoutes.js: destructure/thread documentOnlyEnabled through the same 3 call sites websearchEnabled flows through into prepareChatRequest.
  • Client
    • client/src/shared/hooks/useAppSettings.js: new documentOnlyEnabled state, default false (i.e. general knowledge allowed — current behavior unchanged), persisted per app like the other chat toggles.
    • client/src/features/chat/components/ChatInputActionsMenu.jsx: new toggle section, "Use general knowledge" (checked = general knowledge allowed / off = document-only), gated on app?.upload?.enabled === true, positioned between the Web Search and Transcription sections.
    • client/src/features/chat/components/ChatInput.jsx / client/src/features/apps/pages/AppChat.jsx: prop threading and inclusion in outgoing chat request params, gated on app?.upload?.enabled.
  • Docs
    • docs/apps.md: note under Upload Configuration.
    • docs/releases/5.5.0/features.md: changelog entry.
  • Tests
    • server/tests/document-only-notice.test.js: covers no-op when upload isn't configured, no-op when the toggle is off/undefined, notice appended when both conditions hold, idempotency, and no-op when there's no system message.

Scope notes (from the issue's implementation-plan comment)

  • Toggle only shows for apps with upload.enabled: true (not gated on a file actually being attached in the current turn — kept simple for v1, matching the issue's "either conclusion is a small config change" framing).
  • Outlook add-in (OfficeChatPanel.jsx) is out of scope for this PR, per the plan's note that it's lower priority / deferrable.
  • The maintainer's request for a concrete repro example (issue comment) is still unanswered; this toggle is useful regardless of that answer since it gives users explicit control either way.

Test plan

  • node --test server/tests/document-only-notice.test.js — 5/5 passing
  • npx eslint on all touched files — 0 errors (only pre-existing warning patterns, same shape as websearchEnabled)
  • npx vite build (client) — succeeds
  • timeout 12 node server/server.js — starts cleanly, no errors
  • Manual verification in a running dev server: enable upload.enabled on a test app, upload a document, toggle "Use general knowledge" off, confirm the assistant declines to use outside knowledge

🤖 Generated with Claude Code

https://claude.ai/code/session_01EYnNs4GibUyXfvWLRRgjBV

Generated by Claude Code

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds a per-app toggle allowing document chats to exclude general LLM knowledge.

Changes:

  • Threads and validates document-only mode through client and server.
  • Adds prompt enforcement and unit tests.
  • Documents the feature and release entry.

Reviewed changes

Copilot reviewed 10 out of 10 changed files in this pull request and generated 4 comments.

Show a summary per file
File Description
server/validators/index.js Validates the new request option.
server/tests/document-only-notice.test.js Tests prompt-notice behavior.
server/services/chat/RequestBuilder.js Adds the document-only directive.
server/routes/chat/sessionRoutes.js Threads the option into request preparation.
docs/releases/5.5.0/features.md Adds the release note.
docs/apps.md Documents the toggle.
client/src/shared/hooks/useAppSettings.js Persists toggle state per app.
client/src/features/chat/components/ChatInputActionsMenu.jsx Adds the toggle UI.
client/src/features/chat/components/ChatInput.jsx Threads toggle props.
client/src/features/apps/pages/AppChat.jsx Includes the option in chat requests.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment on lines +564 to +568
{t('documentOnly.toggleLabel', 'Use general knowledge')}
</div>
<div className="text-xs text-gray-500 dark:text-gray-400 mt-0.5">
{t(
'documentOnly.toggleDescription',

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added documentOnly.toggleLabel/toggleDescription to both shared/i18n/en.json and shared/i18n/de.json in a560880.

Generated by Claude Code

Comment thread client/src/features/chat/components/ChatInputActionsMenu.jsx
? { enabledTools: effectiveEnabledTools }
: {}),
...(app?.websearch?.enabled ? { websearchEnabled } : {}),
...(app?.upload?.enabled ? { documentOnlyEnabled } : {}),

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch — websearchEnabled had the same stale-closure gap. Added both to the deps array in a560880.

Generated by Claude Code

Comment on lines +443 to +446
// When the user has turned on "document only" mode, tell the model to
// stick to the uploaded document(s) instead of blending in general
// knowledge (see #1662).
appendDocumentOnlyNotice(llmMessages, app, documentOnlyEnabled);

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in a560880: document-only mode now forces an effectiveWebsearchEnabled = documentOnlyEnabled ? false : websearchEnabled, used for tool loading, native search resolution, and the web-search-disabled notice, so web search tools aren't available at all when document-only mode is on.

Generated by Claude Code

manzke pushed a commit that referenced this pull request Sep 1, 2026 
- Force web search off whenever document-only mode is enabled, so the
 model can't pull in outside info via web search while promising
 document-only answers (RequestBuilder.js).
- Fix stale-closure bug in handleSkillSelect: websearchEnabled and
 documentOnlyEnabled were missing from its useCallback deps, so
 skill-triggered messages could send an outdated toggle value.
- Add missing documentOnly.toggleLabel/toggleDescription i18n keys to
 en.json/de.json.
- Add an aria-label to the document-only toggle checkbox for
 screen-reader accessibility.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EYnNs4GibUyXfvWLRRgjBV
@github-actions github-actions Bot added enhancement New feature or request i18n labels Sep 1, 2026

@claude claude Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nothing blocking. The comments below are optional suggestions. There is no need to push a fix for them before merging.

Beyond the inline divider-styling finding, I also checked whether the "document only" toggle actually restricts the model to the uploaded document: appendDocumentOnlyNotice in server/services/chat/RequestBuilder.js only appends a soft instruction to the system prompt — it does not remove the websearch tool/native web search from the request (built earlier from context.websearchEnabled, which is independent of documentOnlyEnabled). So with both toggles left in their non-default combination, the model can still fetch outside information despite "document only" being on. This looks like intentional v1 scope per the PR description rather than a regression, but flagging it since it's easy to miss.

Extended reasoning...

The diff only contains one confirmed inline finding (missing hasDocumentOnly in the border condition at line 489 of ChatInputActionsMenu.jsx), which I independently verified by reading the surrounding code — lines 532/559/589 were correctly updated to include hasDocumentOnly in their divider logic, but line 489 (governing the "Message context" host-toggle section's bottom border) was left as hasTools || hasWebsearch, so it will render without a divider when an app has upload.enabled but no websearch/tools. This is a real, low-severity UI cosmetic bug.

I also independently verified the ruled-out candidate issue about RequestBuilder.js: reading lines 415-447 confirms documentOnlyEnabled is passed into appendDocumentOnlyNotice which only mutates the system message text; the tools array and nativeWebSearch flag built just above are derived solely from context.websearchEnabled, unaware of documentOnlyEnabled. This means the feature is a soft prompt-level nudge only, not an enforced restriction — consistent with the PR's own "kept simple for v1" framing, so I'm treating it as informational rather than a blocking bug.

Since a confirmed inline finding exists (and per the approval guidelines, findings that seem important enough to delay approval mean I should not approve), I'm not approving this PR. I am posting a brief informational note under the narrow "findings-present ruled-out" exception, since I independently examined and can concretely describe the websearch-tool-not-disabled behavior beyond what's already covered inline, and it's relevant context for whoever reviews this manually.

Additional findings (outside the current diff — GitHub can't attach inline comments there):

  • 🟡 client/src/features/chat/components/ChatInputActionsMenu.jsx — The host-context toggles section's bottom-border condition (hasTools || hasWebsearch) was not updated to include the new hasDocumentOnly, so when an app has upload.enabled but no websearch/tools, the 'Message context' section (Outlook/extension toggles) renders with no divider before the new Document-only section, unlike every other sibling border check in this file which was updated for the new section. Fix: add hasDocumentOnly to this condition (mirroring the fixes already applied at the Web Search, Document-only, and Transcription section borders in the same diff).

    Extended reasoning...

    Render order is: host-context toggles section -> Web Search -> Document-only -> Transcription -> Tools. The diff correctly added hasDocumentOnly to the border conditions at lines 532, 559, and 589, but left line 489 unchanged. Trigger: an embedded host (Outlook taskpane/browser extension) declares contextToggles, and the app has upload.enabled: true but websearch.enabled and tools are both falsy/empty. Then hasHostContextToggles and hasDocumentOnly are true while hasTools/hasWebsearch are false, so the border-b class is omitted and the context-toggles block visually runs into the Document-only toggle block with no separator, unlike the base branch where no such adjacent section existed.

    Verification: nit severity. Desktop menu render order: host-context (line 486) -> Web Search (530) -> Document-only (557) -> Transcription (587) -> Tools (617). The diff added hasDocumentOnly to the border conditions of the sibling sections (Web Search line 532, Document-only 559, Transcription 589) but left the host-context section's condition at line 489 as hasTools || hasWebsearch. When... | nit — missing...

 

)
Apps with file upload enabled now expose a per-conversation toggle,
mirroring the existing websearchEnabled pattern end to end (validator,
RequestBuilder, sessionRoutes, useAppSettings, ChatInputActionsMenu).
When turned off, a directive is appended to the system prompt telling
the model to answer strictly from the uploaded document(s) instead of
blending in general knowledge, addressing BMAS feedback that answers
often pulled in unwanted outside information.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EYnNs4GibUyXfvWLRRgjBV 
- Force web search off whenever document-only mode is enabled, so the
 model can't pull in outside info via web search while promising
 document-only answers (RequestBuilder.js).
- Fix stale-closure bug in handleSkillSelect: websearchEnabled and
 documentOnlyEnabled were missing from its useCallback deps, so
 skill-triggered messages could send an outdated toggle value.
- Add missing documentOnly.toggleLabel/toggleDescription i18n keys to
 en.json/de.json.
- Add an aria-label to the document-only toggle checkbox for
 screen-reader accessibility.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EYnNs4GibUyXfvWLRRgjBV
manzke force-pushed the claude/charming-fermat-2d707j branch from a560880 to 6579407 Compare September 1, 2026 13:17 
The host-context toggles section's bottom-border condition wasn't
updated when the Document-only section was added between it and Web
Search, so it rendered with no divider when an app has upload.enabled
but no websearch/tools. Caught by automated review on #2042.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EYnNs4GibUyXfvWLRRgjBV
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

Copilot code review Copilot
Copilot review effort, defaults to Balanced
Applies to this pull request for everyone.Learn more about Copilot code review.
Copilot left review comments
@claude claude[bot] claude[bot] left review comments

Assignees

No one assigned

Labels

api backend chat documentation Improvements or additions to documentation enhancement New feature or request frontend i18n testing

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Usage of general knowledge of LLM when uploading a document

3 participants

AltStyle によって変換されたページ (->オリジナル) /