Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

Sign up
Appearance settings

feat: add Focused Iteration - Standard strategy with adaptive search #917

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
LearningCircuit merged 1 commit into dev from feature/focused-iteration-standard-strategy
Oct 10, 2025

Conversation

Copy link
Owner

@LearningCircuit LearningCircuit commented Oct 10, 2025

  • Add FlexibleBrowseCompQuestionGenerator with simpler, less prescriptive prompts
  • Track result counts per iteration and show in question generation context
  • Automatically detect when searches return 0 results and suggest broader queries
  • Remove limits on knowledge summary (show all results, full snippets)
  • Fix citation numbering to start at [1] instead of offset
  • Display actual search questions in iteration findings (not just original query)
  • Temporarily disable early termination for testing
  • Add new strategy to frontend dropdown

The new strategy uses:

  • BrowseComp question generator (flexible variant)
  • Standard citation handler (allows "information not found")
  • Adaptive search strategy based on result feedback

- Add FlexibleBrowseCompQuestionGenerator with simpler, less prescriptive prompts
- Track result counts per iteration and show in question generation context
- Automatically detect when searches return 0 results and suggest broader queries
- Remove limits on knowledge summary (show all results, full snippets)
- Fix citation numbering to start at [1] instead of offset
- Display actual search questions in iteration findings (not just original query)
- Temporarily disable early termination for testing
- Add new strategy to frontend dropdown
The new strategy uses:
- BrowseComp question generator (flexible variant)
- Standard citation handler (allows "information not found")
- Adaptive search strategy based on result feedback
@Copilot Copilot AI review requested due to automatic review settings October 10, 2025 16:26
Copy link

claude bot commented Oct 10, 2025
edited
Loading

Claude finished @LearningCircuit's task —— View job


Comprehensive PR Review: Focused Iteration - Standard Strategy

I've completed a thorough review of this PR implementation. Here's my detailed feedback:

Todo List:

  • Read repository guidelines (CLAUDE.md)
  • Analyze the new FlexibleBrowseCompQuestionGenerator
  • Review modifications to existing browsecomp_question.py
  • Examine changes to focused_iteration_strategy.py
  • Check search_system_factory.py integration
  • Review frontend template changes
  • Assess code quality and best practices
  • Check for potential bugs and security issues
  • Evaluate performance considerations
  • Review test coverage
  • Provide comprehensive feedback

🎯 Overall Assessment: HIGH QUALITY with Minor Issues

This is a well-implemented feature that adds valuable functionality to the search system. The code follows good patterns and integrates cleanly with the existing architecture.


Strengths

1. Code Quality & Architecture

  • Clean inheritance pattern: FlexibleBrowseCompQuestionGenerator extends BrowseCompQuestionGenerator appropriately
  • Proper separation of concerns: New strategy variant cleanly separated from base implementation
  • Consistent error handling: Proper exception handling with fallback mechanisms
  • Good documentation: Clear docstrings explaining the purpose and behavior

2. Adaptive Search Features

  • Smart failure detection: Tracks zero-result searches and adapts strategy (lines 32-37 in flexible_browsecomp_question.py)
  • Flexible prompting: Less prescriptive prompts give LLM more freedom
  • Robust fallback: Falls back to parent implementation when insufficient results (lines 89-99)

3. Integration Quality

  • Clean factory integration: New strategy properly registered in search_system_factory.py
  • Frontend integration: UI properly updated with new option
  • Backward compatibility: Existing strategies remain unchanged

⚠️ Issues & Concerns

1. Missing Export (__init__.py)

Severity: Medium

# src/local_deep_research/advanced_search_system/questions/__init__.py
# Missing FlexibleBrowseCompQuestionGenerator in __all__

The new FlexibleBrowseCompQuestionGenerator class is not exported in the package's __init__.py, making it less discoverable and potentially causing import issues.

Fix needed:

from .flexible_browsecomp_question import FlexibleBrowseCompQuestionGenerator
__all__ = [
 # ... existing exports ...
 "FlexibleBrowseCompQuestionGenerator",
]

2. Test Coverage Gap

Severity: Medium

  • No tests found specifically for FlexibleBrowseCompQuestionGenerator
  • No tests for the new focused-iteration-standard strategy
  • Critical adaptive search behavior (zero-result detection) not tested

3. Potential Performance Issues

Severity: Low-Medium

In focused_iteration_strategy.py:274-275:

self.all_search_results.extend(iteration_results)
self.all_links_of_system.extend(iteration_results)

Results are stored in two separate lists, potentially causing memory duplication for large result sets.

4. Code Style Issues

Severity: Low

In browsecomp_question.py:95:

# current_category = None # Not currently used

Dead code comment should be removed.

In flexible_browsecomp_question.py:71-86:

for prefix in [
 "Q:", "Search:", "-", "*", "•", "1.", "2.", "3.", "4.", "5.",
]:

This hardcoded list could be extracted to a class constant for maintainability.


🔒 Security Assessment

  • No security vulnerabilities identified
  • Input validation handled by parent classes
  • No direct SQL or file system access
  • LLM inputs properly structured

🚀 Performance Considerations

Positive

  • Adaptive search reduces unnecessary queries when results are poor
  • Fallback mechanism prevents infinite loops
  • Result tracking helps optimize future iterations

Areas for Improvement

  • Consider result deduplication to reduce memory usage
  • Monitor performance impact of tracking results_by_iteration

📋 Recommendations

High Priority

  1. Add missing export to __init__.py
  2. Add test coverage for new components:
    • Unit tests for FlexibleBrowseCompQuestionGenerator
    • Integration tests for focused-iteration-standard strategy
    • Tests for adaptive search behavior

Medium Priority

  1. Address memory efficiency - consider deduplicating result storage
  2. Extract magic numbers - move hardcoded prefixes to constants

Low Priority

  1. Remove dead code comments
  2. Add type hints where missing

🧪 Test Coverage Gaps

The following areas lack test coverage:

  • Zero-result detection and fallback behavior
  • Flexible question generation vs. standard behavior
  • Integration between new citation handler and question generator
  • Frontend dropdown functionality with new strategy

📝 Documentation

Strengths:

  • Good inline documentation explaining adaptive behavior
  • Clear PR description of changes and features

Could improve:

  • Add example usage scenarios in docstrings
  • Document performance characteristics of adaptive search

🎯 Final Verdict

This is a solid implementation that adds valuable functionality. The adaptive search features are well-designed and the code integrates cleanly. The main issues are:

  1. Missing package export (easy fix)
  2. Insufficient test coverage (needs attention)
  3. Minor performance and style improvements

Recommendation: APPROVE with requested changes - The core functionality is sound, but the missing export and test coverage should be addressed before merging.


Copy link
Contributor

@Copilot Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR introduces a new "Focused Iteration - Standard" strategy that combines BrowseComp question generation with a standard citation handler, enabling more flexible search behavior and allowing "information not found" responses.

  • Adds FlexibleBrowseCompQuestionGenerator with simplified, less prescriptive prompts
  • Implements adaptive search strategy that detects failing searches and suggests broader queries
  • Removes limits on knowledge summaries and fixes citation numbering to start at [1]

Reviewed Changes

Copilot reviewed 5 out of 5 changed files in this pull request and generated 3 comments.

Show a summary per file
File Description
research.html Adds new strategy option to frontend dropdown
search_system_factory.py Creates factory method for new strategy with standard citation handler
focused_iteration_strategy.py Updates strategy to track result counts, show actual search questions, and temporarily disable early termination
flexible_browsecomp_question.py New question generator with adaptive search based on result feedback
browsecomp_question.py Adds result tracking and adaptive strategy for failing searches

Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.

Comment on lines 289 to +296
finding = {
"phase": f"Iteration {iteration}",
"content": f"Searched with {len(questions)} questions, found {len(iteration_results)} results.",
"question": query,
"question": questions[0]
if len(questions) == 1
else "\n".join(questions)
if questions
else query,
Copy link

Copilot AI Oct 10, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[nitpick] The nested ternary operators make this code difficult to read. Consider using an if-elif-else block or extracting this logic into a helper method for better readability.

Copilot uses AI. Check for mistakes.

Comment on lines +303 to +305
# if self._should_terminate_early(iteration):
# logger.info(f"Early termination at iteration {iteration}")
# break
Copy link

Copilot AI Oct 10, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Commented-out code should be removed before merging to production. If this functionality needs to be temporarily disabled, consider using a feature flag or configuration parameter instead.

Suggested change
# if self._should_terminate_early(iteration):
# logger.info(f"Early termination at iteration {iteration}")
# break

Copilot uses AI. Check for mistakes.

Current findings:
{current_knowledge}
Create {num_questions} diverse search queries. Avoid exact duplicates. One per line.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should this include an example to make it more likely that it produces questions in the expected format?

Copy link
Owner Author

@LearningCircuit LearningCircuit Oct 10, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would prefere if the LLM could create the right format adaptive to the situation. But maybe we also need to think more about this?

<select id="strategy" name="strategy" class="ldr-form-control">
<option value="source-based" {% if settings.search_strategy == 'source-based' %}selected{% endif %}>Source-Based (~70% SimpleQA, Comprehensive Research & Text Creation)</option>
<option value="focused-iteration" {% if settings.search_strategy == 'focused-iteration' %}selected{% endif %}>Focused Iteration (~95% SimpleQA, Fast & Precise Q&A)</option>
<option value="focused-iteration-standard" {% if settings.search_strategy == 'focused-iteration-standard' %}selected{% endif %}>Focused Iteration - Standard (Standard citation handler, allows "information not found")</option>
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I still think we would benefit from more intuitive names for these, but it's probably alright for now.

Copy link
Owner Author

@LearningCircuit LearningCircuit Oct 10, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Totally agree. I am still searching to make focused iteration more viable for daily use.

@LearningCircuit LearningCircuit merged commit 3ce5c45 into dev Oct 10, 2025
33 of 34 checks passed
@LearningCircuit LearningCircuit deleted the feature/focused-iteration-standard-strategy branch October 10, 2025 17:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

Copilot code review Copilot Copilot left review comments

@djpetti djpetti djpetti approved these changes

@HashedViking HashedViking Awaiting requested review from HashedViking HashedViking is a code owner

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

AltStyle によって変換されたページ (->オリジナル) /