-
Notifications
You must be signed in to change notification settings - Fork 615
fix(server): ensure graph clear API does not clear meta table for hbase#2911
fix(server): ensure graph clear API does not clear meta table for hbase #2911LYD031106 wants to merge 10 commits into
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The test only verifies version equality but doesn't validate:
- Meta table content: Check that specific meta entries (like backend version, graph name, etc.) are preserved
- Data table clearing: Verify that actual graph data (vertices, edges) are properly cleared
- Concurrent operations: Test behavior when truncate is called during other operations
Suggestion:
@Test public void testHbaseMetaVersion(){ // Insert some test data // ... add vertices/edges ... String beforeVersion = this.store.storedVersion(); // Get other meta entries this.store.truncate(); String afterVersion = this.store.storedVersion(); Assert.assertEquals(beforeVersion, afterVersion); // Verify data is cleared but meta is intact // ... check vertices/edges are gone ... // ... check other meta entries preserved ... }
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Both BaseHbaseUnitTest and HbaseUnitTest have setup/teardown methods that may cause issues:
- Duplicate cleanup:
@Afterin both base and child class will close store/provider twice - Setup order: Child's
@Beforeruns after parent's, but callsprovider.open()which may conflict
Suggestion:
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
🧹 Code quality: Silent exception swallowing
Using empty catch blocks makes debugging difficult. Consider:
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
🧹 Naming: Space missing in class declaration
Minor style issue:
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
After reviewing the code, I found the real problem:
Current Issue:
HbaseSystemStore.tableNames()(line 572-576) includes the meta table in the listtruncate()clears ALL tables returned bytableNames(), including the meta table- Adding
init()is a workaround, but meta table should never be cleared in the first place
Proper Fix:
Override truncate() in HbaseSystemStore to exclude meta table:
@Override public void truncate() { // Save meta table before truncate List<String> originalTables = this.tableNames(); // Temporarily remove meta table from truncation // Then call super.truncate() on data tables only // Or better: add a separate method tableNamesToTruncate() }
Alternative approach (cleaner):
protected List<String> tableNamesToTruncate() { // Only return data tables, not meta/system tables return super.tableNames(); // Don't include meta.table() }
Then use tableNamesToTruncate() in the truncate() method instead of tableNames().
This matches how MySQL backend works - it only truncates data tables, never system tables.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Your solution is indeed a superior approach. By overriding the truncate() method to exclude metadata tables, we are solving the problem at the design level, which is far more elegant and robust than relying on a temporary solution involving initialization calls.
I will proceed with the modifications as suggested, implementing this fix for the HBase backend. For consistency, I will also apply the same change to the MySQL backend in the corresponding PR (#2888).
Regarding the current implementation for the RocksDB backend, which uses the direct init() call workaround, I believe it should also be optimized. Adopting this unified strategy of overriding the truncate() method will enhance code consistency and maintainability across all backends.Here is the implementation of the truncate() method in RocksStore.
@Override
public synchronized void truncate() {
Lock writeLock = this.storeLock.writeLock();
writeLock.lock();
try {
this.checkOpened();
this.clear(false);
this.init();
// Clear write-batch
this.dbs.values().forEach(BackendSessionPool::forceResetSessions);
LOG.debug("Store truncated: {}", this.store);
} finally {
writeLock.unlock();
}
}
Thank you again for your valuable feedback. It has truly helped me arrive at a much more elegant solution!
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The Counters table contains graph-specific data like vertex/edge ID sequences. When truncating a graph, this data should be cleared to avoid ID conflicts when repopulating the graph.
Only the Meta table (which stores backend version metadata) should be preserved during truncation.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The test method is named testMysqlMetaVersion() but it's testing HBase functionality. This appears to be a copy-paste error from MySQL tests.
This makes the test's purpose clearer and matches the actual functionality being tested.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The test only verifies that meta version is preserved, but doesn't verify the actual truncation behavior. Consider adding assertions to verify:
- Data tables are actually truncated (e.g., insert some data, truncate, verify data is gone)
- Meta table content remains intact
- The graph can be used normally after truncation
Example enhancement:
@Test public void testHbaseMetaVersionAfterTruncate() { BackendStore systemStore = this.provider.loadSystemStore(config); BackendStore graphStore = this.provider.loadGraphStore(config); // Record initial version String beforeVersion = systemStore.storedVersion(); // Insert some test data to verify truncation // ... add test data insertion code ... // Perform truncation this.provider.truncate(); // Verify version preserved String afterVersion = systemStore.storedVersion(); Assert.assertEquals(beforeVersion, afterVersion); // Verify data tables are empty // ... add verification code ... }
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
🧹 Code Style: Formatting
Missing space after class declaration:
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
🧹 Minor: Unnecessary Whitespace Change
Removing this blank line is cosmetic and adds noise to the diff without functional benefit. Consider keeping the original formatting to make the diff cleaner and focus on actual logic changes.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
wrong header format
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The filter condition HugeType.META == e.getKey() may not correctly handle null keys, though this is unlikely in practice. However, the main concern is whether HugeType.META is actually the correct type to filter out.
Consider adding a comment explaining why META type should be preserved during truncation to improve code maintainability.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@Before annotation
The method is named setup() but the annotation @Before is missing. JUnit won't automatically call this method before each test. This could cause tests to fail or behave unexpectedly.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
down() should be tearDown()
The cleanup method should follow JUnit naming conventions and be named tearDown() to match the common pattern.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
For code consistency, add a space after commas in method calls.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The sessions field is opened in setup() but never closed in the cleanup method. This could lead to HBase connection leaks.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
If an assertion fails, the subsequent close() calls won't execute. Consider using try-with-resources or try-finally blocks.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The method name truncatedTableNames() is grammatically incorrect and misleading. It suggests tables that have already been truncated, when it actually returns tables that SHOULD be truncated.
Alternative names: getClearableTableNames(), getDataTableNames(), or getTableNamesExcludingMeta()
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
While unlikely, if e.getKey() is null, this comparison would fail with NPE. Consider using Objects.equals() or put the enum constant on the left side.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The current code 'HugeType.META == e.getKey()' is already NPE-safe regardless of which operand is null, as the '==' operator never throws NullPointerException.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
If any exception occurs after opening stores but before sessions.open(), the opened stores won't be properly closed. This can lead to resource leaks in test execution.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
🧹 Code style: inconsistent spacing
Missing space after if keyword. Per project code style guidelines, control flow statements should have space before parentheses.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
🧹 Code style: inconsistent spacing
Missing space after catch keyword.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
🧹 Code style: inconsistent spacing
Missing space after closing brace and before catch. Should be } catch, not }catch.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The session testsession is not explicitly closed, which could lead to resource leaks. While it might be managed by the parent sessions object, it's better to be explicit.
Consider wrapping the session usage in a try-with-resources block or explicitly closing it after use.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The test inserts data with specific row keys (row_trunc_v, row_trunc_oe, row_trunc_ie). If this test runs multiple times or fails mid-execution, residual data might affect subsequent runs.
Consider:
- Using unique row keys per test run (e.g., append timestamp/UUID)
- Adding explicit cleanup in
@Beforesetup - Verifying the
@Afterteardown properly cleans all test data
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Pull request overview
This PR fixes a critical bug where the HBase graph clear API (truncate()) was incorrectly clearing the system metadata table. The fix introduces a new truncatedTableNames() method that returns only the tables that should be cleared during truncation, while preserving system metadata like backend version information stored in the meta table.
Key Changes
- Added
truncatedTableNames()method inHbaseStoreto filter tables for truncation - Modified
truncate()to usetruncatedTableNames()instead oftableNames() - Added comprehensive unit tests to verify meta table preservation after truncation
Reviewed changes
Copilot reviewed 3 out of 3 changed files in this pull request and generated 5 comments.
| File | Description |
|---|---|
hugegraph-server/hugegraph-hbase/src/main/java/org/apache/hugegraph/backend/store/hbase/HbaseStore.java |
Introduces truncatedTableNames() method to exclude meta table and updates truncate() to use it instead of tableNames() |
hugegraph-server/hugegraph-test/src/main/java/org/apache/hugegraph/unit/hbase/BaseHbaseUnitTest.java |
New base test class for HBase unit tests with setup/teardown infrastructure |
hugegraph-server/hugegraph-test/src/main/java/org/apache/hugegraph/unit/hbase/HbaseUnitTest.java |
New test case verifying meta table version persists after truncation while graph data is cleared |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Copilot
AI
Dec 9, 2025
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This filter condition !(HugeType.META == e.getKey()) is redundant because HugeType.META is never registered via registerTableManager() and therefore will never exist in the this.tables map. The meta table is added separately in HbaseSystemStore.tableNames() (line 582) by calling this.meta.table().
The fix works correctly because truncatedTableNames() simply doesn't include the meta table at all (since it's not in the map), but the explicit filter is misleading and suggests META might be in the map. Consider removing the filter:
protected List<String> truncatedTableNames() { // Exclude meta table to preserve system metadata during graph clear return this.tables.values().stream() .map(BackendTable::table) .collect(Collectors.toList()); }
Copilot
AI
Dec 9, 2025
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Missing space after if keyword. Should be if (this.provider != null) { for consistency with the code style used elsewhere in the method (line 56).
Copilot
AI
Dec 9, 2025
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Missing space after catch keyword. Should be } catch (Exception e) { for consistency with the code style used on line 59.
Copilot
AI
Dec 9, 2025
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Missing space after catch keyword. Should be } catch (Exception e) { for consistency with the code style used on line 59.
Copilot
AI
Dec 9, 2025
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Missing space after comma in log message. Should be "Failed to close provider ", e for consistency with line 60.
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@ ## master #2911 +/- ## ============================================ - Coverage 39.38% 31.89% -7.50% - Complexity 456 488 +32 ============================================ Files 812 801 -11 Lines 68660 67513 -1147 Branches 8968 8771 -197 ============================================ - Hits 27044 21533 -5511 - Misses 38824 43579 +4755 + Partials 2792 2401 -391 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
LYD031106
commented
Jan 14, 2026
@imbajin
In the current CI/CD process, when a test task starts, backend server instance (such as memory, MySQL, HBase, etc.) is installed and launched. Recent PRs #2911 and #2888 have introduced test cases targeted at specific backends (e.g., MySQL or HBase).
However, when the CI/CD job is configured to run with the memory backend, no MySQL or HBase instances are actually launched in the environment. This causes the test cases that depend on those specific backends to fail.
The current fix is to use the Assume mechanism in the relevant test methods. Before executing a test, it checks the type of backend currently running. If the backend does not meet the test's requirements, the test is skipped, preventing build failures due to environment mismatch.
Is this solution appropriate, and are there any best practices for current projects?
Purpose of the PR
Main Changes
The PR fixes the issue where the graph clear API mistakenly clears the meta table in Hbase backend storage.
Verifying these changes
Does this PR potentially affect the following parts?
Documentation Status
Doc - TODODoc - DoneDoc - No Need