Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly
Sign up
Appearance settings

test: fix dbcrash and fee_estimation extended tests#1298

Open
delta1 wants to merge 3 commits intoElementsProject:master from
delta1:issues-1297
Open

test: fix dbcrash and fee_estimation extended tests #1298
delta1 wants to merge 3 commits intoElementsProject:master from
delta1:issues-1297

Conversation

@delta1
Copy link
Member

@delta1 delta1 commented Dec 1, 2023

It seems these tests have never been run for Elements, since the extended tests are not run in the CI process.

This PR fixes both of them, however there may be a further issue with feature_dbcrash.py running for an extremely long time which we are still in the process of testing.

#1297
#1296

cc @whitslack

Copy link
Member Author

delta1 commented Dec 8, 2023 •
edited
Loading

Force pushed to reduce the number of iterations in the dbcrash test, such that it still passes but in a "reasonable" amount of time (about 6 hours in my testing).

I have tested these 2 tests manually and they are now passing.

2023年12月08日T19:50:34.715000Z TestFramework (INFO): Tests successful
real 350m18.127s
user 348m12.078s
sys 1m56.746s

Copy link
Collaborator

jamesdorfman commented Apr 4, 2024 •
edited
Loading

@delta1 am I running the correct test?
It's not passing for me -- I see the following:

$ ./test/functional/feature_dbcrash.py 
2024年04月03日T20:32:29.724000Z TestFramework (INFO): Initializing test directory /tmp/bitcoin_func_test_ikctzo2y
2024年04月03日T21:35:24.776000Z TestFramework (INFO): Prepped 5001 utxo entries
2024年04月03日T21:36:00.556000Z TestFramework (INFO): Iteration 0, generating 2500 transactions [0, 0, 0]
2024年04月03日T23:10:26.156000Z TestFramework (INFO): Iteration 1, generating 2500 transactions [0, 0, 0]
2024年04月04日T00:45:22.052000Z TestFramework (INFO): Iteration 2, generating 2500 transactions [1, 0, 0]
2024年04月04日T02:20:44.177000Z TestFramework (INFO): Verifying utxo hash matches for all nodes
2024年04月04日T02:20:45.540000Z TestFramework (INFO): Restarted nodes: [1, 0, 0]; crashes on restart: 0
2024年04月04日T02:20:45.540000Z TestFramework (ERROR): Assertion failed
Traceback (most recent call last):
 File "/home/james/elements/test/functional/test_framework/test_framework.py", line 132, in main
 self.run_test()
 File "/home/james/elements/./test/functional/feature_dbcrash.py", line 285, in run_test
 assert self.crashed_on_restart > 0
 ^^^^^^^^^^^^^^^^^^^^^^^^^^^
AssertionError
2024年04月04日T02:20:45.600000Z TestFramework (INFO): Stopping nodes
2024年04月04日T02:20:45.856000Z TestFramework (WARNING): Not cleaning up dir /tmp/bitcoin_func_test_ikctzo2y
2024年04月04日T02:20:45.856000Z TestFramework (ERROR): Test failed. Test logging available at /tmp/bitcoin_func_test_ikctzo2y/test_framework.log
2024年04月04日T02:20:45.856000Z TestFramework (ERROR): 
2024年04月04日T02:20:45.857000Z TestFramework (ERROR): Hint: Call /home/james/byron-elements-clone/elements/test/functional/combine_logs.py '/tmp/bitcoin_func_test_ikctzo2y' to consolidate all logs
2024年04月04日T02:20:45.857000Z TestFramework (ERROR): 
2024年04月04日T02:20:45.857000Z TestFramework (ERROR): If this failure happened unexpectedly or intermittently, please file a bug and provide a link or upload of the combined log.
2024年04月04日T02:20:45.857000Z TestFramework (ERROR): https://github.com/ElementsProject/elements/issues
2024年04月04日T02:20:45.857000Z TestFramework (ERROR): 

$ git log
commit 5af2426c14fe62f7b60b84586221cc531251eac8 (HEAD -> issues-1297, origin/issues-1297)
Author: Byron Hambly <bhambly@blockstream.com>
Date: Fri Dec 1 09:19:34 2023 +0200
 test: fix dbcrash and fee_estimation extended tests
 
 reduces the number of iterations for the dbcrash test so that it "only"
 takes 6 hours

Copy link
Member Author

delta1 commented Apr 4, 2024 •
edited
Loading

@jamesdorfman yes you ran the right test. It seems I've reduced the values too much for it to fail (correctly) every time.

Bumped the number of iterations from 3 to 4, please test again. It will take closer to 8 hours now.

Copy link
Member Author

delta1 commented Apr 4, 2024

latest run:

> test/functional/feature_dbcrash.py
2024年04月04日T09:04:35.877000Z TestFramework (INFO): Initializing test directory /tmp/bitcoin_func_test_dybbv4yh
2024年04月04日T09:48:20.887000Z TestFramework (INFO): Prepped 5001 utxo entries
2024年04月04日T09:48:47.443000Z TestFramework (INFO): Iteration 0, generating 2500 transactions [0, 0, 0]
2024年04月04日T10:54:39.535000Z TestFramework (INFO): Iteration 1, generating 2500 transactions [0, 0, 0]
2024年04月04日T12:00:57.244000Z TestFramework (INFO): Iteration 2, generating 2500 transactions [1, 0, 0]
2024年04月04日T13:07:27.698000Z TestFramework (INFO): Iteration 3, generating 2500 transactions [1, 0, 0]
2024年04月04日T14:14:32.395000Z TestFramework (INFO): Verifying utxo hash matches for all nodes
2024年04月04日T14:14:32.482000Z TestFramework (INFO): Restarted nodes: [2, 0, 0]; crashes on restart: 1
2024年04月04日T14:14:32.482000Z TestFramework (WARNING): Node 1 never crashed during utxo flush!
2024年04月04日T14:14:32.482000Z TestFramework (WARNING): Node 2 never crashed during utxo flush!
2024年04月04日T14:14:32.536000Z TestFramework (INFO): Stopping nodes
2024年04月04日T14:14:32.840000Z TestFramework (INFO): Cleaning up /tmp/bitcoin_func_test_dybbv4yh on exit
2024年04月04日T14:14:32.840000Z TestFramework (INFO): Tests successful

approx 6 hours

Copy link
Collaborator

jamesdorfman commented Apr 5, 2024 •
edited
Loading

Hmm... seems to still be happening though:

[james@elements-workspace elements]$ ./test/functional/feature_dbcrash.py 
2024年04月04日T20:07:55.335000Z TestFramework (INFO): Initializing test directory /tmp/bitcoin_func_test_e8dp9mz5
2024年04月04日T21:11:27.390000Z TestFramework (INFO): Prepped 5001 utxo entries
2024年04月04日T21:12:03.369000Z TestFramework (INFO): Iteration 0, generating 2500 transactions [1, 0, 0]
2024年04月04日T22:47:36.411000Z TestFramework (INFO): Iteration 1, generating 2500 transactions [1, 0, 0]
2024年04月05日T00:22:26.239000Z TestFramework (INFO): Iteration 2, generating 2500 transactions [2, 0, 0]
2024年04月05日T01:57:35.443000Z TestFramework (INFO): Iteration 3, generating 2500 transactions [2, 0, 0]
2024年04月05日T03:33:23.923000Z TestFramework (INFO): Verifying utxo hash matches for all nodes
2024年04月05日T03:33:24.848000Z TestFramework (INFO): Restarted nodes: [3, 0, 0]; crashes on restart: 0
2024年04月05日T03:33:24.848000Z TestFramework (ERROR): Assertion failed
Traceback (most recent call last):
 File "/home/james/elements/test/functional/test_framework/test_framework.py", line 132, in main
 self.run_test()
 File "/home/james/elements/./test/functional/feature_dbcrash.py", line 285, in run_test
 assert self.crashed_on_restart > 0
 ^^^^^^^^^^^^^^^^^^^^^^^^^^^
AssertionError
2024年04月05日T03:33:24.913000Z TestFramework (INFO): Stopping nodes
2024年04月05日T03:33:25.220000Z TestFramework (WARNING): Not cleaning up dir /tmp/bitcoin_func_test_e8dp9mz5
2024年04月05日T03:33:25.220000Z TestFramework (ERROR): Test failed. Test logging available at /tmp/bitcoin_func_test_e8dp9mz5/test_framework.log
2024年04月05日T03:33:25.220000Z TestFramework (ERROR): 
2024年04月05日T03:33:25.221000Z TestFramework (ERROR): Hint: Call /home/james/elements/test/functional/combine_logs.py '/tmp/bitcoin_func_test_e8dp9mz5' to consolidate all logs
2024年04月05日T03:33:25.221000Z TestFramework (ERROR): 
2024年04月05日T03:33:25.221000Z TestFramework (ERROR): If this failure happened unexpectedly or intermittently, please file a bug and provide a link or upload of the combined log.
2024年04月05日T03:33:25.221000Z TestFramework (ERROR): https://github.com/ElementsProject/elements/issues
2024年04月05日T03:33:25.221000Z TestFramework (ERROR): 
[james@elements-workspace elements]$ git log
commit 1ed7f84f05b54234439fc85436bd2b8854353812 (HEAD, origin/issues-1297)

Copy link
Member Author

delta1 commented Apr 5, 2024 •
edited
Loading

strange, worked for me again. i'm going to bump up the iteration and run it again.

2024年04月04日T15:10:11.209000Z TestFramework (INFO): Initializing test directory /tmp/bitcoin_func_test_1c04byhm
2024年04月04日T15:53:58.589000Z TestFramework (INFO): Prepped 5001 utxo entries
2024年04月04日T15:54:27.777000Z TestFramework (INFO): Iteration 0, generating 2500 transactions [0, 0, 0]
2024年04月04日T17:00:24.418000Z TestFramework (INFO): Iteration 1, generating 2500 transactions [0, 0, 0]
2024年04月04日T18:06:32.628000Z TestFramework (INFO): Iteration 2, generating 2500 transactions [0, 0, 0]
2024年04月04日T19:12:57.430000Z TestFramework (INFO): Iteration 3, generating 2500 transactions [1, 1, 0]
2024年04月04日T20:19:31.617000Z TestFramework (INFO): Verifying utxo hash matches for all nodes
2024年04月04日T20:19:32.528000Z TestFramework (INFO): Restarted nodes: [1, 1, 0]; crashes on restart: 2
2024年04月04日T20:19:32.528000Z TestFramework (WARNING): Node 2 never crashed during utxo flush!
2024年04月04日T20:19:32.585000Z TestFramework (INFO): Stopping nodes
2024年04月04日T20:19:32.889000Z TestFramework (INFO): Cleaning up /tmp/bitcoin_func_test_1c04byhm on exit
2024年04月04日T20:19:32.889000Z TestFramework (INFO): Tests successful

Copy link
Member Author

delta1 commented Apr 5, 2024 •
edited
Loading

5 iterations

> time test/functional/feature_dbcrash.py
2024年04月05日T07:05:16.203000Z TestFramework (INFO): Initializing test directory /tmp/bitcoin_func_test_8_zlhs9q
2024年04月05日T07:48:56.506000Z TestFramework (INFO): Prepped 5001 utxo entries
2024年04月05日T07:49:24.107000Z TestFramework (INFO): Iteration 0, generating 2500 transactions [0, 0, 0]
2024年04月05日T08:55:19.380000Z TestFramework (INFO): Iteration 1, generating 2500 transactions [0, 0, 0]
2024年04月05日T10:01:41.702000Z TestFramework (INFO): Iteration 2, generating 2500 transactions [1, 0, 0]
2024年04月05日T11:08:10.640000Z TestFramework (INFO): Iteration 3, generating 2500 transactions [1, 0, 0]
2024年04月05日T12:15:03.874000Z TestFramework (INFO): Iteration 4, generating 2500 transactions [2, 0, 0]
2024年04月05日T13:21:56.336000Z TestFramework (INFO): Verifying utxo hash matches for all nodes
2024年04月05日T13:21:58.731000Z TestFramework (INFO): Restarted nodes: [3, 1, 0]; crashes on restart: 2
2024年04月05日T13:21:58.731000Z TestFramework (WARNING): Node 2 never crashed during utxo flush!
2024年04月05日T13:21:58.791000Z TestFramework (INFO): Stopping nodes
2024年04月05日T13:21:59.045000Z TestFramework (INFO): Cleaning up /tmp/bitcoin_func_test_8_zlhs9q on exit
2024年04月05日T13:21:59.045000Z TestFramework (INFO): Tests successful
test/functional/feature_dbcrash.py 22436.98s user 118.79s system 99% cpu 6:16:42.95 total

and

> time test/functional/feature_dbcrash.py
2024年04月05日T14:43:53.223000Z TestFramework (INFO): Initializing test directory /tmp/bitcoin_func_test_y3m7o8tu
2024年04月05日T15:27:36.512000Z TestFramework (INFO): Prepped 5001 utxo entries
2024年04月05日T15:28:06.834000Z TestFramework (INFO): Iteration 0, generating 2500 transactions [1, 0, 0]
2024年04月05日T16:33:52.073000Z TestFramework (INFO): Iteration 1, generating 2500 transactions [1, 0, 0]
2024年04月05日T17:40:09.012000Z TestFramework (INFO): Iteration 2, generating 2500 transactions [2, 0, 0]
2024年04月05日T18:46:39.010000Z TestFramework (INFO): Iteration 3, generating 2500 transactions [3, 1, 0]
2024年04月05日T19:53:15.120000Z TestFramework (INFO): Iteration 4, generating 2500 transactions [4, 1, 0]
2024年04月05日T21:00:19.098000Z TestFramework (INFO): Verifying utxo hash matches for all nodes
2024年04月05日T21:00:19.222000Z TestFramework (INFO): Restarted nodes: [4, 2, 0]; crashes on restart: 1
2024年04月05日T21:00:19.222000Z TestFramework (WARNING): Node 2 never crashed during utxo flush!
2024年04月05日T21:00:19.280000Z TestFramework (INFO): Stopping nodes
2024年04月05日T21:00:19.734000Z TestFramework (INFO): Cleaning up /tmp/bitcoin_func_test_y3m7o8tu on exit
2024年04月05日T21:00:19.734000Z TestFramework (INFO): Tests successful
test/functional/feature_dbcrash.py 22428.39s user 119.89s system 99% cpu 6:16:26.63 total

Copy link
Member Author

delta1 commented Apr 7, 2024

@jamesdorfman had a similar failure. bumped the iterations to 6 and will run a number of tests

Copy link

@delta1: It looks as though you're trying to tune a probabilistic test. You're never going to find a number that gives the desired result on every machine from the slowest single-core SoC to the fastest NUMA supercomputer. Instead, the test case should be fixed so that it is deterministic. If that is not possible, then the test case should be discarded.

Copy link
Member Author

delta1 commented Apr 8, 2024

I suspect there’s a threshold that works reliably like the Bitcoin test.

@delta1 delta1 force-pushed the issues-1297 branch 2 times, most recently from 9dc2afe to 98ad8c1 Compare April 9, 2024 07:35
JBetz added a commit to SequentiaSEQ/SEQ-Core-Elements that referenced this pull request May 2, 2024
JBetz added a commit to SequentiaSEQ/SEQ-Core-Elements that referenced this pull request May 3, 2024 
* Introduce CValue for distinguishing amounts from values
* Remove random character
* Define insertion operator for CValue
* Move CValue to separate module
* Remove consensus changes
* Remove CValue usage for fee rates, restrict to mempool
* More cleanup
* Use CValue for descendant and ancestor fee updates
* More mempool fixes
* Remove unused include
* Fix variable shadowing
* More conversions
* Factor in fee asset when validating package fees
* More conversions
* Convert dust relay fee from RFU
* More cleanup
* Cleanup
* Don't factor in asset in when calculating dust threshold
* Convert package fees
* Return CValue from CalculateExchangeValue
* Use CValue for reverse exchange rate conversion
* Update method documentation
* Rename currency conversion methods
* Fix typo
* Add justification for RFU to comments
* Use accessor instead of public instance variable unpacking CValue
* Use GetValue() in fee rate conversion
* Only apply dust check to outputs denominated in same asset as fee
* Interpret coin selection effective feerate in RFU
* Change GetModifiedFee() method to use RFU for summing with ancestor and descendant transactions
* Add functional test for "fee_rate" parameter in fundrawtransaction RPC and -mintxfee node configuration parameter
* Normalize fees to RFU during fee estimation
* Lint fixes
* More linting fixes
* More linting
* Apply method renams to fee amount conversions
* Add any asset fee rates test to test runner
* More amount conversion fixes
* Add compiler switch for currency constants
* Add new configuration flag to documentation
* Apply fixes to fee estimation functional test from ElementsProject/elements#1298
* Add tests for paytxfee parameter
* Cleanup
* Fix compiler flag
* Add tests for blockmintxfee
* Remove whitespace
* Add comments clarifying what's being demonstrated
* Remove nFeeValue, recompute as needed
* Revert "Remove nFeeValue, recompute as needed"
This reverts commit ab3a67b.
* Move No Coin configuration documentation to separate line
* Add constant for full name of currency atom
* Fix constant reference
* Fix typo
JBetz added a commit to SequentiaSEQ/SEQ-Core-Elements that referenced this pull request May 3, 2024 
* Introduce CValue for distinguishing amounts from values
* Remove random character
* Define insertion operator for CValue
* Move CValue to separate module
* Remove consensus changes
* Remove CValue usage for fee rates, restrict to mempool
* More cleanup
* Use CValue for descendant and ancestor fee updates
* More mempool fixes
* Remove unused include
* Fix variable shadowing
* More conversions
* Factor in fee asset when validating package fees
* More conversions
* Convert dust relay fee from RFU
* More cleanup
* Cleanup
* Don't factor in asset in when calculating dust threshold
* Convert package fees
* Return CValue from CalculateExchangeValue
* Use CValue for reverse exchange rate conversion
* Update method documentation
* Rename currency conversion methods
* Fix typo
* Add justification for RFU to comments
* Use accessor instead of public instance variable unpacking CValue
* Use GetValue() in fee rate conversion
* Only apply dust check to outputs denominated in same asset as fee
* Interpret coin selection effective feerate in RFU
* Change GetModifiedFee() method to use RFU for summing with ancestor and descendant transactions
* Add functional test for "fee_rate" parameter in fundrawtransaction RPC and -mintxfee node configuration parameter
* Normalize fees to RFU during fee estimation
* Lint fixes
* More linting fixes
* More linting
* Apply method renams to fee amount conversions
* Add any asset fee rates test to test runner
* More amount conversion fixes
* Add compiler switch for currency constants
* Add new configuration flag to documentation
* Apply fixes to fee estimation functional test from ElementsProject/elements#1298
* Add tests for paytxfee parameter
* Cleanup
* Fix compiler flag
* Add tests for blockmintxfee
* Remove whitespace
* Add comments clarifying what's being demonstrated
* Remove nFeeValue, recompute as needed
* Revert "Remove nFeeValue, recompute as needed"
This reverts commit ab3a67b.
* Move No Coin configuration documentation to separate line
* Add constant for full name of currency atom
* Fix constant reference
* Fix typo
JBetz added a commit to SequentiaSEQ/SEQ-Core-Elements that referenced this pull request May 3, 2024 
* Introduce CValue for distinguishing amounts from values
* Remove random character
* Define insertion operator for CValue
* Move CValue to separate module
* Remove consensus changes
* Remove CValue usage for fee rates, restrict to mempool
* More cleanup
* Use CValue for descendant and ancestor fee updates
* More mempool fixes
* Remove unused include
* Fix variable shadowing
* More conversions
* Factor in fee asset when validating package fees
* More conversions
* Convert dust relay fee from RFU
* More cleanup
* Cleanup
* Don't factor in asset in when calculating dust threshold
* Convert package fees
* Return CValue from CalculateExchangeValue
* Use CValue for reverse exchange rate conversion
* Update method documentation
* Rename currency conversion methods
* Fix typo
* Add justification for RFU to comments
* Use accessor instead of public instance variable unpacking CValue
* Use GetValue() in fee rate conversion
* Only apply dust check to outputs denominated in same asset as fee
* Interpret coin selection effective feerate in RFU
* Change GetModifiedFee() method to use RFU for summing with ancestor and descendant transactions
* Add functional test for "fee_rate" parameter in fundrawtransaction RPC and -mintxfee node configuration parameter
* Normalize fees to RFU during fee estimation
* Lint fixes
* More linting fixes
* More linting
* Apply method renams to fee amount conversions
* Add any asset fee rates test to test runner
* More amount conversion fixes
* Add compiler switch for currency constants
* Add new configuration flag to documentation
* Apply fixes to fee estimation functional test from ElementsProject/elements#1298
* Add tests for paytxfee parameter
* Cleanup
* Fix compiler flag
* Add tests for blockmintxfee
* Remove whitespace
* Add comments clarifying what's being demonstrated
* Remove nFeeValue, recompute as needed
* Revert "Remove nFeeValue, recompute as needed"
This reverts commit ab3a67b.
* Move No Coin configuration documentation to separate line
* Add constant for full name of currency atom
* Fix constant reference
* Fix typo
Mixa84 pushed a commit to SequentiaSEQ/SEQ-Core-Elements that referenced this pull request May 6, 2024 
* Introduce CValue for distinguishing amounts from values
* Remove random character
* Define insertion operator for CValue
* Move CValue to separate module
* Remove consensus changes
* Remove CValue usage for fee rates, restrict to mempool
* More cleanup
* Use CValue for descendant and ancestor fee updates
* More mempool fixes
* Remove unused include
* Fix variable shadowing
* More conversions
* Factor in fee asset when validating package fees
* More conversions
* Convert dust relay fee from RFU
* More cleanup
* Cleanup
* Don't factor in asset in when calculating dust threshold
* Convert package fees
* Return CValue from CalculateExchangeValue
* Use CValue for reverse exchange rate conversion
* Update method documentation
* Rename currency conversion methods
* Fix typo
* Add justification for RFU to comments
* Use accessor instead of public instance variable unpacking CValue
* Use GetValue() in fee rate conversion
* Only apply dust check to outputs denominated in same asset as fee
* Interpret coin selection effective feerate in RFU
* Change GetModifiedFee() method to use RFU for summing with ancestor and descendant transactions
* Add functional test for "fee_rate" parameter in fundrawtransaction RPC and -mintxfee node configuration parameter
* Normalize fees to RFU during fee estimation
* Lint fixes
* More linting fixes
* More linting
* Apply method renams to fee amount conversions
* Add any asset fee rates test to test runner
* More amount conversion fixes
* Add compiler switch for currency constants
* Add new configuration flag to documentation
* Apply fixes to fee estimation functional test from ElementsProject/elements#1298
* Add tests for paytxfee parameter
* Cleanup
* Fix compiler flag
* Add tests for blockmintxfee
* Remove whitespace
* Add comments clarifying what's being demonstrated
* Remove nFeeValue, recompute as needed
* Revert "Remove nFeeValue, recompute as needed"
This reverts commit ab3a67b.
* Move No Coin configuration documentation to separate line
* Add constant for full name of currency atom
* Fix constant reference
* Fix typo
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

AltStyle によって変換されたページ (->オリジナル) /