This is a proposal to add some more information to the SHOW PROCESSLIST output.
It adds a counter for each active partition. A partition gets removed from the list when it finishes.
This is how it looks:
$ mysql -u root -h 127.0.0.1 -P 3306 gitbase --execute "show processlist"
+------+------+-----------------+---------+---------+------+------------------------------------------------------------------+-----------------------+
| Id | User | Host | db | Command | Time | State | Info |
+------+------+-----------------+---------+---------+------+------------------------------------------------------------------+-----------------------+
| 395 | root | 127.0.0.1:35250 | gitbase | query | 2 | cangallo(45/?), commits(0/3), octoprint-tft(45/?), upsilon(45/?) | select * from commits |
| 396 | root | 127.0.0.1:35250 | gitbase | query | 0 | running | show processlist |
+------+------+-----------------+---------+---------+------+------------------------------------------------------------------+-----------------------+
$ mysql -u root -h 127.0.0.1 -P 3306 gitbase --execute "show processlist"
+------+------+-----------------+---------+---------+------+------------------------------+-----------------------+
| Id | User | Host | db | Command | Time | State | Info |
+------+------+-----------------+---------+---------+------+------------------------------+-----------------------+
| 395 | root | 127.0.0.1:35256 | gitbase | query | 11 | commits(2/3), upsilon(229/?) | select * from commits |
| 398 | root | 127.0.0.1:35256 | gitbase | query | 0 | running | show processlist |
+------+------+-----------------+---------+---------+------+------------------------------+-----------------------+
$ mysql -u root -h 127.0.0.1 -P 3306 gitbase --execute "show processlist"
+------+------+-----------------+---------+---------+------+------------------------------------------------------------------------------------------------------------+--------------------------------------------------------------------+
| Id | User | Host | db | Command | Time | State | Info |
+------+------+-----------------+---------+---------+------+------------------------------------------------------------------------------------------------------------+--------------------------------------------------------------------+
| 173 | root | 127.0.0.1:35236 | gitbase | query | 65 | SquashedTable(commits, commit_blobs, blobs)(0/3), cangallo(1289/?), octoprint-tft(1280/?), upsilon(1281/?) | select * from commits natural join commit_blobs natural join blobs |
| 393 | root | 127.0.0.1:35236 | gitbase | query | 0 | running | show processlist |
+------+------+-----------------+---------+---------+------+------------------------------------------------------------------------------------------------------------+--------------------------------------------------------------------+
$ mysql -u root -h 127.0.0.1 -P 3306 gitbase --execute "show processlist"
+------+------+-----------------+---------+--------------+------+------------------------------+---------------------------------------------------------------------------------------------------------+
| Id | User | Host | db | Command | Time | State | Info |
+------+------+-----------------+---------+--------------+------+------------------------------+---------------------------------------------------------------------------------------------------------+
| 395 | root | 127.0.0.1:35270 | gitbase | create_index | 8 | commits(2/3), upsilon(160/?) | CREATE INDEX commits_hash_idx ON commits USING pilosa (repository_id, commit_hash) WITH (async = false) |
| 402 | root | 127.0.0.1:35270 | gitbase | query | 0 | running | show processlist |
+------+------+-----------------+---------+--------------+------+------------------------------+---------------------------------------------------------------------------------------------------------+
The new output can be used to notice when for some reason a repository takes more time than others to finish.
Because there aren't totals, the counter of each partition cannot be used to get an estimate of the progress made. Right now its only function is to let the user know that the iterator is working.
I am concerned with the onNext function, because I'm not sure if updating the progress for every row will have an impact on performance. I'm working on testing this, but so far gitbase-regression has been giving me unreliable times.
If we removed onNext we could still benefit from having the name of the active partitions, maybe with a (?/?) progress.
Another generic improvement would be to have a way to get the total number of rows for a table. Maybe implementing a new interface. I didn't explore this possibility because as far as I know in gitbase we won't be able to use this. But it may be an interesting generic addition later for go-mysql-server itself.
This is a proposal to add some more information to the
SHOW PROCESSLISToutput.It adds a counter for each active partition. A partition gets removed from the list when it finishes.
This is how it looks:
The new output can be used to notice when for some reason a repository takes more time than others to finish.
Because there aren't totals, the counter of each partition cannot be used to get an estimate of the progress made. Right now its only function is to let the user know that the iterator is working.
I am concerned with the
onNextfunction, because I'm not sure if updating the progress for every row will have an impact on performance. I'm working on testing this, but so fargitbase-regressionhas been giving me unreliable times.If we removed
onNextwe could still benefit from having the name of the active partitions, maybe with a(?/?)progress.Another generic improvement would be to have a way to get the total number of rows for a table. Maybe implementing a new interface. I didn't explore this possibility because as far as I know in gitbase we won't be able to use this. But it may be an interesting generic addition later for go-mysql-server itself.