Change DB counting mechanism

Code Issues Proposed changes

The mysql performance_schema method for counting per-database queries
is very heavyweight in that it requires full logging (in a table) of
every query. We do hundreds of thousands in the course of a tempest
run, which ends up creating its own performance problem.
This changes the approach we take, which is to bundle a very tiny
sqlalchemy plugin module which counts just what we care about in
a special database.
It is more complex than just enabling the features in mysql, but it
is a massively smaller runtime overhead. It also provides us the
opportunity to easily zero the counters just before a tempest run.
Change-Id: I361bc30bb970cdaf18b966951f217862d302f0b9

This commit is contained in:

Dan Smith

2022年04月28日 12:34:38 -07:00

parent d450e146cc

commit fe52d7f0a8

6 changed files with 173 additions and 11 deletions

Show all changes Ignore whitespace when comparing lines Ignore changes in amount of whitespace Ignore changes in whitespace at EOL

Download Patch File Download Diff File Expand all files Collapse all files

6
tools/get-stats.py

View File

@@ -83,13 +83,11 @@ def get_processes_stats(matches):

def get_db_stats(host, user, passwd):

dbs = []

db = pymysql.connect(host=host, user=user, password=passwd,

database='performance_schema',

database='stats',

cursorclass=pymysql.cursors.DictCursor)

with db:

with db.cursor() as cur:

cur.execute(

'SELECT COUNT(*) AS queries,current_schema AS db FROM '

'events_statements_history_long GROUP BY current_schema')

cur.execute('SELECT db,op,count FROM queries')

for row in cur:

dbs.append({k: tryint(v) for k, v in row.items()})

return dbs

Change DB counting mechanism

6 tools/get-stats.py Unescape Escape View File

6
tools/get-stats.py

View File