fe52d7f0a88de2dc330923cf6cf52c83ccb92bd6
5 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
|
Dan Smith
|
fe52d7f0a8 |
Change DB counting mechanism
The mysql performance_schema method for counting per-database queries is very heavyweight in that it requires full logging (in a table) of every query. We do hundreds of thousands in the course of a tempest run, which ends up creating its own performance problem. This changes the approach we take, which is to bundle a very tiny sqlalchemy plugin module which counts just what we care about in a special database. It is more complex than just enabling the features in mysql, but it is a massively smaller runtime overhead. It also provides us the opportunity to easily zero the counters just before a tempest run. Change-Id: I361bc30bb970cdaf18b966951f217862d302f0b9 |
||
|
Zuul
|
85c2999e27 | Merge "Tolerate missing deps in get-stats.py" | ||
|
Dan Smith
|
1b601c7b1e |
Tolerate missing deps in get-stats.py
In order to run on systems where not all requirements are present, we should be tolerant of missing external dependencies, such as psutil and pymysql. Print a warning (to stderr) and just leave out those stats in that case. Also make running the stats collector use ignore_errors:yes to avoid failures in the future. I think the stats is not critical enough to fail a job for bugs like this. Related-Bug: #1970195 Change-Id: I132b0e1f5033c4f109a8b8cc776c0877574c4a49 |
||
|
Harald Jensås
|
bab0c92103 |
Use tryint() for stats value
In some cases the value is [not set], in this case the conversion to integer does not work. Closes-Bug: #1970431 Change-Id: I74df7d8bc9f5cbe0709a6471cf7639caea0b58e8 |
||
|
Dan Smith
|
c2772c2984 |
Gather performance data after tempest
This makes us gather a bunch of consistent statistics after we run tempest that can be use to measure the impact of a given change. These are stable metrics such as "number of DB queries made" and "how much memory is each service using after a tempest run." Note that this will always run after devstack to generate the JSON file, but there are two things that control its completeness: - MYSQL_GATHER_PERFORMANCE must be enabled to get per-db stats - Unless tls-proxy is enabled, we will only get API stats for keystone Change-Id: Ie3b1504256dc1c9c6b59634e86fa98494bcb07b1 |