Ceph latency monitoring #8

fnetX commented 2023年05月25日 14:01:49 +02:00

We have the default Ceph dashboard and Grafana performance metrics of Ceph, but they do not mean much if we don't know what to look for.

The most interesting metric for Codeberg would be the average and maximum latency for requests on the CephFS filesystem itself (as seen by e.g. Git operations). However, I don't know if we can obtain this from the existing metrics.

The closest we get is probably the OSD Latencies metrics, but they might or might not be super relevant. (Also note that read performance is most important for us).

Any pointers on where to configure and improve the latency monitoring are very welcome. Maybe I'm missing something?

We have the default Ceph dashboard and Grafana performance metrics of Ceph, but they do not mean much if we don't know what to look for. The most interesting metric for Codeberg would be the average and maximum latency for requests on the CephFS filesystem itself (as seen by e.g. Git operations). However, I don't know if we can obtain this from the existing metrics. The closest we get is probably the OSD Latencies metrics, but they might or might not be super relevant. (Also note that read performance is most important for us). Any pointers on where to configure and improve the latency monitoring are very welcome. Maybe I'm missing something?

yoctozepto commented 2023年08月30日 14:39:54 +02:00

As you are interested in the client experience, then you should measure this on the client. You would usually capture the wait times with a tool like sar and stream this data to a central location for metric aggregation so see what is happening with the actual experience (and when).

As you are interested in the client experience, then you should measure this on the client. You would usually capture the wait times with a tool like `sar` and stream this data to a central location for metric aggregation so see what is happening with the actual experience (and when).