Hi there,

I’m trying to collect metrics for a Python web application. I’ve instrumented my app using the metric classes and utilities exported by the Python client (e.g. Gauge and timers) to collect a few basic HTTP request metrics. As the application is served using Gunicorn with multiple worker processes, I’ve followed the multiprocess instructions from the docs. This all works perfectly fine!

I’d also like to expose some metrics based on database records (e.g. number of users) or queue lengths (e.g. number of queued tasks). In order to do that, I have implemented a few custom collectors that fetch data from data stores and return Prometheus metrics.

As the docs state that the multiprocess mode doesn’t support custom collectors, I’m currently running a sidecar service to export metrics using my custom collectors:

• app service: uses MultiProcessCollector to expose request metrics from multiple worker processes
• prom-collector service: runs a single process and exposes only metrics from the databases, queues using custom collectors

I was wondering whether this is the best/recommend way to handle this use case? I had a look at the issues/discussion in this repository and searched other resources on the web, but haven’t found any recommendations/guidance for this use case

Is the fact that custom collectors are not supported in a multi-process environment a fundamental limitation or is there a possibility to make this work at some point (at least if the metrics collected by custom collectors are not dependent on the current process)?

Thanks a lot!