API request tracing with NextJS, Nginx and Django

Question

I’m trying to measure the exact time spent in each stage of my API request flow — starting from the browser, through Nginx, into Django, then the database, and back out through Django and Nginx to the client.

Essentially, I want to capture timestamps and time intervals for:

• When the browser sends the request

• When Nginx receives it

• When Django starts processing it

• Time spent in the database

• Django response time

• Nginx response time

• When the browser receives the response

Is there any Django package or recommended approach that can help log these timing metrics end-to-end? Currently I have to manually add timestamps in nginx conf file, django middleware, before and after the fetch call in the frontend. There is Datadog but for now I want a django lightweight solution.

Thanks!

Answer 1

django-silk

It is usefull in your case:

• lightweight
• native
• no heavy setup

Let me know you want a setup for that or you can figure out.

Answer 2

Additionally to the already recommended Django middleware, you can log the following nginx internal variables:

• $upstream_connect_time (the time taken to establish a connection with an upstream application server);
• $upstream_response_time (the time from establishing a connection to an upstream server until the last byte of the response body is received from that server);
• $request_time (represents the total time elapsed for a request, from the first byte read from the client to the last byte of the response body sent).

• Smruti Ranjan Sahu: For the nginx part, I am currently using the variables you have mentioned. But I am looking for a package or module that would kind of do both - django profiling as well as nginx monitoring / logging since going through the nginx access log, parsing the variables etc is a bit tedious if the number of requests is large. Are you aware of any such package / module?
• Ivan Shatsky: I don't think such a package/module exists. However, if you want to track a relationship between the nginx access log entries and django-silk reports, you can try to utilize the $request_id nginx variable, adding it to nginx log and passing it to the Django app (as a WSGI environment variable using uwsgi_param or as a custom HTTP request header using proxy_set_header, depending on your backend application server architecture).