Multiple simultaneous requests to the same http host

Question

I have a C# desktop application, which needs to make multiple simultaneous http requests to the same web server. Here's the test I performed to check whether my requests actually do happen simultaneously:

1. On the web server, created a test page that sleeps for 3 seconds (simulates some long-running task), then returns current date/time. Here's the code (VB.Net):

   <%system.threading.thread.sleep(3000)%><%=now%>

2. In the C# app, I have a function MakeRequest() which uses System.Net.Http.HttpClient to make a web request, and returns the response as string.

3. Then in the C# app there's a function invoked by a button click, that calls MakeRequest() multiple times asynchronously:

   var responses = await Task.WhenAll(MakeRequest(), MakeRequest(), MakeRequest());

In the above example, MakeRequest() is called 3 times. What I see on the web server when I monitor the Requests/sec performance counter, is that it gets 2 requests, and then 3 sec later 1 more request. That's by design, because the default value for System.Net.ServicePointManager.DefaultConnectionLimit is 2, so my C# app can only send 2 requests at a time, though I asked for 3. Overall time the C# app took to complete the button click was 6 sec.

Now in the C# app, I set ServicePointManager.DefaultConnectionLimit = 1000000. This time Requests/sec on the web server shows 3, and the button click in the C# app completes in 3 seconds. All good so far.

Next I change the C# app to make 60 simultaneous calls rather than 3. That's where things get interesting. When I click the button, I expect to see Requests/sec on the web server to spike to 60, and the C# app to complete the button click in 3 sec. What I get instead is web server Requests/sec showing something like 5, after 3 seconds 5 more, etc., until all 60 requests have been made. When I click the button again, the same picture except for Requests/sec spike to 10 or 15, and the button click obviously completes faster. Click again - this time Requests/sec show two spikes to 30, and the button click completes in 6 sec. Subsequent button clicks result in the Requests/sec spiking first to 40 then to 20 (for the total of 60 requests), then 50 and 10, then maybe 55 and 5, and eventually my C# app starts sending all 60 requests in one go. The button click completes in 3 sec, and Requests/sec on the web server shows one spike to 60. If I continue pushing the button, I consistently get all 60 requests being made simultaneously.

But that's not all. If I stop pushing the button, my C# app seems to "forget" it's previous achievement. I do not restart the app, just stop pushing the button for a minute, and on the next push I go back to 5 requests at a time, and the above scenario would repeat if I keep pushing the button continuously.

I also performed the above test from MVC - just copied and pasted the code from the C# app to a MVC page. Got exactly the same result.

Can anyone please explain what's going on? Thank you

---

Here's the code of the C# app. It's a WinForms app, and the code lives inside a Form class:

<!-- language: lang-c# -->
private HttpClient _httpClient = new HttpClient();
private async void button1_Click(object sender, EventArgs e)
{
 ServicePointManager.DefaultConnectionLimit = 1000000;
 var sw = Stopwatch.StartNew();
 var responses = await Task.WhenAll(MakeRequest(), MakeRequest(), MakeRequest());
 sw.Stop();
 var msg = String.Join("\r\n", responses) + "\r\n\r\nTime Elapsed: " + sw.ElapsedMilliseconds;
 MessageBox.Show(msg);
}
private async Task<string> MakeRequest()
{
 using (var message = await _httpClient.GetAsync("http://TestServer/TestPage.aspx"))
 {
 return await message.Content.ReadAsStringAsync();
 }
}

Answer 1

Two reasons:

1. Client Windows SKUs limit the number of concurrent IIS requests to 10.
2. Apparently, your client uses synchronous IO or is otherwise blocking. It takes a while to get the thread pool warmed up. Threads are injected only over time. Either use async IO and be non-blocking or make sure that the threads you need are actually there. Regarding "forgetting": Idle pool threads are retired over time. Your tests therefore are timing sensitive. Very unreliable, don't put this into production.

• Andrew: thank you for your reply. 1. Yes I ran into this problem, so moved the web page to an IIS on Win Server 2008. In fact I tried to run the MVC page that sends the requests on Server 2008 too - the same result as running from Win 10. 2. Not sure why my client code would by synchronous. I posted the code in the original question. Can you suggest how I should modify it?
• usr: That code is fine. Can you post the code that does 60 requests all at once? The ASP.NET thread pool will be overloaded, btw, with all those synchronous waits. Use an async mvc action and use await Task.Delay.
• Andrew: The code that does 60 requests all at once is the same as posted above, except in the line var responses = await Task.WhenAll(MakeRequest(), MakeRequest(), MakeRequest()); you repeat MakeRequest() 60 times. Regarding ASP.Net thread pool overload, I do no think that happens because when my client app eventually sends 60 request simultaneously, the web server successfully processes all of them simultaneously. Check here: link, section "Configure ASP.NET 4 MaxConcurrentRequests for IIS 7.5/7.0 Integrated mode"
• usr: "when my client app eventually sends" How do you know they have not been sent in all cases but the server is slow to take them on? There are many perf counters, some misleading. Use an async mvc action and use await Task.Delay so that we can retire this guess of mine.
• Andrew: Ok, changing the server side to async pipeline did change something. I changed it to: <%System.Threading.Tasks.Task.WaitAll(System.Threading.Tasks.Task.Delay(3000))%><%=now%>. Now the fist button click sends 5 or so requests straight away, and then starts sending one request at a time. On the server Requests/sec first shows a small spike, then stays at 1 for ~ 50 sec. The 2nd and subsequent button clicks all result in Requests/sec spiking to 60. As before, if I wait for a minute, the client "forgets its achievement" and again sends ~5 requests first an then 1 at a time.