Http Request response time within C# application

Question

I have this method :

private string GetCheckResultFor(string url)
 {
 Stopwatch reveil = new Stopwatch();
 reveil.Start();
 HttpWebResponse response = null;
 url = Regex.Replace(url, @"\s+", "");
 if (!url.StartsWith("http") && !url.StartsWith("https")) url = "http://" + url;
 string res = "";
 try
 {
 HttpWebRequest request = (HttpWebRequest)HttpWebRequest.Create(url);
 request.Method = "GET";
 response = (HttpWebResponse)request.GetResponse();
 if (response.ResponseUri.ToString() == url) res = "No Action";
 else res = "Redirect to " + response.ResponseUri.ToString();
 }
 catch (WebException e)
 {
 if (e.Status == WebExceptionStatus.ProtocolError)
 {
 response = (HttpWebResponse)e.Response;
 res = "Errorcode: " + (int)response.StatusCode;
 }
 else
 {
 res = "Error: " + e.Status;
 }
 }
 finally
 {
 if (response != null)
 {
 response.Close();
 }
 }
 reveil.Stop();
 Debug.WriteLine("For this url : " + url + " = " + reveil.ElapsedMilliseconds + " ms");
 return res;
 }

It takes between 200 and 700 ms. I know it depends on internet connection speed but I need to improve it because I only need the reponse url. I have Three possiblities :

• Reponse Url = request Url ==> No action
• Redirect to another url
• Request Error

So How can I edit this snippet to reduce method response time ?

• RobH: request.Method = "GET"; -> request.Method = "HEAD"; might save you some time and should still work.

Answer 1

I found that a webclient with custom settings gave me the fastest result. Personally I was building a framework for web crawling & scraping, where you want it to do the requests quickly.

Might be an alternative to check out.

public class DefaultWebClient : WebClient {
 public DefaultWebClient() {
 ServicePointManager.ServerCertificateValidationCallback += (sender, certificate, chain, sslPolicyErrors) => true;
 ServicePointManager.DefaultConnectionLimit = Int32.MaxValue;
 ServicePointManager.Expect100Continue = false;
 }
 protected override WebRequest GetWebRequest(Uri address) {
 var request = base.GetWebRequest(address) as HttpWebRequest;
 if (request != null) {
 //This might work in your case, it didnt do the trick for me.
 //request.Method = WebRequestMethods.Http.Head;
 request.Proxy = null;
 request.Timeout = 5000;
 request.AutomaticDecompression = DecompressionMethods.Deflate | DecompressionMethods.GZip;
 }
 return request;
 }
}

This, in combination with retrieving the response url should do the trick.

 protected override WebResponse GetWebResponse(WebRequest request) {
 var response = base.GetWebResponse(request);
 if (response != null) {
 var responseUri = response.ResponseUri;
 // Redirect found.
 }
 else {
 // No Action.
 }
 return response;
 }

If you do a lot of requests to get the redirects, it might be a good idea to raise it as an event.