F#: Handling web exceptions

Question

I am new to programming and F# is my first language.

Here are the relevant snippets of my code:

let downloadHtmlToDiskAsync (fighterHtmlDirectory: string) (fighterBaseUrl: string) (fighterId: int) = 
 let fighterUrl = fighterBaseUrl + fighterId.ToString()
 try 
 async {
 let! html = fetchHtmlAsync fighterUrl
 let fighterName = getFighterNameFromPage html
 let newTextFile = File.Create(fighterHtmlDirectory + "\\" + fighterId.ToString("00000") + " " + fighterName.TrimEnd([|' '|]) + ".html")
 use file = new StreamWriter(newTextFile) 
 file.Write(html) 
 file.Close()
 }
 with
 :? System.Net.WebException -> async {File.AppendAllText("G:\User\WebScraping\Invalid Urls.txt", fighterUrl + "\n")}
let downloadFighterDatabase (directoryPath: string) (fighterBaseUrl: string) (beginningFighterId: int) (endFighterId: int) =
 let allFighterIds = [for id in beginningFighterId .. endFighterId -> id]
 allFighterIds
 |> Seq.map (fun fighterId -> downloadHtmlToDiskAsync directoryPath fighterBaseUrl fighterId)
 |> Async.Parallel
 |> Async.RunSynchronously

I have tested the functions fetchHtmlAsync and getFighterNameFromPage using F# Interactive. They both work fine.

When I build and run the solution, however, I get the following error message:

An unhandled exception of type 'System.Net.WebException' occurred in FSharp.Core.dll Additional information: The remote server returned an error: (404) Not Found.

What went wrong? What changes should I make?

Answer 1

Put your try with inside the async.

let downloadHtmlToDiskAsync (fighterHtmlDirectory: string) (fighterBaseUrl: string) (fighterId: int) = 
 let fighterUrl = fighterBaseUrl + fighterId.ToString()
 async {
 try
 let! html = fetchHtmlAsync fighterUrl
 let fighterName = getFighterNameFromPage html
 let newTextFile = File.Create(fighterHtmlDirectory + "\\" + fighterId.ToString("00000") + " " + fighterName.TrimEnd([|' '|]) + ".html")
 use file = new StreamWriter(newTextFile) 
 file.Write(html) 
 file.Close()
 with
 :? System.Net.WebException -> File.AppendAllText("G:\User\WebScraping\Invalid Urls.txt", fighterUrl + "\n")
 }

• albertjan: haha yeah I was just commenting the exact same thing on your answer
• M.Y. Babt: Thanks, albertjan! It turns out that I was an idiot... I originally wrote my code the way you did, but because I forgot to rebuild my solution, I kept getting an error message, which prompted me to start making unnecessary and inaccurate changes. Now I have just rebuilt, and everything works great. Thanks again for your help. :-)
• albertjan: That happens to the best of us :)