How do I asynchronously download a web page in F# and catch errors

Question

I've been trying to figure out the idiomatic F# way to download a webpage asynchronously and handle any errors/HTTP failure codes, I think this is my closest attempt so far but I'm getting a type error on the Choice1Of2 line

I'd love to

a) Understand why this fails (I'm still learning F#)
b) Know if this is the right approach/how to make this work or if I'm completely on the wrong track here

FS0001 This expression was expected to have type
 'Async<Choice<string,exn>>' 
but here has type
 'Choice<'a,'b>'

 let fetchAsync url = async {
 return! Async.Catch(async {
 let! str = Http.AsyncRequestString(url)
 return str })
 }
 let result = fetchAsync "http://www.example.bad"
 match result with 
 | Choice1Of2 v -> logger.LogInformation("worked")
 | Choice2Of2 ex -> logger.LogInformation("failed") 

Answer 1

a) Why it fails

Your fetchAsync function returns an Async<Choice<_>> and you pattern match as if the function only returns Choice<_>. Unfortunately you can't pattern match on Async because that would be a blocking operation, exactly what Async tries move away from.

b) Idiomatic way of dealing with these things.

What you can do, however, is stay within the Async context and handle the failure within. F# provides (at least) two common ways to deal with this. E.g. you can use the async helper functions that allow you to write a pipeline:

let fetchAsyncPipeline (url: string) =
 FSharp.Data.Http.AsyncRequestString(url)
 |> Async.Catch
 |> Async.map (function
 | Choice1Of2 v -> Ok v
 | Choice2Of2 e -> Error e.Message)

Unfortunately, Async.map is not yet included. But you can define it for yourself like this:

namespace global
[<RequireQualifiedAccess>]
module Async =
 let map f xA =
 async {
 let! x = xA
 return f x
 }

Or, you can use F#'s computation expressions that provide syntactic sugar to write the above in a more imperative style:

let fetchAsyncCe (url: string) =
 async {
 try
 return!
 FSharp.Data.Http.AsyncRequestString(url)
 |> Async.map Ok
 with
 | e -> return Error e.Message
 }

In both these solutions, I converted the exception to F#'s result type, which I personally find the nicest way to deal with errors.

Finally, as indicated by brianberns, your expression is only wrapped in the Async type. But unlike C#'s Task, async computations represent the program how to calculate something, but that program has not yet started and you have to explicitly run the asynchronous operation. One of the ways to do this is to use Async.RunSynchronously:

Async.RunSynchronously (fetchAsyncCe "https://fsharpforfunandprofit.com/")

PS: You probably came across Scott Wlaschin's excellent F# for fun and profit, but if you didn't you should check it out. Personally, I've found it the best resource to teach you F# and functional programming in general.

• David Hayes: What is the Async.map call? I can't seem to find it?
• Tom Moers: Well, that's embarrassing. I've been using it for so long now that I forgot I defined it myself. I updated my answer with the definition of the function.

Answer 2

The reason this doesn't compile is because you're never actually running the computation created by fetchAsync. Something like this should work instead:

let fetchAsync url =
 Http.AsyncRequestString(url)
 |> Async.Catch
 |> Async.RunSynchronously

Answer 3

After a few more hours and a lot of coffee I have this

let fetchAsync2 (url: string) =
async { return! Async.Catch(Http.AsyncRequestString(url)) }
|> Async.RunSynchronously
let result = fetchAsync2 url
match result with
| Choice1Of2 html -> html
| Choice2Of2 error -> error.Message

This code works the way I want it to but I'll leave the question open in case anyone has a better solution

Merging my version and the one provided by @brianberns I think I like this one best?

let fetchAsync3 (url: string) =
Http.AsyncRequestString(url)
|> Async.Catch 
|> Async.RunSynchronously
let result = fetchAsync3 url
match result with
| Choice1Of2 html -> html
| Choice2Of2 error -> error.Message

• Brian Berns: One small comment: For any computation builder, cb, the expressions cb { return! func() } and cb { let! foo = func(); return foo } are the same as just func() alone.