Handling parsing failure in Scala without exceptions

Question

I have a Scala (Play!) application that must get and parse some data in JSON from an external service. I want to be able to gently handle failure in the response format, but it is becoming messy. What I am doing right now is, for instance,

def columnsFor(json: JsValue): Option[Map[String, String]] =
 try {
 val cols = (json \ "responseBody" \ "columns") match {
 case JsArray(xs) => xs map { col =>
 ((col \ "leafid").as[String], (col \ "name").as[String])
 }
 case _ => throw new Exception("Response unparsable")
 }
 Some(cols.toMap)
 }
 catch {
 case e: Exception => None
 }

That is, I go parsing optimistically and throw an exception whenever I find something that does not match my expected format (the methods as[String] also raise exceptions when they do not find strings). Then I recover from the exception and give back a None.

This approach is working and allows me to go from a potentially broken textual response to some type-safe data.

The problem is that having all these try..catch blocks looks ugly and messy. In theory it would be nicer to use optional types during all parsing steps. But then all my data structures become messy, as at each level you must have Options. When the data is deeply nested, I do not know how to handle failure at all levels. Moreover, I do not want to return something like an Option[Map[Option[String], Option[String]], but just an Option[Map[String, String]].

Is there some better/more idiomatic way to handle failures during parsing?

EDIT

I try to make the question more clear. Say I am expecting a response like

{
 "foo": [1, 2, 3],
 "bar": [2, 5]
}

I want to be able to parse it; taking failure into account I want to get something of type Option[Map[String, List[Int]]]. Now the problem is that parsing errors can appear at any level. Maybe I get a map with string keys, but its values are not lists. Or maybe the values are lists, but the content of the lists are strings instead of ints. And so on.

Whenever I want to parse a value, say as a string, I can choose between x.as[String] and x.asOpt[String]. The former will throw an exception when dealing with wrong input, while the latter returns a Option[String]. Similarly, when dealing with an array, I can do pattern matching like

x match {
 case JsArray(xs) => // continue parsing
 case _ => // deal with failure
}

I can deal with failure returning None or with an exception.

If I choose to use exceptions all along, then I can just wrap everything inside a try..catch and be sure to have handled all failures. Otherwise, my problem is that at any level, I have to put an option. So I have to

1. get something with an awful type like Option[Map[Option[String], Option[List[Option[Int]]]]]
2. find some way to flatten this all the way down to turn into the more manageable Option[Map[String, List[Int]]]

Of course, thing become even worse as soon as the JSON gets more nested.

• HEX: and become more worse if you need async call to the database with Future() =)

Answer 1

Having monads in your data structures is not messy, they allow you always to flatten their content during nested calculation steps. A simple definition of them can be:

trait Monad[A] {
 def map[B](f: A => B): Monad[B]
 def flatMap[B](f: A => Monad[B]): Monad[B]
}

Because Option is a monad you do not have to work with nested types:

scala> def inc(i: Int) = Option(i+1)
inc: (i: Int)Option[Int]
scala> val opt = inc(0)
opt: Option[Int] = Some(1)
scala> val nested = opt map inc
nested: Option[Option[Int]] = Some(Some(2))
scala> val flattened = opt flatMap inc
flattened: Option[Int] = Some(2)

Thus, there should never be a reason to nest monads deeply. Scala also has the for-comprehension which automatically decomposes monads:

scala> :paste
// Entering paste mode (ctrl-D to finish)
val none = for {
 o1 <- inc(0)
 o2 <- inc(o1)
 o3 <- inc(o2)
} yield o3
val some = for {
 o1 <- inc(0)
 o2 <- inc(o1)
 o3 <- inc(o2-1)
} yield o3
// Exiting paste mode, now interpreting.
none: Option[Int] = None
some: Option[Int] = Some(2)

In order to work with exceptions scala.util.Try is introduced in 2.10:

scala> import scala.util.{Try, Success, Failure}
import scala.util.{Try, Success, Failure}
scala> def err: Int = throw new RuntimeException
err: Int
scala> val fail = Try{err} map (_+1)
fail: scala.util.Try[Int] = Failure(java.lang.RuntimeException)
scala> val succ = Try{0} map (_+1)
succ: scala.util.Try[Int] = Success(1)

Nevertheless, Try should only used when you have to work with exceptions. When there is no special reason, exceptions should be avoided - they are not here to control the control flow but to tell us that an error occurred which normally can not be handled. They are some runtime thing - and why to use runtime things in a statically typed language? When you use monads the compiler always enforces you to write correct code:

scala> def countLen(xs: List[String]) = Option(xs) collect { case List(str) => str } map (_.length)
countLen: (xs: List[String])Option[Int]
scala> countLen(List("hello"))
res8: Option[Int] = Some(5)
scala> countLen(Nil)
res9: Option[Int] = None
scala> countLen(null) // please, never use null in Scala
res13: Option[Int] = None

With collect you ensure that you don't throw an exception during matching the contents. After that with map one can operate on the String and doesn't care anymore if an empty or a full list is passed to countLen.

Now, look at the documentation of Option, there are more useful methods which allow safe error handling. The only thing to keep in mind is not to use method get or a pattern match on the monad during calculation:

scala> Some(3).get
res10: Int = 3
scala> None.get // this throws an exception
java.util.NoSuchElementException: None.get
// this looks ugly and does not safe you from anything because it is equal
// to a null check
anyOption match {
 case Some(e) => // no error
 case None => // error
}

Now you may ask how to get the contents of a monad? That is a good question and the answer is: you won't. When you use a monad you say something like: Hey, I'm not interested if my code threw an error or if all worked fine. I want that my code works up to end and then I will look what's happened. Pattern matching on the content of a monad (means: accessing their content explicitly) is the last thing which should be done and only if there is no other way any more to go further with control flow.

There a lot of monads available in Scala and you have the possibility to easily create your own. Option allows only to get a notification if something happened wrong. If you wanna have the exact error message you can use Try (for exceptions) or Either (for all other things). Because Either is not really a monad (it has a Left- and a RightProjection which are the monads) it is unhandy to use. Thus, if you want to effectively work with error messages or even stack them you should take a look at scalaz.Validation.

EDIT

To address your edit, it seems that you never know the type you can get. This will complicate things a lot since you have to check the type of each element explicitly. I don't think it is possible to do this in a clean way without the use of a library which can do the typechecks for you. I suggest to take a look at some Scalaz or Shapeless code.

Nevertheless is is far easier to do such type checks in the parser and not in the AST traverser. Thus, I suggest using a JSON parser which can handle a type format given by the user and returns an error if it finds some unexpected content. I don't know if Play! can parse things like the following (which can be done in lift-json):

val json = """{ "foo": [1, 2, 3], "bar": [2, 5] }"""
class Content(foo: List[Double], bar: List[Double])
parse(json).extract[Content]

• Andrea: I think I may have phrased the question poorly. The problem with nesting is not that I have Option[Option[String]]; I know I can flatten it. The problem is if I expect my Json return data to be already nested, say of type Map[String, List[Int]]. What I want to avoid is having to deal with a Option[Map[Option[String], Option[List[Option[Int]]]]. You see, it becomes messy very easily. And I have a None anywhere, this means the parsing has failed and I jsut want a None at the outer level. I do not know how to deal with this kind of nested flattening.
• kiritsuku: @Andrea: Can you please edit your question and show an example how you get such an awful type like the one mentioned above? I do not understand how you get such a thing.
• Andrea: I have made the edit, I hope now it is more clear
• kiritsuku: @Andrea: I did a small edit.
• Andrea: Thank you for the edit. Probably a more capable library like lift-json is the way to go. Just to be sure, though, I actually know the types I want to get. What I do not know is whether the JSON conforms to the format - in theory it should, but I am adding error handling for a reason! :-)

Answer 2

1. You can use Try:

   val cols = Try {(json \ "responseBody" \ "columns")}
   

   you can match it:

   cols match {
    case Success(value) => ???
    case Failure(exception) => 
    println(exception.getMessage)
   }
   

2. You can convert it to Option to avoid exception

   val cols = Try {(json \ "responseBody" \ "columns")}.toOption
   

   It gives None or Some(value).

   cols match {
   case Some(value) => ???
   case None => ??
   }
   

3. You can use asOpt

   val cols = (json \ "responseBody" \ "columns").asOpt[String].getOrElse("") 
   

   OR

   val cols = (json \ "responseBody" \ "columns").asOpt[String].orNull