Showing posts with label Scala. Show all posts
Showing posts with label Scala. Show all posts

Friday, October 5, 2012

Scala-IO Core: Unmanaged Resources

The main design of Scala-IO is around automatic closing of resources each time a resource is accessed in order to ensure that a programmer cannot unintentionally leave resources open in the face of exceptions or other unexpected situations. However, there are cases where the Scala-IO API is desired but the resource management is undesired. The classic case is of reading or writing to System.in and out. Thus Unmanaged resources exist to satisfy this use-case.

Since unmanaged resources is a less common use-case there is not a factory object like there is for normal managed Resources. Instead certain objects can be converted to unmanaged resources using the JavaConverters implicit methods as follows:

Wednesday, September 26, 2012

Scala-IO Core: To Resource Converters

In order to simplify integration with existing libraries, most commonly Java libraries, Scala-IO provides a JavaConverters object with implicit methods that add as*** methods (asInput, asOutput, asSeekable, etc...) to several types of objects. It is the same pattern as in the scala.collection.JavaConverters object.

These methods can be used instead of the Resource.from*** methods to provide a slightly nicer appearing code.

There is one warning. When using JavaConverters, instead of Resource.from*** for creating Input/Output/Seekable/etc... objects, the chances of falling into the trap of creating non-reusable resources or causing a resource leak is increased. See: scala-io-core-reusable-resources for more details on this.

Wednesday, September 19, 2012

Scala-IO Core: Reusable Resources


One aspect of resources in Scala-IO that can cause problems is the construction of resource objects. The factory methods that are provided in the Resource object each have a lazy parameter for opening the underlying resource. However a common error developers can make is to pass in the already open resource to the method which has multiple problems.

Consider:
In the example above the stream is created and opened at the definition of stream (it is a val). This has two effects:

  1. the stream is open and if the resource object is not closed you will have a resource leak
  2. since the stream is opened the resource can only be used once since it will be closed after each use.
The correct way to create the resource would be to change val to def so that the stream is only created on demand and therefore there will be no chance of a resource leak. The following is the correct example:

This anti-pattern is also a risk when using the converter methods in the JavaConverters object. (A future post will look into this in more detail.) The following example shows the anti-pattern in effect: The asOutput method can only be applied to an object (at time of this writing) and therefore the resulting object has all of the negative characteristics mentioned above. Therefore it is recommended that asOutput/asInput/etc... only be used on 1 time use resources (like InputStream) within a scope and not passed out to an external method so that it is easy to view the entirety of the operation.

Thursday, September 13, 2012

Scala-IO Core: ReadChars and WriteChars

The Input and Output objects of Scala-IO assume that the underlying data is composed of bytes. However, another common pattern is to have the underlying data be composed of characters instead of bytes, for example java.io.Reader and java.io.Writer. While it is possible to decompose the output into Bytes and construct an Input object from the decorated object, ReadChars and WriteChars can be used in this situation to reduce the work needed to interact with such resources.

ReadChars and WriteChars are traits that contain the character and string methods of Input and Output. The primary difference is that the Charset is defined by the underlying resource rather than supplied at the method invocation site.

Compare two methods:

Input:
def chars(implicit codec: Codec = Codec.default): LongTraversable[Char]
ReadChars:
def chars: LongTraversable[Char]
You will notice that the ReadChars method does not have the codec parameter because there translation is not required, unlike in Input which requires the characters to be created from raw bytes.

Not many examples are needed to explain these concepts but here are a few examples on how to create ReadChar and WriteChar objects:

Sunday, August 19, 2012

Scala-IO Core: Seekable

At the same level of abstraction as Input and Output is the fine trait called Seekable. As the name implies it provides random access style methods for interacting with a resource. The example that comes immediately to mind is a random access file.

The design of Seekable largely mimics the scala.collection.Seq patch and insert methods. Not much more to say beyond getting into some examples:
IMPORTANT: Each time truncate() or patch or insert is called a new connection to the file is opened and closed. The Processor API is to be used to perform multiple operations within one connection.

Tuesday, August 14, 2012

Scala-IO Core: Output - OutputConverter

As mentioned in the last post on Output, it is possible to write arbitrary objects to an output object and have it serialized to disk.

The way this is handled in Scala-IO is via OutputConverters. If you are familiar with the type-class pattern then this should be very clear to you how this works. For a very quick introduction you can read: http://www.sidewayscoding.com/2011/01/introduction-to-type-classes-in-scala.html.

The clue is in the signature of write:
def write[T](data: T)(implicit writer: OutputConverter[T]): Unit

the last parameter is the object that defines how the object is serialized. The OutputConverter trait essentially converts and object into bytes and has a few built-in implementations in its companion object for objects like Int, Float, Byte, Char, etc...

Since the parameter is implicit the compiler will search for an implementation that satisfies the requirements (that the OutputConverter has the type parameter T). This allows:
import scalax.io._
val output:Output = Resource.fromFile("scala-io.out")
output write 3
// and
output write Seq(1,2,3)
// one can be more explicit and declare the OutputConverter
output.write(3)(OutputConverter.IntConverter)
The last line in the example shows the explicit declaration of the OutputConverter to use when writing the data. This indicates how one can provide their own converter.

Since the parameter is implicit there are two ways that custom OutputConverters can be used.
  • defining an implicit object for the object to be written. In this case all the possible ways implicits can be defined can be used. For example as an implicit value or in the companion object of the object to be written (serialized)
  • Explicitly declare the converter to use at the method call site

First let's examine the use-case where the object is from a different library and therefore we cannot create a companion object for the object. The second case is where you are implementing the class and therefore can add a companion object:
For this next bit to work you need to paste it into a file and run that or use the paste mechanism of the REPL (type :paste into repl and press enter)

Wednesday, August 8, 2012

Scala-IO Core: Output

The Output object is the primary trait for writing data to a resource. The basic usage is very simple but can get more complex when one wishes to serialize objects.

Lets start with the basic usage: A common need is to write several times to a single Output without overwriting the data. To do this one can use the processing API. A future post(s) will look at the processing API in more detail but for now a simple example:

Monday, August 6, 2012

Scala-IO Core: Long Traversable

The LongTraversable trait is one of the most important objects in Scala IO. Input provides a uniform way of creating views on the data (as a string or byte array or LongTraversable of something like bytes.)

LongTraversable is a scala.collection.Traversable with some extra capabilities. A few of the salient points of LongTraversable are:
  • It is a lazy/non-strict collection similar to Stream. In other words, you can perform operations like map, flatmap, filter, collect, etc... without accessing the resource
  • Methods like slice and drop will (if possible for the resource) skip the dropped bytes without reading them
  • Each usage of the LongTraversable will typically open and close the underlying resource.
  • Has methods that one typically finds in Seq. For example: zip, apply, containsSlice
  • Has methods that take or return Longs instead of Ints like ldrop, lslice, ltake, lsize
  • Has limitFold method that allows fold like behaviour with extra features like skip and early termination
  • Can be converted to an AsyncLongTraversable which has methods that return Futures instead and won't block the program
  • Can be converted to a Process object for advanced data processing pipelines
Example usage:

The limitFold method can be quite useful to process only a portion of the file if you don't know ahead of time what the indices of the portion are:

Thursday, August 2, 2012

Scala-IO Core: Resource, Input

Just a note: all these examples have been tested in REPL so go ahead and fire up the sbt console in the example project and try these out.

Resource


Resource is the fundamental component of Scala-IO. A Resource is essentially anything that has a simple open/close lifecycle. The Resource trait handles the lifecycle for the developer allowing him to focus on the IO logic.

In the typical use-case one of the Resource subclasses will be used. They are more useful in general because they will have one of higher level traits mixed in like Input or Output.

The most typical way to create a Resource is with the Resource object which is a factory method for creating Resource objects from various types of Java objects.

While Resource is the foundation Trait, Input and Output are the Traits most commonly used, The user-facing traits if you will.

Here are a few examples of creating Resources: There are advanced usages of Resource that we will get into in later posts. At the moment I want to focus on Input, Output and Seekable Traits. In later posts we will look at how to integrate with legacy Java APIs and how to access the underlying resource using the loan pattern.

Input


The Input Trait provides methods for accessing the data of the underlying resource in various different way. As bytes, strings, lines, etc...

There are two basic types of methods. Methods that return LongTraversable objects and methods that load the entire Resource into memory. For example: string and byteArray load the entire resource into memory while bytes and chars return a LongTraversable.

What is a LongTraversable? That will be the next post :-). Summarized, it is a specialized Lazy/non-strict Traversable.

Friday, July 27, 2012

Scala-IO Getting Started

For the next several posts you will need to have Scala-IO installed and probably should have a sbt project as well.

There are currently 2 Scala-IO 0.4 releases.

  • Scala-io 0.4-seq - A version of Scala 0.4 without the Akka dependency and therefore no ASync support
  • Scala-io 0.4 - The full version that contains an Akka dependency
The Scala 2.10 versions will have no Akka dependency but can optionally use Akka.

So getting started:

Download the example project on the docs website (http://jesseeichar.github.com/scala-io-doc/latest):
  • Go to Getting Started and follow instructions for downloading and running the example project. The following goes through the steps for the 0.4.1 instructions.



The last line (Right(770)) is not a command to enter; it is the result of the asynchonous call.

Thursday, July 26, 2012

Introducing Scala-IO


This is the start of a series of posts on Scala-IO. Scala-IO is as the name implies a library for performing input and output operations with Scala. There are 4 main facets to the library


  • Basic IO - Reading and writing to some underlying resource. The current implementation is Java based and thus allows reading and writing to resources like java.io.Readers, Writers, Channels, Streams, etc...
  • File API - A library loosely designed after java 7 nio.file API with an additional simple unix like DSL for traversing and searching the filesystem. It is a pluggable architecture which allows plugins for systems like WebDav or Zip filesystems to be addressed in a similar manner as the local filesystem. The included implementation is for the local file system and is implemented on the java.io APIs
  • Asynchronous Access - Throughout the APIs are both synchronous and asynchronous options allowing both models of programming to be easily used.
    • In the 2.10.x + versions the future implementations are pluggable but require no additional libraries if so that is the desire
    • In 2.9.x versions there are two different dependencies one with asynchronous APIs implemented on Akka and one without any asynchronous APIs
  • Processor API - An API for defining complex IO processes declaratively.
This series will look at normally a small and simple IO operation each day (or so) rather than only a few in-depth articles. This is required because of my limited available time.

With the introduction done lets look at two small examples:

Read File with Core API (not File API):
Same thing but with File API:

Thursday, May 27, 2010

zipWithIndex

A common desire is to have access to the index of an element when using collection methods like foreach, filter, foldLeft/Right, etc... Fortunately there is a simple way.

List('a','b','c','d').zipWithIndex.

But wait!

Does that not trigger an extra iteration through the collection?. Indeed it does and that is where Views help.

List('a','b','c','d').view.zipWithIndex

When using a view the collection is only traversed when required so there is no performance loss.

Here are some examples of zipWithIndex:
  1. scala> val list = List('a','b','c','d')
  2. list: List[Char] = List(a, b, c, d)
  3. /*
  4. I like to use functions constructed with case statements
  5. in order to clearly label the index.  The alternative is 
  6. to use x._2 for the index and x._1 for the value
  7. */
  8. scala> list.view.zipWithIndex foreach {case (value,index) => println(value,index)}
  9. (a,0)
  10. (b,1)
  11. (c,2)
  12. (d,3)
  13. // alternative syntax without case statement
  14. scala> list.view.zipWithIndex foreach {e => println(e._1,e._2)}
  15. (a,0)
  16. (b,1)
  17. (c,2)
  18. (d,3)
  19. /*
  20. Fold left and right functions have 2 parameters (accumulator, nextValue) 
  21. using a case statement allows you to expand that but watch the brackets!
  22. */
  23. scala> (list.view.zipWithIndex foldLeft 0) {case (acc,(value,index)) => acc + value.toInt + index} 
  24. res14: Int = 400
  25. // alternative syntax without case statement
  26. scala> (list.view.zipWithIndex foldLeft 0) {(acc,e) => acc + e._1.toInt + e._2} 
  27. res23: Int = 400
  28. /*
  29. alternative foldLeft operator.  The thing I like about this
  30. syntax is that it has the initial accumulator value on the left 
  31. in the same position as the accumulator parameter in the function.
  33. The other thing I like about it is that visually you can see that it starts with
  34. "" and the folds left
  35. */
  36. scala> ("" /: list.view.zipWithIndex) {                          
  37.      | case (acc, (value, index)) if index % 2 == 0 => acc + value
  38.      | case (acc, _) => acc                                       
  39.      | }
  40. res15: java.lang.String = ac
  41. /*
  42. This example filters based on the index then uses map to remove the index
  43. force simply forces the view to be processed.  (I love these collections!)
  44. */
  45. scala> list.view.zipWithIndex.filter { _._2 % 2 == 0 }.map { _._1}.force
  46. res29: Seq[Char] = List(a, c)

Wednesday, May 26, 2010

Return value of a block

A common misunderstanding is that a code block (without parameters) is a function. That is not the case. A code block is a sequence of statements that are executed and result the last statement is returned. That sounds like a Function0, however, if the block is passed to a method/function only the last statement will be returned to the function/method. If that method/function expects a function as the parameter the last statement maybe returned as a function not a value, this means that the block itself is not a function.
  1. scala> var count = 0                                                                                                                         
  2. count: Int = 0
  3. // the last statement is returned as a function so count
  4. // is incremented only one during the creation of the function
  5. scala> List(1,2,3,4).map{count += 1;_ + 1}
  6. res9: List[Int] = List(2, 3, 4, 5)
  7. scala> count
  8. res10: Int = 1
  9. // now the count increment is within the function
  10. scala> List(1,2,3,4).map{i => count += 1;i + 1}
  11. res11: List[Int] = List(2, 3, 4, 5)
  12. scala> count
  13. res12: Int = 5
The previous example demonstrates a Gotcha if I ever saw one. Map expects a function so the block essentially constructs a function. The last statement being the function. The first line count += 1 executed only once because it is part of creating the function not part of the resulting function. This is equivalent to:
  1. scala> val x = {count += 1 ; i:Int => i +1}
  2. x: (Int) => Int = < function1>
  3. scala> List(1,2,3,4).map(x)
  4. res15: List[Int] = List(2, 3, 4, 5)
Beginning a block with the parameter list signals that the entire block is a function.

Rule of thumb: Functions with placeholder parameters should be a single statement.

Thursday, May 20, 2010

Type Inference with Abstract Types

A second "gotcha" that one might get tripped up when dealing with abstract types is the signature of the concrete class contains type information about the abstract type. So if you are not explicit when assigning a variable or defining a function you can get unexpected compiler errors.
  1. scala> trait S {
  2.      |   type x
  3.      |   def get : x
  4.      | }
  5. defined trait S
  6. scala> var sample = new S{ 
  7.      |   type x = Int
  8.      |   def get = 3
  9.      | }
  10. sample: java.lang.Object with S{type x = Int} = $anon1ドル@397af435
  11. scala> sample = new S {
  12.      |   type x = Double
  13.      |   def get = 3.0
  14.      | }
  15. < console>:7: error: type mismatch;
  16.  found   : java.lang.Object with S
  17.  required: java.lang.Object with S{type x = Int}
  18.        sample = new S {
In this example sample uses type inference so the actual type is S with underlying type Int. The consequence is that sample can only be assigned with instances of S with type x = Int. The fix is to explicitly declare the variable type:
  1. scala> var sample2 : S = new S{ 
  2.      |   type x = Int
  3.      |   def get = 3
  4.      | }
  5. sample2: S = $anon1ドル@31602bbc
  6. scala> sample2 = new S {
  7.      |   type x = Double
  8.      |   def get = 3.0
  9.      | }
  10. sample2: S = $anon1ドル@4de5ed7b
The same thing happens when declaring functions and allows type inference for function definition
  1. scala> class Fac {
  2.      |   def newS = new S {
  3.      |     type x = Int
  4.      |     def get = 3
  5.      |   }
  6.      | }
  7. defined class Fac
  8. scala> class SubFac extends Fac{
  9.      |   override def newS = new S {
  10.      |     type x = Double
  11.      |     def get = 3.0
  12.      |   }
  13.      | }
  14. < console>:8: error: type mismatch;
  15.  found   : java.lang.Object with S
  16.  required: java.lang.Object with S{type x = Int}
  17.          override def newS = new S {
The fix for this example is to be explicit in the definition of the function in the superclass

Monday, May 3, 2010

Abstract Types vs Parameter

This topic (and the next) are intended to discuss abstract types. A class/trait with an abstract type is quite similar to a class/trait type parameter. For example:

  1. trait C[A] {
  2. def get : A
  3. def doit(a:A):A
  4. }
  5. trait C2 {
  6. type A
  7. def get : A
  8. def doit(a:A):A
  9. }

Both implementations have similar properties. However they are NOT the same. At first I thought that I could used them inter-changeably. However, consider the following examples:

  1. //compiles
  2. def p(c:C[Int]) = c.doit(c.get)
  3. // doesn't compile
  4. def p2(c:C2) = c.doit(c.get)

So why doesn't p2 compile? Because it returns A. From the signature of p2 it is impossible to know what p2 returns. There are several ways to fix this problem. One make the method return Unit:

  1. // compiles because the internals of C2 does not leak out
  2. def p(c:C2):Unit = c.doit(c.get)

Another fix would be to change doit to return Unit or an explicit return value like Int

  1. trait C2 {
  2. type A
  3. def get : A
  4. def doit(a:A):Int
  5. }
  6. // compiles correctly
  7. def p(c:C2) = c.doit(c.get)

A second difference between parameterized types and types with abstract type values is illustrated below:

  1. trait C2 {
  2. type A
  3. def get : A
  4. }
  5. scala> var c : C2 = new C2 {
  6. | type A = Int
  7. | def get = 3
  8. | }
  9. c: C2 = $anon1ドル@11a40fff
  10. // what is the type of result if at compile time the
  11. // value of c is not known
  12. scala> var result = c.get
  13. result: C2#A = 3
  14. scala> c = new C2 {
  15. | type A = String
  16. | def get = "hi"
  17. | }
  18. c: C2 = $anon1ドル@5f154718
  19. // crazy eh :) the variable can be anything but does not
  20. // have type Any so you cannot assign arbitrary values
  21. scala> result = c.get
  22. result: C2#A = hi
  23. scala> result.isInstanceOf[String]
  24. res0: Boolean = true
  25. // while the dynamic type of result is a string the
  26. // static type is not so you cannot assign a string to result
  27. scala> result = "4"
  28. < console> :8: error: type mismatch;
  29. found : java.lang.String("4")
  30. required: C2#A
  31. result = "4"
  32. ^

The obvious question is what use are abstract types. I don't claim to know them all but the main point is that they do not expose the internal implementation details to the world. The famous cake pattern is one such example usage of abstract types.

I read the following as well (wish I could remember where):

Abstract types are good when extending and there will be concrete subclasses. Param type good for when a type is useful without extension but can handle several types.

A simpler example is examined here. It is loosely based on a real world usecase.
The example below is contrived so that it is smaller than the actual usecase, so consider the design and not the fact that the example could be easier done with other examples. In the real scenario this design reduced the lines of duplicated code from around 500 to 10.

The example below shows how a Traversable like object can be created from InputStreams and Readers. The important aspect is that the type signature of Foreach does not leak information about the implementation. Users of a Foreach object don't care whether it is backed onto an InputStream or Reader. They just care about the type of object contained.

I am leaving this already long post here. The next post will investigate different ways you can get in trouble trying to implement using abstract types.



  1. import java.io.{InputStream, Reader, ByteArrayInputStream, StringReader}
  2. import java.net.URL
  3. object Foreach {
  4. def fromStream(s: => InputStream) = new Foreach[Int] {
  5. type I = InputStream
  6. def source = new Source {
  7. def in = s
  8. def next(_in : InputStream) = _in.read match {
  9. case -1 => None
  10. case i => Some(i)
  11. }
  12. }
  13. }
  14. def fromReader(s: => Reader) = new Foreach[Char] {
  15. type I = Reader
  16. def source = new Source {
  17. def in = s
  18. def next(_in : Reader) = _in.read match {
  19. case -1 => None
  20. case i => Some(i.toChar)
  21. }
  22. }
  23. }
  24. def fromInputAndFunction[A](s: => InputStream, f: Int => A) = new Foreach[A] {
  25. type I = InputStream
  26. def source = new Source {
  27. def in = s
  28. def next(_in : InputStream) = _in.read match {
  29. case -1 => None
  30. case i => Some(f(i))
  31. }
  32. }
  33. }
  34. }
  35. trait Foreach[A] {
  36. type I <: java.io.Closeable
  37. trait Source {
  38. def in : I
  39. def next(in : I) : Option[A]
  40. }
  41. def source : Source
  42. def foreach[U](f : A => U) : Unit = {
  43. val s = source.in
  44. try {
  45. def processNext : Unit = source.next(s) match {
  46. case None =>
  47. ()
  48. case Some(value) =>
  49. f(value)
  50. processNext
  51. }
  52. processNext
  53. } finally {
  54. // correctly handle exceptions
  55. s.close
  56. }
  57. }
  58. }
  59. object Test {
  60. def main(args : Array[String]) = {
  61. val data = "Hello World"
  62. val bytes = data.toArray.map { _.toByte }
  63. import Foreach._
  64. fromStream(new ByteArrayInputStream(bytes)).foreach {a => print(a.toChar)}
  65. println
  66. fromReader(new StringReader(data)) foreach print
  67. println
  68. fromInputAndFunction(new ByteArrayInputStream(bytes), i => i.toChar) foreach print
  69. println
  70. }
  71. }

Thursday, April 29, 2010

Filter with FlatMap (or collect)

I picked up this tip from one of Daniel Spiewak's tweets. He tweeted a pro tip that uses flatMap to create a filtered list:
  1. list flatMap {
  2.   case st: String => Some(st)
  3.   case _ => None
  4. }

At a glance one might wonder why not simply use list.filter{_.isInstanceOf[String]}. The difference is that the flatMap will return a List[String].

However Scala 2.8 offers the collect method for doing a similar thing.
  1. def strings(list: List[Any]) = list flatMap {
  2.   case st: String => Some(st)
  3.   case _ => None
  4. }
  5. // returned list is a List[String]
  6. scala> strings("hi" :: 1 :: "world" :: 4 :: Nil)
  7. res11: List[String] = List(hi, world)
  8. // returned list is a List[Any] (not as useful)
  9. scala> "hi" :: 1 :: "world" :: 4 :: Nil filter {_.isInstanceOf[String]}
  10. res12: List[Any] = List(hi, world)
  11. // collect returns List[String]
  12. scala> "hi" :: 1 :: "world" :: 4 :: Nil collect {case s:String => s}           
  13. res13: List[String] = List(hi, world)

Tuesday, April 27, 2010

Implicit Parameter Resolution

This topic is a continuation of the previous implicit parameter topics:

This topic provides some explanation about how implicit parameters are resulted. There are very strict rules for which implicit value is to be applied to a implicit parameter. A simple way to think about it is that the "closest" definition will be used. Local scope, enclosing class, parent class, companion object of the desired type.
  1. class X(val i:Int)
  2. class Y(val i:Int)
  3. object X {
  4.   implicit def xx = new X(1)
  5. }
  6. class Method {
  7.   def x(implicit x:X)=println(x.i)
  8.   def y(implicit y:Y)=println(y.i)
  9. }
  10. trait M { 
  11.   self : Method =>
  12.   implicit def x1 = new X(10)
  13.   implicit def y1 = new Y(100)
  14.   def testy = y
  15.   def testx = x
  16. }
  17. trait SM extends M {
  18.   self : Method =>
  19.   implicit def x2 = new X(20)
  20.   implicit def y2 = new Y(200)
  21.   
  22.   def testy2 = y  
  23. }
  24. // implicit resolved from companion object of X
  25. new Method().x
  26. // explicit applied so that value is used
  27. new Method().x(new X(3))
  28. // implicit resolved from companion object of X
  29. // NOT from M.  This is because the call site of x 
  30. // is not within M therefore does not use the implicits in M
  31. // for resolution.
  32. (new Method with M).x
  33. implicit def x = new X(30)
  34. // local scope overrides companion object implicit
  35. new Method().x
  36. // explicit applied so that value is used
  37. new Method().x(new X(3))
  38. // local scope overrides companion object implicit
  39. (new Method with M).x
  40. // testy is defined within M so the implicits within M
  41. (new Method with M).testy
  42. // testx is defined within M so the implicit within M
  43. // overrides the companion object implicit
  44. (new Method with M).testx
  45. // testy is within M (not SM) so the implicit within M
  46. // is used
  47. (new Method with SM).testy
  48. // testy2 is within SM so the implicit within SM 
  49. // overrides the implicit in M and the companion object
  50. (new Method with SM).testy2

Output:

1
3
1
30
3
30
100
10
100
200

Monday, April 26, 2010

Implicit Parameters

Evidently the topic of implicit parameters has not yet been correctly addressed. There have been several topic that refer to implicit parameters but none that directly discuss them. So before I continue with the topic of implicit parameter resolution I will discuss implicit parameters.

First, implicit parameters are not the same as implicit object conversions. Implicit parameters provide a way to allow parameters of a method to be "found". This is similar to default parameters at a glance but in fact is a different mechanism for finding the "default" value. It differs from implicit object conversion in that it is only a way for parameters for a method to be resolved. Implicit object conversion allows methods to appear to be called on one object when in fact that object is being converted behind the scenes to another type. (more or less)

An implicit parameter is a parameter to method or constructor that is marked as implicit. This means that if a parameter value is not supplied then the compiler will search for an "implicit" value defined within scope (according to resolution rules.) Implicit parameter resolution rules will be discussed soon.

Example:
  1. scala> def p(implicit i:Int) = print(i)
  2. p: (implicit i: Int)Unit
  3. // defining a val/var/def as implicit 
  4. // means that it will be considered during implicit resolution
  5. scala> implicit val v=2
  6. v: Int = 2
  7. // scope is searched for a implicit value to sue
  8. // v is found as marked implicit
  9. scala> p               
  10. 2
  11. // explicit declarations always overrides implicit values
  12. scala> p(1)
  13. 1

Implicit parameters are very nice for simplifying APIs. For example the collections use implicit parameters to supply CanBuildFrom objects for many of the collection methods. This is because normally the user does not need to be concerned with those parameters. Another example is supplying an encoding to an IO library so the encoding is defined once (perhaps in a package object) and all methods can use the same encoding without having to define it for every method call.

One important restriction is that there can only be a single implicit keyword per method. It must be at the start of a parameter list (which also makes all values of that parameter list be implicit). I further understand that only the last parameter list may be implicit.

Here are several illegal examples:
  1. // implicit is not in last parameter list
  2. scala> def pp(implicit i:Int, a:Int)(b:Int) = println(a,i)                 
  3. < console>:1: error: '=' expected but '(' found.
  4.        def pp(implicit i:Int, a:Int)(b:Int) = println(a,i)
  5. // there are 2 implicit parameters
  6. scala> def pp(implicit j:Int, a:Int)(implicit i:Int,b:Int) = println(a,i)
  7. < console>:1: error: '=' expected but '(' found.
  8.       def pp(implicit j:Int, a:Int)(implicit i:Int,b:Int) = println(a,i)
  9. // implicit is not the first parameter of the parameter list
  10. scala> def pp(a:Int, implicit i:Int) = println(i,j)         
  11. < console>:1: error: identifier expected but 'implicit' found.
  12.        def pp(a:Int, implicit i:Int) = println(i,j)
  13.                      ^

Here are several legal examples (Updated with useage examples):
  1. scala> implicit def v = 7
  2. v: Int
  3. scala> implicit var x = 10L
  4. x: Long
  5. // i is implicit
  6. scala> def pp(a:Int)(implicit i:Int) = println(a,i)
  7. pp: (a: Int)(implicit i: Int)Unit
  8. scala> pp(3)
  9. (3,7)
  10. // both i and b are implicit
  11. scala> def pp(a:Int)(implicit i:Int, b:Long) = println(a,i,b) 
  12. pp: (a: Int)(implicit i: Int,implicit b: Long)Unit
  13. scala> pp(4)               
  14. (4,7,10)
  15. // both i and b are implicit
  16. scala> def pp(implicit i:Int, b:Long) = println(i,b)  
  17. pp: (implicit i: Int,implicit b: Long)Unit
  18. scala> pp
  19. (7,10)
  20. // all or none of the parameters must be supplied
  21. scala> pp(2)
  22. < console>:13: error: not enough arguments for method pp: (implicit i: Int,implicit b: Long)Unit.
  23. Unspecified value parameter b.
  24.        pp(2)
  25. // This is syntactically legal but I cannot seem to implicitly invoke this
  26. // I would recommend: def pp(b:Long*)(implicit i:Int) = println(i,b)
  27. scala> def pp(implicit i:Int, b:Long*) = println(i,b)
  28. pp: (implicit i: Int,implicit b: Long*)Unit
  29. scala> pp(3,1,2,3)
  30. (3,WrappedArray(1, 2, 3))
  31. scala> def pp(b:Long*)(implicit i:Int) = println(i,b)
  32. pp: (b: Long*)(implicit i: Int)Unit
  33. scala> pp(1,2,3)
  34. (7,WrappedArray(1, 2, 3))

A related topic is Companion Object implicits.

Friday, April 23, 2010

Break Performance

In the Breaks comments there were several questions about the performance of the Scala break command vs the Java break command. So I decided to take a look.

The code for the tests is available on GitHub at: Scala Benchmarks. Feel free to play around with it.

I personally don't think these tests say anything of particular import because they only test how fast the Java break is vs the Scala break without doing any work in the loop. So I don't expect these number would ever been seen in the real world. However that said if you have a tight loop with minimal processing then a Scala break may not be the correct construct to use.

Here is the Java test (labelled JavaSimpleBreak)
  1. int i = 0;
  2. while (i < 10) {
  3.   if(i==1) break;
  4.   i += 1;
  5. }

Here is the Scala test (labelled ScalaSimpleBreak)
  1. var i = 0;
  2. breakable {
  3.   while (i < 10) {
  4.     if(i==1) break;
  5.     i += 1;
  6.   }
  7. }

Out of curiosity I also added a test that created a new Exception each iteration (labelled ScalaException):
  1. var i = 0;
  2.   while (i < 10) {
  3.     if(i==1) throw new Exception();
  4.     i += 1;
  5.   }

And also a test that just throws the same ScalaBreak exception each time. This one is weird since Scala Simple Break also throws the same exception but is much much faster so I think there is something about popping the stack in the example compared to the ScalaSimpleBreak test.
  1. var i = 0;
  2. breakable {
  3. while (i < 10) {
  4. if(i==1) break;
  5. i += 1;
  6. }
  7. }

The results of the tests:

First, don't compare the break tests to the Exception tests. They are sufficiently different to not be worth comparing.
Second, remember that this is a micro benchmark and has very little relationship to reality.

90000000 iterations. Swapping every 90000000 tests
JavaSimpleBreak = 254 (0.0016279129387033098)
ScalaSimpleBreak = 2475 (0.015862537493270438)
ScalaBreakException = 18806 (0.12052964852462379)
ScalaException = 156028 (1.0)

90000000 iterations. Swapping every 500000 tests
JavaSimpleBreak = 772 (0.005138547761203965)
ScalaSimpleBreak = 2351 (0.015648608531853004)
ScalaBreakException = 19346 (0.12876987692778744)
ScalaException = 150237 (1.0)

90000000 iterations. Swapping every 500 tests
JavaSimpleBreak = 790 (0.005242446563543097)
ScalaSimpleBreak = 2247 (0.014911110668710557)
ScalaBreakException = 19213 (0.1274976276270298)
ScalaException = 150693 (1.0)

Wednesday, April 21, 2010

Companion Object Implicits

When a method requires an implicit there are several ways that the implicit is resolved. One way is to search for an implicit definition in the companion object of the required type. For example: def x(implicit m:MyClass) parameter m will search local scope, class hierarchy and the MyClass companion object for an implicit val or def. (More on implicit resolution later).

To demonstrate the method put the following code block into a file and run the script:
  1. class X(val i:Int) {
  2.   def add(implicit x:X)=println(x.i+i)
  3. }
  4. object X {
  5.   implicit def xx = new X(3)
  6. }
  7. // implicit is obtained from companion object of X
  8. new X(3).add
  9. val other = new {
  10.   def print(implicit x:X)=println(x.i)
  11. }
  12. // implicit is obtained from companion object of X
  13. other.print
  14. implicit def x = new X(32)
  15. // implicit is obtained local scope
  16. other.print
Running: scala impl.scala should produce:
6
3
32
Subscribe to: Comments (Atom)

AltStyle によって変換されたページ (->オリジナル) /