Jay Fields' Thoughts

I was recently chatting to some friends about all the apprenticeship experiments going on and (name withheld to protect the innocent) lamented:

So, what happens when they quit that first job (or worse, get laid-off) and their only skill is writing a Rails app?

I know what happens. It happened to me.

Okay, I didn't actually participate in an apprenticeship program that taught me Rails. However, I did drop out of college after gaining some experience programming, get a job building web applications, and I was laid off about a year later. Getting another job wasn't so hard, but it was hard to find a job that paid as well. It turns out, there's not a huge demand for people who don't have formal education or much experience, go figure.

While I was looking for a new job I noticed that there were significantly more jobs for people with certain skills. I'm sure it differs by location and time - in Jacksonville FL, around 2000, 75% of the jobs openings were looking for people with Microsoft .net skills. Sadly, I did not have .net skills. That was when I learned that I needed to have not just the skills necessary to do my job today, but also the skills necessary to land a new job tomorrow. Since I wasn't already in that situation, I had to settle for a job making .5 of my previous salary; however, my new job did allow me to learn .net.

I firmly believe: if you have less than 4 years of experience, you should always know what technology is in the highest demand for your area, and you should be proficient enough that you can do well in an interview. Perhaps your day to day job already teaches you the most highly sought after skills, fantastic, you should probably learn the 2nd most sought after skill. Don't wait until you're already looking for a job to try to learn a new technology. The truth is, learning something new will almost certainly help you with your current job as well, so everyone wins.

note: searching hipster-programming-jobs.com to find out what's in demand is a selection bias failure. Use monster.com & dice.com. The goal is to appeal to as many companies as possible, not just the cool ones. You can focus on your dream job once you've got enough experience to ensure that your career doesn't suffer a failure to launch.

Whatever technology you choose will certainly have specific books that you should read; however, there are other books that will also help make up for the experience you (sadly, we) missed by not getting a Computer Science degree.

Refactoring was the first book I ever read that really opened my eyes to better ways to program. It's a pretty easy book to read, even for a beginner, and I can't recommend it enough. It's likely that you find that the most in-demand technology is Java or C#, and Refactoring will complement learning either of those languages. However, if you prefer, there's a Refactoring: Ruby Edition as well.

Once you've read refactoring, it's good to dig into Patterns of Enterprise Application Architecture (PofEAA). PofEAA contains patterns that are found all over the enterprise world (including ActiveRecord). You should read this book not because it is a bible, but because it shows you various solutions to commonly encountered problems. It gives you ideas about how other people are solving problems, and provides reference data that you can refer to if you encounter any of those issues in the future.

Those two books will demonstrate to potential employers that you're interested in building successful business applications; however, you'll also want to read a few books that diversify your experience and open your eyes to what can be done by using less mainstream techniques.

The wizard book and the dragon book sound like interesting fictional books, but are actually building blocks for your programming career. Both books are widely regarded as classics that any serious programmer should read. That said, they are not easy to digest. I'm not going to claim it will be fun, but you should push your way through both of these books - and then do it again a year later. These books can help shape your career in a positive way, especially the wizard book, perhaps more than any other text available.

Once you've pushed through both of those books, there are a few follow up options. If you preferred the wizard book, I would recommend following up with The Joy of Clojure or Learn You a Haskell For Great Good. You're unlikely to get a job using either technology, but both will broaden your horizons and also demonstrate to potential employers that you are deeply interested in increasing your programming capabilities. If you preferred the dragon book, I would move on to Domain Specific Languages. Domain Specific Languages is the definitive guide to DSL programming, and provides you with many examples of how to put DSLs to practical use within your applications.

Once you've made it through all (or most) of those books, you shouldn't have any problem finding a job that is both satisfying and also pays well.

Before Clojure, I had never used a Functional Programming language. My experience has primarily been in C#, Ruby, & Java. So, learning how to use Clojure effectively has been a fun and eye-opening experience.

I've noticed a few things:

• The Clojure language has a lot of built in support for maps and hashes (literal syntax, destructuring, etc)
• Rich Hickey is quoted as saying "It is better to have 100 functions operate on one data abstraction than 10 functions on 10 data structures." Clojure reflects this opinion and has loads of functions that work with Sequences.
• I strongly dislike working with people who think Map<String, Map<String, String>> is generally a good idea in Java. However, all the support for maps and vectors in Clojure makes working with them easy and efficient.
• Once you embrace functions, maps, vectors, etc, composing functions together to transform data becomes effortless.

Like I said, I'm still learning Clojure. Recently I refactored some code in a way that I thought was worth documenting for other people who are also learning Clojure.

Let's start with a simple example. Let's assume we want to track the score in the British Open (Golf). The data structure we will be working with will be a map of maps, keyed by country and then player.

{:England {"Paul Casey" -1}}

The above example shows that Paul Casey plays for England and currently has the score of -1. We need a function that allows you to update a players score. Let's also assume that scores for individual holes come in the following map form.

{:player "Paul Casey" :country :England :score -1}

Given the above code, you would expect the following test to pass.

(def current-score {:England {"Paul Casey" -1}})

(deftest update-score-test
 (is (= 
 {:England {"Paul Casey" -2}} 
 (update-score current-score {:player "Paul Casey" :country :England :score -1}))))

(Let's ignore the fact that I'm def'ing current-score. It's only for simplicity's sake.)

There are a lot of ways to make that test pass. It took me a few tries to come up with something I liked. Below is the path I took.

Step one, get the test passing using the functions I know.

(defn update-score [current update]
 (let [player (:player update)
 country (:country update)
 hole-score (:score update)]
 (merge current {country {player (+ ((current country) player) hole-score)}})))

The above example code is a victory when you are learning Clojure. The tests pass, you've used a few features of the language. It's a great first pass. However, I'm sure the Clojure veterans are drooling over the refactoring possibilities.

Step two, learn the update-in function. Like I previously mentioned, Clojure has a lot of functions for working with sequences (maps, vectors, lists, etc). It's not always obvious which function you should be using. However, I've had loads of success learning from the mailing list, and getting immediate answers in the #clojure chat room on freenode. One way or another, I eventually found the update-in function, which is designed up update a nested key.

(defn update-score [current update]
 (let [player (:player update)
 country (:country update)
 hole-score (:score update)]
 (update-in current [country player] (fn [_] (+ ((current country) player) hole-score)))))

The switch to update-in is relatively easy and cleans up a bit of code, but now I've introduced an anonymous function. Anonymous functions are great, but if I can get away with not creating them, I always take that route.

However, before we eliminate the anonymous function, we're actually better off making sure we understand the new function (update-in). While the current version of the code works, it's much cleaner if you use the current value that is passed into the anonymous function instead of looking up the value yourself.

(defn update-score [current update]
 (update-in current [(:country update) (:player update)] (fn [overall-score] (+ overall-score (:score update)))))

Since we no longer use country and player in multiple places, it seems like removing the let statement is a logical choice. As a result, the amount of code necessary to solve this problem seems to get smaller and smaller.

However, we can actually take this a step farther and use destructuring to provide an even cleaner implementation.

(defn update-score [current {:keys [country player score]}]
 (update-in current [country player] (fn [overall-score] (+ overall-score score))))

If you aren't familiar with destructuring the syntax might be a bit confusing. I created a blog entry for destructuring that should help if you aren't familiar with it. There's also a good write up with destructuring examples available on clojure.org. I would also recommend writing a few simple functions and try out the examples for yourself.

There's one more step to take, but for me it was a big one. The documentation for update-in states:

Usage: (update-in m [k & ks] f & args)

'Updates' a value in a nested associative structure, where ks is a sequence of keys and f is a function that will take the old value and any supplied args and return the new value, and returns a new nested structure.

What I currently have works fine, but the additional args that update-in takes allows me to provide a more concise solution that I greatly prefer.

(defn update-score [current {:keys [country player score]}]
 (update-in current [country player] + score))

This final version doesn't require any anonymous functions. Instead it relies on on update-in to apply the current value and any additional args to the function that you passed in. The final version is so concise you begin to wonder if you really need to define an update-score function.

The final snippet is an example of why I'm very intrigued by Clojure these days. The ability to compose functions and mix them in with the powerful functions of the language results in very concise code.

I consider what I would need to write to accomplish the same thing in Java or Ruby, and I have to wonder if the Functional Programming supporters might just be on to something.

Refactoring: Ruby Edition is available (and In Stock) on amazon.com.



Sorry it took so long, I hope it is worth the wait.

Refactoring: Ruby Edition is now available on Safari as a Rough Cut.

It's been about 18 months since my first entry about Refactoring: Ruby Edition. It's been a long journey, but the book should be seeing day light in the next few months.

It's available for pre-order on Amazon now, and it should also be available on Safari in the near future.

Thanks to everyone who helped, and I hope the book is as fun to read as it was to write.

You have methods that can be chained for greater maintainability.

mock = Mock.new
expectation = mock.expects(:a_method_name)
expectation.with("arguments")
expectation.returns([1, :array])

becomes

mock = Mock.new
mock.expects(:a_method_name).with("arguments").returns([1, :array])

Motivation

Calling methods on different lines technically gets the job done, but at times it makes sense to chain method calls together and provide a more fluent interface. In the above examples, assigning an expectation to a local variable is only necessary so that the arguments and return value can be specified. The solution utilizing Method Chaining removes the need for the local variable. Method Chaining can also improve maintainability by providing an interface that allows you to compose code that reads naturally.

Mechanics

• Return self from methods you wish to allow chaining from
• Test
• Remove the local variable and chain the method calls
• Test

Example

Suppose you were designing a library for creating html elements. This library would likely contain a method that created a select drop down and allowed you to add options to the select. The following code contains the Select class that could enable creating the example html and an example usage of the select class.

class Select
 def options
 @options ||= []
 end
 
 def add_option(arg)
 options << arg
 end
end

select = Select.new
select.add_option(1999)
select.add_option(2000)
select.add_option(2001)
select.add_option(2002)
select # => #<Select:0x28708 @options=[1999, 2000, 2001, 2002]>

The first step in creating a Method Chained solution is to create a method that creates the Select instance and adds an option.

class Select
 def self.with_option(option)
 select = self.new
 select.options << option
 select
 end

 # ...
end

select = Select.with_option(1999)
select.add_option(2000)
select.add_option(2001)
select.add_option(2002)
select # => #<Select:0x28488 @options=[1999, 2000, 2001, 2002]>

Next, change the method that adds options to return self so that it can be chained.

class Select
 # ...
 
 def add_option(arg)
 options << arg
 self
 end
end

select = Select.with_option(1999).add_option(2000).add_option(2001).add_option(2002)
select # => #<Select:0x28578 @options=[1999, 2000, 2001, 2002]>

Finally, rename the add_option method to something that reads more fluently, such as "and".

class Select
 def self.with_option(option)
 select = self.new
 select.options << option
 select
 end

 def options
 @options ||= []
 end
 
 def and(arg)
 options << arg
 self
 end
end

select = Select.with_option(1999).and(2000).and(2001).and(2002)
select # => #<Select:0x28578 @options=[1999, 2000, 2001, 2002]>

Isolate Dynamic Receptor

A class utilizing method_missing has become painful to alter

Introduce a new class and move the method_missing logic to that class.

Motivation

As I previously mentioned, objects that use method_missing often raise NoMethodError errors unexpectedly, or worse you get no more information than: stack level too deep (SystemStackError).

Despite the added complexity, method_missing is a powerful tool that needs to be used when the interface of a class can not be predetermined. On those occasions I like to use Isolate Dynamic Receptor to limit the behavior of an object that also relies on method_missing.

The ActiveRecord::Base (AR::B) class defines method_missing to handle dynamic find messages. The implementation of method_missing allows you to send find messages that use attributes of a class as limiting conditions for the results that will be returned by the dynamic find messages. For example, given a Person subclass of AR::B that has both a first name and a ssn attribute it's possible to send the messages Person.find_by_first_name, Person.find_by_ssn, and Person.find_by_first_name_and_ssn.

It's possible, but not realistic to dynamically define methods for all possible combinations of the attributes of an AR::B subclass. Instead utilizing method_missing is a good solution; however, by defining method_missing on the AR::B class itself the complexity of the class is increased significantly. AR::B would benefit from a maintainability perspective if instead the dynamic finder logic were defined on a class whose single responsibility was to handle dynamic find messages. For example, the above Person class could support find with the following syntax: Person.find.by_first_name, Person.find.by_ssn, or Person.find.by_first_name_and_ssn

Note: very often it's possible to know all valid method calls ahead of time, in which case I prefer Replace Dynamic Receptor with Dynamically Define Method.

Mechanics

• Create a new class whose sole responsibility is to handle the dynamic method calls.
• Copy the logic from method_missing on the original class to the method_missing of the focused class.
• Change all client code that previously called the dynamic methods on the original object.
• Remove the method_missing from the original object.
• Test

Example

Here's a recorder class that records all calls to method_missing.

class Recorder
 instance_methods.each do |meth|
 undef_method meth unless meth =~ /^(__|inspect)/
 end

 def messages
 @messages ||= []
 end

 def method_missing(sym, *args)
 messages << [sym, args]
 self
 end
end

The recorder class may need additional behavior such as the ability to play back all the messages on an object and the ability to represent all the calls as strings.

class Recorder
 def play_for(obj)
 messages.inject(obj) do |result, message|
 result.send message.first, *message.last
 end
 end

 def to_s
 messages.inject([]) do |result, message|
 result << "#{message.first}(args: #{message.last.inspect})"
 end.join(".")
 end
end

As the behavior of Recorder grows it becomes harder to understand what messages are dynamically handled and what messages are actually explicitly defined. By design the functionality of method_missing should handle any unknown message, but how do you know if you've broken something by adding a explicitly defined method?

The solution to this problem is to introduce an additional class that has the single responsibility of handling the dynamic method calls. In this case we have a class Recorder that handles recording unknown messages as well as playing back the messages or printing them. To reduce complexity we will introduce the MessageCollector class that handles the method_missing calls.

class MessageCollector
 instance_methods.each do |meth|
 undef_method meth unless meth =~ /^(__|inspect)/
 end

 def messages
 @messages ||= []
 end

 def method_missing(sym, *args)
 messages << [sym, args]
 self
 end
end

The record method of Recorder will create a new instance of the MessageCollector class and each additional chained call will be recorded. The play back and printing capabilities will remain on the Recorder object.

class Recorder
 def play_for(obj)
 @message_collector.messages.inject(obj) do |result, message|
 result.send message.first, *message.last
 end
 end

 def record
 @message_collector ||= MessageCollector.new
 end

 def to_s
 @message_collector.messages.inject([]) do |result, message|
 result << "#{message.first}(args: #{message.last.inspect})"
 end.join(".")
 end
end

You need to use eval, but want to limit the number of times eval is necessary.

class Person
 def self.attr_with_default(options)
 options.each_pair do |attribute, default_value|
 define_method attribute do
 eval "@#{attribute} ||= #{default_value}"
 end
 end
 end
 
 attr_with_default :emails => "[]", :employee_number => "EmployeeNumberGenerator.next"
end

becomes

class Person
 def self.attr_with_default(options)
 options.each_pair do |attribute, default_value|
 eval "def #{attribute}
 @#{attribute} ||= #{default_value}
 end"
 end
 end
 
 attr_with_default :emails => "[]", :employee_number => "EmployeeNumberGenerator.next"
end

Motivation

premature optimization is the root of all evil -- Knuth, Donald

I'll never advocate for premature optimization, but this refactoring can be helpful when you determine that eval is a source of performance pain. The Kernel#eval method can be the right solution in some cases; but it is almost always more expensive (in terms of performance) than it's alternatives. In the cases where eval is necessary, it's often better to move an eval call from run-time to parse-time.

Mechanics

• Expand the scope of the string being eval'd.
• Test

Example

The following Person class uses eval to define the logic the readers rely upon for returning a default value if no value has previously been set.

class Person
 def self.attr_with_default(options)
 options.each_pair do |attribute, default_value|
 define_method attribute do
 eval "@#{attribute} ||= #{default_value}"
 end
 end
 end
 
 attr_with_default :emails => "[]", :employee_number => "EmployeeNumberGenerator.next"
end

The above example executes without issue, but it relies upon eval each time a reader is called. If multiple calls to eval are determined to be problematic the solution is to expand the eval to include defining the method itself.

class Person
 def self.attr_with_default(options)
 options.each_pair do |attribute, default_value|
 eval "def #{attribute}
 @#{attribute} ||= #{default_value}
 end"
 end
 end
 
 attr_with_default :emails => "[]", :employee_number => "EmployeeNumberGenerator.next"
end

You have methods you want to handle dynamically without the pain of debugging method_missing.

class Decorator
 def initialize(subject)
 @subject = subject
 end
 
 def method_missing(sym, *args, &block)
 @subject.send sym, *args, &block
 end
end

becomes

class Decorator
 def initialize(subject)
 subject.public_methods(false).each do |meth|
 (class << self; self; end).class_eval do
 define_method meth do |*args|
 subject.send meth, *args
 end
 end
 end
 end
end

Motivation

Debugging classes that use method_missing can often be painful. At best you often get a NoMethodError on an object that you didn't expect, and at worst you get stack level too deep (SystemStackError).

There are times that method_missing is required. If the usage of an object is unknown, but must support unexpected method calls you may not be able to avoid the use of method_missing.

However, often you know how an object will be used and using Dynamically Define Method you can achieve the same behavior without relying on method_missing.

Mechanics

• Dynamically define the necessary methods
• Remove method_missing
• Test

Example: Dynamic delegation without method_missing

Delegation is a common task while developing software. Delegation can be handled explicitly by defining methods yourself or by utilizing something from the Ruby Standard Library such as Forwardable. Using these techniques gives you control over what methods you want to delegate to the subject object; however, sometimes you want to delegate all methods without specifying them. Ruby's Standard Library also provides this capability with the delegate library, but we'll assume we need to implement our own for this example.

The simple way to handle delegation (ignoring the fact that you would want to undefine all the standard methods a class gets by default) is to use method_missing to pass any method calls straight to the subject.

class Decorator
 def initialize(subject)
 @subject = subject
 end
 
 def method_missing(sym, *args, &block)
 @subject.send sym, *args, &block
 end
end

This solution does work, but it can be problematic when mistakes are made. For example, calling a method that does not exist on the subject will result in the subject raising a NoMethodError. Since the method call is being called on the decorator, but the subject is raising the error it may be painful to track down where the problem resides.

The wrong object raising a NoMethodError is significantly better than the dreaded stack level too deep (SystemStackError). This can be caused by something as simple as forgetting to use the subject instance variable and trying to use a non-existent subject method or any misspelled method. When this happens the only feedback you have is that something went wrong, but Ruby isn't sure exactly what it was.

These problems can be avoided entirely by using the available data to dynamically define methods at run time. The following example defines an instance method on the decorator for each public method of the subject.

class Decorator
 def initialize(subject)
 subject.public_methods(false).each do |meth|
 (class << self; self; end).class_eval do
 define_method meth do |*args|
 subject.send meth, *args
 end
 end
 end
 end
end

Using this technique any invalid method calls will be correctly reported as NoMethodErrors on the decorator. Additionally, there's no method_missing definition, which should help avoid the stack level too deep problem entirely.

Example: Using user defined data to define methods

Often you can use the information from a class definition to define methods instead of relying on method_missing. For example, the following code relies on method_missing to determine if any of the attributes are nil.

class Person
 attr_accessor :name, :age
 
 def method_missing(sym, *args, &block)
 empty?(sym.to_s.sub(/^empty_/,"").chomp("?"))
 end
 
 def empty?(sym)
 self.send(sym).nil?
 end
end

The code works, but it suffers from the same debugging issues that the previous example does. Utilizing Dynamically Define Method the issue can be avoided by defining the attributes and creating the empty_attribute? methods at the same time.

class Person
 def self.attrs_with_empty_methods(*args)
 attr_accessor *args
 
 args.each do |attribute|
 define_method "empty_#{attribute}?" do
 self.send(attribute).nil?
 end
 end
 end
 
 attrs_with_empty_methods :name, :age
end