On Ruby

I've seen a variety of mumbled pre-announcements, second hand announcements, and outright rumours over the last couple of days. I thought I'd collect them here, and see if anyone wanted to confirm, deny, or add some more.

• There's more autotest functionality on the horizon, after a refactoring
• It looks like ZenTest will be supporting Behavior Driven Development (BDD) and rspec soon
• Wrox is looking into entering the Ruby on Rails book fray with at least a couple of titles
• Canada on Rails went well enough that there will probably be a European Country on Rails Conference this year
• A group of Michigan based groups are putting together a Michigan Ruby Conference for late this summer or early this fall.

I'm looking forward to seeing these come to pass. I thnk they'd all be good things for the Ruby community.

I was IMing with a friend today, and asked him if he was paying attention to the Coverity scan. He told me he wasn't, and I jokingly commented that it had the potential to be as addictive as fantasy baseball.

I little bit later, I realized that I was speaking more truly than I'd thought. I often catch myself looking at different projects and their defect rates, comparing one against another. I'm registered to look at the Ruby results, and occasionally find myself wondering what the other projects look like.

It is possible to see some of the rolled up data both on a per project and on a sitewide basis. For example, although Coverity identifies eight different classes of defects, the three most common account for over 60% of the total tracked defects.

There are some things I'd like to see -- a project's velocity in defects corrected over time and the average lifetime of a defect after identification (by project, sitewide, or by class of problem) for example. Some of the data could probably be harvested from their website, but doing that would mean admitting that I'd crossed the line and was addicted to coverity scan data.

I guess I should be glad there isn't a fantasy coverity project league out there.

To try to learn a bit more about RSpec, I've decided to write an rspec file for checkr. I learn best by doing, and this seemed like a good way to do something worthwhile. Apologies in advance for any mangled terminology, I'm just getting started with this stuff. Please feel free to correct me in a comment. I'll update this post as appropriate.

I decided to start small. So I wrote a simple test script and used the test2rspec tool to convert it.

class TestFoo < Test::Unit::TestCase def test_bar foo = Foo.new assert_equal true, foo.bar end end 

became

context "" do
 specify "Bar" do
 foo = Foo.new
 foo.bar.should.equal true
 end
end

The output changed a little bit too:

$ ruby test_foo.rb
Loaded suite test_foo
Started
.
Finished in 0.000646 seconds.
1 tests, 1 assertions, 0 failures, 0 errors

became

$ spec rspec_foo.rb
.
Finished in 0.000337 seconds
1 context, 1 specification, 0 failures

A useful bit of functionality is the ability to print more verbose output. It's not terribly interesting yet though, so let me play with the rspec file a bit before I show it to you.

In the first line of the rspec file, there's a pair of quotation marks right after the context. These are a place holder for the string I want to use to identify this context. On the next line, "Bar" is given as an identifying string for the specification we're working on (this all sounds like a spec not a test suite, doesn't it -- it's intentional). Let's make these strings a bit more meaningful, shall we:

context "Simple Case" do
 specify "Bar is true" do

Now, when I do spec -v rspec_foo.rb, the output looks a little more meaningful:

Simple Case
- Bar is true
Finished in 0.00054 seconds
1 context, 1 specification, 0 failures

What will this look like when it fails? Let's add a failing specification and find out:

 specify "Bar is false" do
 foo = Foo.new
 foo.bar.should.equal false
 end

Which generates output like this (slightly edited for the faint of heart):

Simple Case
- Bar is true
- Bar is false (FAILED - 1)
1)
 should equal  (Spec::Api::ExpectationNotMetError)
./rspec_foo.rb:12:in `Bar is false'
[most of a stack trace deleted]
Finished in 0.000943 seconds
1 context, 2 specifications, 1 failure

Well, raising an exception and having multiple lines of stacktrace thrown onto my terminal isn't really what I'd hoped for. I'm at the end of space for a blog post and time on the bus though, so I'll see about cleaning this up on tomorrow's ride home.

I'm glad to see some gathering momentum around the coverity scans on the ruby-core list. I've posted a table of the 29 remaining possible defects there, and two of the possible defects in their entirity. (I'll be posting a third, which I think is a false positive) as soon as I'v finished this post.) The developers seem interested, and I know that a couple have registered with coverity.

I've been amazed to see the way the Perl and Python camps have attacked their defect list though. Perl is down to one verified defect and one uninspected possible defect (for an error rate of 0.004). Python has done even better and has no remaining defects (an error rate of 0.000). Wow!

Even more impressive is that there are eight projects with a defect rate of 0.000 and four more with a defect rate under 0.010. I hope we'll see Ruby joining them soon. It'll take some work to identify the false positives and fix the real problems, but we've made a start and it looks like people are interested in finishing the job.

Maybe once we get to zero defects, we can start chasing another big item — getting Ruby to run well under valgrind.

When I first brought up the idea of Coverity's scans, some of my Ruby friends told me that it would be a waste of time. "There will be too many false positives", or similar reasons were given. When the invitation to sign up for the scan readers was posted to the ruby-core mailing list and didn't seem to draw any interest, I was worried that they were right.

I was wrong. I decided to sign up with coverity, and post one of the errors to the ruby-core list. Less than twelve hours later, I got an email from Hidetoshi NAGAI. It said: "Than you for your report. I've fixed it on CVS."

Then I went over to the Coverity's Report and found that they'd already updated it to show just 29 defects and a 4% drop in the defect rate. I'm posting another defect now.

Tim O'Reilly has been posting about the book market over the last couple of days, and it's been quite informative. He calls out Apress, Wrox, and O'Reilly as outperforming the current market

Jim Minatel (of Wrox) has posted a response. He said something that really struck a chord with me:

So, I'm happy to see that Apress's rep is improving and O'Reilly's rep is strong, as always. But Wrox? We're on the right track too. Competition is good for the customer, so we'll just keep getting better.

As an occasional blogger for two of these three publishers (here and here), I enjoy seeing their continued success. From a wider perspective, I agree with Jim's point that the competition is good for all of us. Right now all three of them (plus Manning) are chasing the Pragmatic Programmers in the Ruby book space (well, two of them are, I've not seen anything from Wrox yet).

I've been talking to people at a number of publishers about getting into the Ruby pool for a couple of years now. I won't kid myself and think that I tipped things, but I'm very glad to see that the tipping point has been reached. I'm looking forward to a big batch of Ruby and Rails books coming out over the next six months, I think it will be good for the community and good for the publishers.

---

This post is part of a collection of articles about Publishing, Growing Markets, and Books.

The mystery interview is done, and waiting on some paperwork to be submitted. I'm hoping to see it online at the O'Reilly Network fairly soon.

Just to the end the mystery, I've interviewed Zed Shaw. He's the creator of Mongrel (but you already knew that). He's also got some important things to say about programming, Ruby, Rails, and Mongrel.

A couple of things didn't make it into the interview, so I thought I'd share them here:

The thing I'm most impressed with is how David H. Hansson has finally seen the light and has accepted HTTP as more than just a means of shuttling data. When I first started working on Mongrel he frequently asked what was wrong with FastCGI, since it worked just fine. Since seeing his presentation at Canada on Rails--which focused a lot on how useful HTTP was--I think he's finally embraced the "Just Use HTTP" religion. I'm not sure if it's true, but I'd like to think Mongrel had a part in his conversion.

The majority of questions I got from folks seemed to be serious queries about deployment issues indirectly related to Mongrel, or just complete misunderstandings. My favorite misunderstanding was when Floyd Marinescu from The Server Side asked me about Mongrel then said, "Web server? Isn't that a solved problem?" Kind of frightening that the guy is going to start a TSS for Ruby and has so little understanding of the community's issues.

. . .

In general I think Mongrel is well loved, and that it's really working great for people. I also think that most folks are just now seriously basing their systems on it and the next few months will be when I start to get more feedback about how well it works for them in production. I've been forging ahead of them in my own work with Mongrel so I'm confident they'll have few problems that can't be solved (yet again)

The third article in my column was posted here this morning. It's got some coverage of Canada on Rails and Coverity, and a short introduction to optimizing Ruby programs.

I'm still thinking about test writing and how it helps us understand our code better. Then I noticed Laurent Bossavit's blog entry about code (and bug) ownership. Two paragraphs really struck me.

Joe says he gets it, and writes a new test which shows he does indeed get it. We fix the problem and check in the fix.

Then we get to the important point. "Think you'll remember this? You've been bitten once, so maybe it'll stick..." But Joe wants to make sure: "I've got it now, but I feel like it could slip away. What I'll do is write a blog entry sometime this afternoon - if I can explain it to someone else I'll feel confident I won't get bitten again. And I'm sure there'll be someone to straighten me out if I only think I understand."

I think there's a lot of wisdom there. When we're writing or reading code, we might think we understand what's going on inside it. When we write a test, it's more likely that we do. When we explain it to someone (or blog about it) we're most likely to understand and remember.

I think this is one reason that both Sean and I are blogging about the things we see and do while we work on checkr.

Refactoring Name: Extract Method

Original Code (taken from the checkr project):

 def process_args(exp)
 @saw_asgn = true
 args = []
 exp.length.times do
 element = exp.shift
 if element.kind_of? Array
 args << process(element) else args << init_var(element) end end s(:args, *args) end def process_block(exp) unless exp[1] != [:nil] @errors << "no-op in block" end args = [] exp.length.times do element = exp.shift if element.kind_of? Array args << process(element) else args << init_var(element) end end s(:block, *args) end 

Explanation and mechanics:
process_block was written by creating the new functionality (the unless block at the beginning) and copying the args building code from process_args. This created a block of duplicate code that I didn't want to maintain, so I extracted a method like this:

• I created a new method, build_sexp
• I moved the code from process_block into it
• I replaced the code in process_block with a call to build_sexp
• I reran my test suite to verify that nothing broke[1]
• I replaced the original code in process_args with a call to build_sexp
• I reran my test suite to verify that nothing broke

New Code (ordered by method name):

 def build_sexp(exp)
 args = []
 exp.length.times do
 element = exp.shift
 if element.kind_of? Array
 args << process(element) else args << init_var(element) end end args end def process_args(exp) @saw_asgn = true args = build_sexp(exp) s(:args, *args) end def process_block(exp) unless exp[1] != [:nil] @errors << "no-op in block" end args = build_sexp(exp) s(:block, *args) end 

This actually lead to a couple of other refactorings. We didn't like the fact that build_sexp did two things (emptied the original array, and processed each member of the new array), so we extracted another method. Then we realized that our two new methods were poorly named, so we performed Rename Method on both of them. we're not sure we'll use the code we ended up with, as it still hides some functionality that we might want to keep explicit, but it was a good exercise to go through.

I took an interviewing class a long time ago, and the teacher talked about an idea that I'm still finding new ways to apply. She called it a "Betty has buttons" file. The idea was that as she interacted with people, she'd keep a running file with little personal notes like:

Betty has a button collection

or:

Jim likes chocolate chip cookies

Then, when she went back to talk to Jim about something she might know to take some chocolate chip cookies (or a maybe a new button if she was going to talk to Betty). We were encouraged to keep a little notebook with these kinds of notes so that we'd be able to remember things about jobs, interviewers, etc. as we were out looking for a job.

I never did invest in a little notebook for that (and yes, I've managed to keep myself meaningfully employed without it). I have found myself keeping mental lists of all kinds of things to remember and, somehow, I stil refer to them as my "Betty has Buttons" lists. The other day, I happened across a new application, one I should have seen a long time ago.

Sean and I have been ping ponging on checkr for a little while, and I've noticed thathe's much better at finding the next test than I am. It's like a ritual, I get an IM that says "Ok. I solved your test.". Then, just a few minutes later, "Tag! You're it.". Then I go to work. Eventually, I get his test solved and I IM Sean, "Heh, got it.". Then I get stuck. What should we test next? Sometimes I spend longer trying to figure out the next test than I did solving the last one.

This morning, it hit me. I saw Sean's latest test and thought to myself, "Hmm, that looks a lot like the code he wrote a test or two ago. He must have a 'Betty has Buttons' list for tests".

To me, the challenge of getting the test to pass, and tossing it back to Sean had been challenge enough. I'd failed to see that I could be thinking harder about the code I was writing. What it was doing, where it was weak and needed more testing. If I were to make that little investment, If I were to keep a "Betty has Buttons" list about this, I'd be writing better tests faster — and checkr (and my other projects) would all benefit.

I've got a few things that I'm working on right now, and I wanted to list them here -- both to get feedback and to push myself a little bit (it always seems easier to finish something I've already promised to get done).

I'm doing an interview with an up and coming member of the Ruby community. I'm hoping to get it published at O'Reilly, but if worse comes to worse I can always publish it here. Is there someone in the Ruby world that you'd like to see interviewed? Drop me a line

I'm putting together a set of book reviews about Ruby books for Linux Journal. It looks like I'll need to write one or two more sets before the end of summer at the rate books are coming out.

There's also a lot of stuff going on the in the Ruby community that I'm trying to fit into my next column. This time around, I'm also going to be writing about profiling and optimization. Some of the things on my list to cover in upcoming columns are:

• Rake
• Ruby Gems
• the Ruby Programming Shop
• Mr. GUID

What should I be adding to my list?

Also, it looks like one of my RubyQuiz submissions has been selected as an upcoming RubyQuiz. This will be the second time I've had a quiz run. I'm pretty psyched.

It looks like some other folks agree with me about Coverity. I saw a post over here (in German) that pointed back here, and provided a synopsis of the data and a call for people to help fix defects. (If I'm reading the German correctly.)

Well, James Gray asked for some quiz submissions, and I decided to respond. He's gotten back to me to let me know he liked the one that went in. I hope it goes well because I think it will be useful to the Ruby community at large if enough people respond (and, oh yeah, getting a signed copy of The Best of Ruby Quiz would be cool too. If you've got a good idea for a Ruby Quiz, please, send it in to the dropbox.

Recently, Coverity added Ruby to it's list of scanned projects (overview data is available here). Because Perl and Python were already in Coverity's target list, this allows us to look at a code quality comparison between these three languages. Below, I've listed each langage with its Lines of Code (LoC), current number of defects, original number of defects, current rate of defects per KLoC, and the original rate of defects per KLoC. I've had to calculate the last figure since it's not explicitly available on Coverity's wesite. Python looks very good here, probably because they've been aggressive about attacking the reported defects. In the grand scheme of things though, All three languages look pretty good. This is where Ruby shows a bit better than the 'competition'. Ruby has a pretty good (low) defect rate before we've even gotten any feedback from Coverity. Hopefully we can be at least as aggressive about attacking our defects as the Python community. In fact, there's an even better target — AMANDA: (By the way, they started out with a rate of 1.227)

ZenMacs — too cool for words, except Sean made it even cooler by writing about it.

Sean and I decided to work on finding noops in conditionals for checkr. It turns out ParseTree made this a lot easier than we thought it would. Here's the abbreviated code:

class CheckR
 def clean_conditional?(klass)
 ParseTest.new.process(ParseTree.new.parse_tree(klass))
 false
 end
end
class ParseTest < SexpProcessor def assignment_in_conditional?(exp) @saw_lasgn = false test_result = process(exp.shift) raise CheckRAssignmentInConditional if @saw_lasgn test_result end # new method def noop_in_conditional?(exp) raise CheckRNoop unless exp exp end def process_if(exp) sexp_type = exp.shift else_exp = exp.pop then_exp = exp.pop s(sexp_type, assignment_in_conditional?(exp), process(noop_in_conditional?(then_exp)), process(else_exp)) end end 

I'm sure there's some more refactoring in there, but I can't see it right now. Probably because I should already be in bed. I guess I'll write a new test for Sean in the morning.

Ok, with two articles in the series, I now feel like it's a column. I'm looking forward to a long run, but I'll need more feedback to make it really useful.

• Second Article — mostly community news
• First Article — Test First Tools

Sometimes, it doesn't look like there's anything as dark and forbidding as a coal mine, unless it's the inside of Ruby's C implementation. Recently, Sean Carley and I have been poking around in there while we work on checkr. Fortunately, zenspider and Eric Hodel have been a wonderful pair of guides for us. We started working with case statements, and we quickly ran into big problems. It turns out that parse_tree doesn't deal well with things like

case;
when foo == 1; puts "it was one"
when foo == 2; puts "it was two"
end

Digging into it has meant grabbing gdb, gprof, and looking at lots of parse_tree output. It's turned Sean into another fan of unit_diff. It's also given me a chance to figure out a lot more about C than I thought possible in such a short time. Who would have thought that this would have been part of troubleshooting a Ruby issue:

$ gdb ~/ruby-debug/bin/ruby
GNU gdb 6.3
Copyright 2004 Free Software Foundation, Inc.
GDB is free software, covered by the GNU General Public License, 
and you are welcome to change it and/or distribute copies of it 
under certain conditions.
Type "show copying" to see the conditions.
There is absolutely no warranty for GDB. Type "show warranty" 
for details.
This GDB was configured as "i586-suse-linux"...Using host libthread_db/library/lib/tls/libthread_db.so.1".
(gdb) set args 'good_case.rb'
(gdb) b eval.c :2949
Breakpoint 1 at 0x8056ea2: file eval.c, line 2949.
(gdb) run
Starting program: /home/eylerpm/ruby-debug/bin/ruby 'good_case.rb'
1
Program exited normally.
(gdb) set args 'case.rb'
(gdb) run
Starting program: /home/eylerpm/ruby-debug/bin/ruby 'case.rb'
Breakpoint 1, rb_eval (self=1075685744, n=0x401cb024) at eval.c:2949
2949 while (node) {
(gdb) c
Continuing.
Breakpoint 1, rb_eval (self=1075685744, n=0x401cb024) at eval.c:2949
2949 while (node) {
(gdb) c
Continuing.
Breakpoint 1, rb_eval (self=1075685744, n=0x401cb024) at eval.c:2949
2949 while (node) {
(gdb) c
Continuing.
nothing matched
Program exited normally.
(gdb)

But it turned out to be just the key Sean and zenspider needed to track down the issue in parse_tree.