less code, more software

Here's the slide-deck on pair-programming I recently presented for a client.

[フレーム]

I did a very enjoyable live coding presentation using cyber-dojo at XP Days Ukraine last week. At the start I asked the XP question and, as usual, the audience told me lots about the XP practices (such as pair-programming) but struggled to name the four XP values, and failed to name courage.

On more than one occasion, while consulting or training at a customer's site trying to help explain and convey "an agile mindset", I've been told

you need to tell this to our managers

If I did tell their managers I'm pretty sure I know what would happen.
Can you guess?
Scroll down for my answer.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
I think their managers would say

you need to tell this to our managers

.
.
.
.
.

Last week, at a clients site, I noticed an old, coffee-stained copy of the Cutter IT Journal. It was titled "XP and culture change", dated September 2002. Here are some quotes from it.

From Kent Beck:

Because culture embodies perception and action together, changing culture is difficult and prone to backsliding.

Is it easier to change your perception or go back to designing the old way?

From Laurent Bossavit:

A process change will always involve a cultural change.

We were also a culture of Conviviality, which you could easily mistake (as I did at first) for a culture of Communication... In Conviviality what is valued is the act of sharing information in a group setting - rather than the nature, quantity, or quality of the information thus shared.

Culture is what remains when you have forgotten everything else.

From Mary Poppendieck and Ron Moriscato:

If there were one thing that Ron's team would do differently next time, it would be to do more refactoring.

XP is a process that doesn't feel like a process.

The theory of punctuated equilibrium holds that biological species are not likely to change over a long period of time because mutations are usually swamped by the genes of the existing population. If a mutation occurs in an isolated spot away from the main population, it has a greater chance of surviving.

From Ken Schwaber:

Agile process management represents a profound shift in the development of products and software. Agile is based on an intuitive feel of what is right, springs from a completely different theoretical basis than traditional development processes, and is in sum a wholly different approach to building products in complex situations.

From Matt Simons and Chaitanya Nadkarny

A fixed-bid contract changes the very nature of the relationship between customer and vendor from collaborative to "contentious". "Embrace change" undergoes a fatal transformation into "outlaw change."

There is no way to pretend everything is fine when you have to deliver software to your customer every few weeks.

From Nancy Van Schooenderwoert and Ron Moriscato:

The advantages of pair programming hit you hard and fast. As you explain an area of code to your partner, you get a deeper understanding of how it fits into the current architecture. You're your own peer reviewer!

After pair programming for a while, we found ourselves in a situation where the entire team had worked somewhere in the module in the recent past. Code reviews became exciting idea-exchanging periods where refactoring tasks were discussed and planned.

With schedule pressure, there is a huge temptation to put off refactoring, and we did too much of that.

It's not enough for the code to work; it also has to serve as a solid base for the next wave of features that will be added.

All through the project, a frequent cause was that unit testing wasn't thorough enough.

is an excellent book by Carlos Castaneda (isbn 978-0140192384). As usual I'm going to quote from a few pages

By experiencing other worlds, then, we see our own for what it is and are thereby enabled also to see fleetingly what the real world, the one between our own cultural construct and those other worlds, must in fact be like.

There is nothing wrong with being afraid. When you fear, you see things in a different way.

Fear is the first natural enemy a man must overcome on his path to knowledge.

You dwell upon yourself too much. That's the trouble. And that produces a terrible fatigue.

Are you angry at me don Juan? I asked when he returned.
He seemed surprised at my question.
No! I'm never angry at anybody! No human being can do anything important enough for that. You get angry at people when you feel their acts are important. I don't feel that way any longer.

What will happen to the man if he runs away in fear?
Nothing happens to him except that he will never learn.
He will never become a man of knowledge. He will perhaps be a bully or a harmless, scared man, at any rate, he will be a defeated man. His first enemy will have put an end to his cravings.
And what must he do to overcome fear?
The answer is very simple. He must not run away. He must defy his fear, and in spite of it he must take the next step in learning, and the next, and the next. He must be fully afraid, and yet he must not stop. That is the rule! And a moment will come when his first enemy retreats. The man begins to feel sure of himself. His intent becomes stronger. Learning is no longer a terrifying task. When this joyous moment comes, the man can say without hesitation that he had defeated his first natural enemy.
Does it happen at once, don Juan, or little by little?
It happens little by little, and yet the fear is vanquished suddenly and fast. But won't the man be afraid again if something new happens to him? No. Once a man has vanquished fear, he is free from it for the rest of his life because, instead of fear, he has acquired clarity - a clarity of mind which erases fear. By then a man knows his desires; he knows how to satisfy those desires. He can anticipate the new steps of learning, and a sharp clarity surrounds everything. The man feels that nothing is concealed.

The freedom to choose a path imparted a sense of direction through the expression of personal inclinations.

Exertion entailed not only drama, but also the need of efficacy. Exertion had to be effective; it had to possess the quality of being properly channelled, of being suitable.

To become a man of knowledge was a task that could not be fully achieved; rather, it was an unceasing process comprising (1) the idea that one had to renew the quest of becoming a man of knowledge; (2) the idea of one's impermanency; and (3) the idea that one had to follow the path with heart.

is an excellent book by Laurie Williams and Robert Kessler. As usual I'm going to quote from a few pages

Programmers admit to working harder and smarter on programs because they do not want to let their partner down.

The pair results were also more consistent, while the individuals varied more about the mean. Individuals intermittently didn't hand in a program or handed it in late; pairs handed in their assignments on time.

Widespread use of pair programming involves a cultural shift in values of the organization - away from individual and toward team recognition and goals.

We have observed that effective pair programmers communicate with each other at least once a minute.

Interestingly enough, the more experience a developer has, the more likely he or she is to ask for help; novices are less likely to ask for help.

We used to consider a new person unproductive for their first three months. Now, we find that new people can help out almost immediately.

We still feel that a novice pair is a better alternative to a solo novice.

We believe pair programming is an integral part of XP, and it is dangerous to do XP without doing pair programming.

Many years ago, in a taxi, in Athens, I watched with a mixture of amazement and fear as my driver freed up both hands by steering with his elbows. It was quite an experience!

Multi-tasking is a bad idea when you're doing tasks requiring "immersion". After being interrupted it takes you a long time to get back to where you were. One of the least talked about reasons why pair-programming can be so effective is that a pair seems to be much more resilient to interruptions than an individual. In other words, yet again, pair-programming is partly about programming, but it's mostly about the pairing.

Jerry Weinberg observed that if you have two task to choose from you don't in fact have two tasks to choose from. You have three. Your third task is deciding which of the other two tasks you should tackle!

Recently, on a train, a man sitting opposite me was reading The Telegraph. An article on the front page about multi-tasking caught my eye. It quoted some research by Professor David Strayer from the University of Utah. It said multi-taskers often end up juggling activities not because they are good at it, but because they are easily distracted and cannot concentrate on the job at hand. And in contrast, the most efficient multi-tasker is the person least likely to do so because they can focus on one thing at a time. The implication is that someone who claims to be good at multi-tasking probably isn't!

is the title of an excellent book by Jerry Weinberg. As usual I'm going to quote from a few pages:

Testing a system is a process of gathering information with the intent that the information could be used for some purpose.

Without a process that includes regular technical reviews, no project will rise above mediocrity, no matter how good its machine-testing process.

At least half your testing costs can be cut before anybody ever runs a test, if only your systems are designed with testability in mind.

"We absolutely need this software in place twenty-four weeks from tomorrow. We need two weeks to staff up and get approvals. Then we'll need four weeks for requirements, four weeks for architecture. four weeks for design, and eight weeks for coding. That adds up to twenty-two weeks, so we'll have two weeks left for testing."

If you have ten kilograms of pure uranium-235 and you add another ten kilograms, you'll have twenty kilograms. But if you do this a few more times, you won't have fifty kilograms, you'll have a nuclear explosion. One plus one doesn't always equal two.

The more bugs you find, the more you're going to find, not the other way around.

The human mind craves meaning. If you feed people a random bit of data, they'll struggle to divine meaning from it - and they'll move from the intake phase to the meaning phase so fast they won't be aware of doing so.

You have to know what you're expecting before you give meaning to a test report, otherwise everything looks or sounds right. That's why I'm a strong advocate of the test-first philosophy, whereby developers write their tests to include expected results before they write a line of code. It's what we did fifty years ago, but the practice was gradually lost when industry trends separated testing from development.
That separation occurred initially because it's psychologically difficult for people to test their own programs. There's still significant risk if you rely on test-first without pair-programming or some other process that casts more than one pair of eyes, and more than one brain, on a program.

If test-first is a good idea, then significance-first is even better. Why? … if you actually perform even an enormous number of tests, you would likely lose the valuable information among all the worthless crud. The number of tests performed should be as small as possible, but no smaller.

"We didn't have any problems until we started testing. We were right on schedule. Testing screwed up everything."

"What the American public wants in the theatre is a tragedy with a happy ending." [William Dean Howells]

is an excellent book by Hans Ulrich Rudel (isbn 978-1908476876). As usual I'm going to quote from a few pages:

He tells me that a large scale offensive is in preparation in my sector... approximately three hundred tanks are to be employed in this operation... The number three hundred flabbergasts me... I reply that I find some difficulty in believing it... He says to me, half in earnest, half in jest: "If I didn't know you, for two pins I would have you put under arrest for saying such a thing. But we will soon find out." He goes to the telephone and is connected with the Chief of the General Staff. "You have just given the Fuhrer the figure of three hundred tanks for operation X." "Yes I did." "I want to know the names of the divisions concerned with their present strength in tanks. I have somebody with me who is well acquainted with the position." ... the Chief of the General Staff has the bad luck to begin with the 14th armoured division. He says it has sixty tanks. Goering can hardly contain himself. "My man reports that the 14th has one!" A lengthy silence at the other end of the line. "When did he leave the front?" "Four days ago." Again silence. Then "Forty tanks are still on their way to the front. The rest are in repair shops on the line of communications, but will certainly reach their units by zero day, so that the figures are correct." He has the same answer for the other divisions. The Reichsmarschall slams down the receiver in a rage. "That is how it is!" The Fuhrer is given a totally false picture based on incorrect data and is surprised when operations do not have the success expected... The South Eastern zone with its network of communications is being incessantly blanketed by the enemy's bomber formations. Who knows how many of those forty tanks, for example, will ever reach the front or when? Who can say if the repair shops will get their spare parts in time and if they will be able to complete their repairs within the specified time?

In ministries and departments, however, mistakes are denied on principle.

Another confirmation of the truth of our old Stuka maxim: "Nothing comes off - except what you have practised."

We have long since ceased to develop practice from theory; we do just the opposite.

The fitters have their hands full, for the aircraft have been heavily damaged by flak. The life of such an aeroplane will always be limited.

Little by little I discover all the tricks. Skill is often the result of getting hurt.

I now see that perfectly plainly. We are alone to possess this knowledge; the responsibility is ours.

My very good friend Syver was running a coding dojo in Erlang. A test for the filter exercise looked like this:

odd(Integer) when Integer rem 2 == 1 ->
 true;
odd(_) ->
 false.
filter_test() ->
 ?assertEqual([3, 1], 
 filter(fun(Elem) -> odd(Elem) end, [1, 2, 3, 4])). 

I don't know Erlang at all, so Syver helped me out with this first-cut version:

filter(_, []) ->
 [];
filter(Fun, [Head|Tail]) ->
 filter_helper(Fun(Head), Fun, Head, Tail, []).
filter_helper(true, Fun, TrueElement, [Head|Tail], Result) ->
 filter_helper(Fun(Head), Fun, Head, Tail, [TrueElement|Result]);
filter_helper(true, _, TrueElement, [], Result) ->
 [TrueElement|Result];
filter_helper(false, Fun, _, [Head|Tail], Result) ->
 filter_helper(Fun(Head), Fun, Head, Tail, Result);
filter_helper(false, _, _, [], Result) ->
 Result

I realized Erlang is a lot like Prolog which I knew in the dim and distant past. I stared at the code and after several minutes something about it started nagging me. It was the splitting of the list into it's Head and Tail elements in both filter and filter_helper. I wondered if this duplication could be avoided by making filter_helper call back into filter. After several false attempts I came up with:

filter(_, []) ->
 [];
filter(Fun, [Head|Tail]) ->
 filter_helper(Fun(Head), Fun, Head, Tail).
filter_helper(true, Fun, TrueElement, List) ->
 X = filter(Fun,List),
 [TrueElement|X];
filter_helper(false, Fun, _FalseElement, List) ->
 filter(Fun, List).

Then Syver showed me how the use of X could be collapsed:

filter(_, []) ->
 [];
filter(Fun, [Head|Tail]) ->
 filter_helper(Fun(Head), Fun, Head, Tail).
filter_helper(true, Fun, TrueElement, List) ->
 [TrueElement|filter(Fun, List)];
filter_helper(false, Fun, _FalseElement, List) ->
 filter(Fun, List).

We did some argument renaming:

filter(_, []) ->
 [];
filter(Fun, [Head|Tail]) ->
 filter_helper(Fun(Head), Fun, Head, Tail).
filter_helper(true, Fun, Head, List) ->
 [Head|filter(Fun, List)];
filter_helper(false, Fun, _, List) ->
 filter(Fun, List).

Then Syver noticed that we could pass Head,Tail as a single [Head|Tail] argument:

filter(_, []) ->
 [];
filter(Fun, [Head|Tail]) ->
 filter_helper(Fun(Head), Fun, [Head|Tail]).
filter_helper(true, Fun, [Head|Tail]) ->
 [Head|filter(Fun, [Head|Tail])];
filter_helper(false, Fun, [_|Tail]) ->
 filter(Fun, Tail).

We agreed that filter_helper was better as filter:

filter(_, []) ->
 [];
filter(Fun, [Head|Tail]) ->
 filter(Fun(Head), Fun, [Head|Tail]).
filter(true, Fun, [Head|Tail]) ->
 [Head|filter(Fun, [Head|Tail])];
filter(false, Fun, [_|Tail]) ->
 filter(Fun, Tail).

As a final polish Syver refactored to this:

filter(_, []) ->
 [];
filter(Fun, [Head|Tail] = List) ->
 filter(Fun(Head), Fun, List).
filter(true, Fun, [Head|Tail]) ->
 [Head|filter(Fun, [Head|Tail])];
filter(false, Fun, [_|Tail]) ->
 filter(Fun, Tail).

and then, again, after the dojo, thanks to Syver's comment below, to this:

filter(_, []) ->
 [];
filter(Fun, [Head|_] = List) ->
 filter(Fun(Head), Fun, List).
filter(true, Fun, [Head|Tail]) ->
 [Head|filter(Fun, [Head|Tail])];
filter(false, Fun, [_|Tail]) ->
 filter(Fun, Tail).

We agreed that the final version was definitely better than either of us could have come up with individually.

At the recent Agile on the beach conference I was chatting to Paul Massey who runs BlueFruit Software in Redruth (which, by the way, is a great company to work if you're looking for a job as an embedded developer). I was reminded of something Paul mentioned to me a while back that relates to pair-programming. He and a friend played go against other people on a server. Their go rankings were 8k and 9k respectively. Interestingly, they then played against other people as a pair. And their joint-ranking went up to 6k. Paul writes:

When I play go, I have a problem that I struggle to find the patience for strategic thinking. I combat this problem by doing strategic thinking when my opponent is playing and tactical thinking when it's my turn. Yesterday I realised the parallel with pair programming, i.e. tactical thinking when at the keyboard, and strategic thinking when away from the keyboard. Both are important.

is an excellent book by Ansgar Gerstner (isbn 978-988-18154-7-7). As usual I'm going to quote from a few pages:

Taoism emphasizes compassion, spontaneity, and respect for nature.

Taoism believes in a kind of self-organizing system, that there are patterns to the world and its affairs, and that it is best to let these patterns operate without interference.

Wuwei is closely linked with this idea. In order to be able to apply wuwei - to "act by not acting" - you need good intuition and instinct and be excellent at sensing what is going on.

You have to be careful to recognize the limitations of your knowledge and not interfere with processes in the false belief that you completely understand a situation and can predict precisely the outcome of a certain action.

The point of karaoke is not singing, which is almost always awful. It is getting to know your partners.

The focus in the Daodejing is not "adding", it is on "reducing".

It is often the fear of losing control that makes people add issues and complexities to a situation.

Keeping things simple is an art. And, as with any art, simplicity needs to be cultivated.

You will not be dealing with the abstract notion of a business, but with real individuals.

What makes water "invincible" is that nothing can change its inherent quality of utmost adaptability and agility.

An effective way to maintain flexibility and with it creativity, is to foster a learning culture in your business.

It is important in all kinds of contexts not to hold on to things too tightly.