A key property of agile development is figuring out how to make a system go live with a small subset of features. We build software for the business value it offers, the quicker we go live, the faster we get at least some of that business value.
My colleague Dave Leigh-Fellows told me one of my favorite examples of this kind of thinking. It came when we has working for a brokerage firm. They had a new kind of product that they wanted to get into the market. The full software support for this was a web page that the customer filled in that generated the necessary transactions against the back-end system. But Dave came up with a way to get the product into the market faster than that.
The first two versions may not have been the most elegant solutions ever conceived, but they did get the product into the market much more quickly. I've not come across any other examples of iterative development that use roller-skates, but that may be more due to lack of imagination rather than lack of need.
Like most Brits my age I grew up with a sci-fi Children's program on BBC called Doctor Who. (For those who know, my doctor was Jon Pertwee, although I also saw a good bit of early Tom Baker.) It was actually the longest-running sci-fi TV series in the world, running from 1963-89.
A few years ago it was revived in the UK and has become a big hit - and not just for children. Doctor Who always had themes and scripts that went beyond the children's' audience and the series developed a huge fan base that lived off books and audio series even when the TV series died. The revival follows this with shows that are written to appeal as much to adults as kids. It was really great to sit with a couple my age and their 8 and 10 year old daughters and enjoy the new series. The scripts and acting are good, the only change is that the special effects are also good now (the old special effects made classic star trek look high-tech).
At home we don't watch much telly, the last shows we watched regularly were Buffy and Angel. Cindy, being American, had never seen Doctor Who growing up, but she loves the new series. When we get a new set of DVDs there's usually several nights of "it's late, we're tired, but maybe we can do one more".
If you've never seen Doctor Who the place to start is the opening episode of the revived series: Rose. (Wikipedia has mind-bogglingly comprehensive coverage, but I won't link from here as it's naturally full of spoilers.) Rose not only introduces the set up you'll need for other episodes (who the Doctor is, what the TARDIS is) but also does a really good job of distilling the tone of Doctor Who, capturing the mix of adventure and comedy.
If you like Rose then you can either carry on with the full first season or cherry pick highlights. If you prefer the latter I'll suggest my favorites. My big favorite from the first series was the two part The Empty Child / The Doctor Dances. I rate this as better than most films I've seen, certainly better than most TV. (It won a Hugo award so it's not just me.) It was written by Steven Moffat who is also known for writing the comedy series Coupling. Almost as good is Dalek. It lacks the humor but scores due to a wonderfully intense performance from Christopher Eccleston. I also really like the final two part (Bad Wolf / The Parting of the Ways) but you really need to see the whole series to appreciate it properly. (A tip if you do watch the whole series: don't watch the trailer for Bad Wolf (it appears at the end of Boom Town) as it gives away an important part of the plot of Bad Wolf.)
The second series has a different actor playing the Doctor (they have a nice technobabble rationalization to allow them to change actors easily). It doesn't quite hit the high spot of The Empty Child but is still really good. My suggestions for cherry pickers here would by The Girl in the Fireplace (another Moffat Hugo win) and The Impossible Planet / The Satan Pit.
When I said the second series didn't quite hit the heights of The Empty Child, I'm not being very reasonable because those two episodes are far too good for any TV series to live up to. However the third series (not yet all broadcast in the US - it's good to have friends in the UK) hits those high notes twice. Human Nature / The Family of Blood is a super two-parter that threatens to take away Steven Moffat's crown of writing the best episode. Moffat's response is Blink, which is as good a 45 minutes of TV as you could ever hope for. Not just has it got a great story and some cracking humor, it also achieves Doctor Who's higher purpose. You see Doctor Who is only secondarily about entertainment, it's primary purpose was always to scare the living daylights out of small children. I may be too old now to get behind the sofa, but I do remember how much I enjoyed it.
I was in Boston, about to fly out to our office in Calgary. I look at my calendar to see if I have a meeting. First one is at 10.30am - cool no need to rush out of bed in the morning.
I turn up and am two hours late. What happened was that I was invited to the meeting that begun at 8.30am Calgary time. Lotus Notes saw my computer was set to Boston time, and helpfully converted the time zones for the two hour shift.
You could argue that this was my fault for not paying attention. After all I know this is how Notes works and was being careless when I read my diary. I don't buy this, Donald Norman noted a while ago that we tend to blame ourselves for errors that are due to bad usability - like a door with a handle that you should push.
Time zones are particularly vulnerable to this kind of problem It's most notably a problem in calendaring applications, but you see this issue in enterprise software as well. There is a temptation to try to be clever with handling time zones, but this temptation leads to trouble if the software isn't quite clever enough, which is what happened in this case.
I'd prefer Notes ignore time zones completely. You set the time for the place the meeting occurs in and that's what it should show you. Who cares about the time zones? It's only the time on the ground that usually matters. When I look at my calendar for a day in Calgary, I want to see the Calgary times for that day - wherever I happen to be when I'm looking at it.
The exception, of course, is a phone meeting that spans time zones. But here you should do something exceptional for a phone meeting rather than complicate handling a physical one. It could be something that allows you to set a flag for a phone meeting, and then you get a different time display. Or it could be as simple as just allowing you to put the timezone on the time display, and leaving the conversion to the reader. With phone meetings you think more about time-zones than you do for physical meetings (or at least I do).
The important lesson here is to make the most common case (physical meetings) simple and only do complicated things for less common cases as exceptions (quite possibly manual exceptions). Calendaring time zones get into trouble because they make the common and simple case be more complex. The problem occurs because the designers wanted to use the same data for both the simple and phone cases - but that just gets the simple case in trouble.
Usually when I hand out a prize for worst-user experience Lotus Notes is at the top of the queue. (Indeed I find it embarrassing to admit that ThoughtWorks uses the damn thing.) But the worst time-zone experience award goes to Microsoft Office, although to be fair this was many years ago. I had recently bought a PDA (running Windows CE version 2). I had put some all day meetings into my calendar, flown to Chicago, and to get the PDA to alert me to meetings I changed the timezone of the PDA to one hour earlier.
Suddenly every one of my all day meetings shifted to a day earlier. This was due to a catalog of errors. First off they had stored an all day meeting as a meeting from midnight to midnight - that's the kind of representation error on a TimePoint that often gets people into trouble. Then it was compounded by shifting the times of meeting when I changed time zones - so all day meetings were now 11pm to 11pm. This, of course, is due to putting the time zone into the meeting so it looks like it shifted when I changed time zone on the PDA. Then to cap it off, the software was clever enough to know that an all day meeting should only show the day of the meeting, but the day it chose was the day of the start of the meeting - that was now one day earlier. That's the poor timepoint representation biting back.
I was in the Calgary office last week and had a good chat with John Kordyback, one of our most trusted technical leads. He's worked on, and dug into, a number of travel loyalty software systems (frequent flyer/sleeper etc) and we talked about the nature of these kinds of things and how to think about them in a more fruitful manner.
The core of of a loyalty system is a system to keep track of points (or miles). This should allow customers to see their points and also for the company to manage the unredeemed points. Although it seems that most people don't see it this way, this is essentially an accounting system, just switching points for dollars. John's observation was that repeatedly he runs into what people see as difficult problems that are much easier to deal with once you put accounting spectacles on.
An example of this is dealing with ad-hoc changes. However good your automated rules processing is, there always cases when something odd happens and you have to intervene manually. The result for many systems is and ad hoc change to totals that is error prone and unaudited. With an accounting frame of mind, however, you look at these changes as accounting adjustments and the patterns for this are well understood.
A notable difference between a loyalty program and most accounting systems is that a loyalty program is more about managing liabilities rather than managing assets. Hence there's more focus on things like risk management, as well as common themes like taxes and revenue reporting.
Many loyalty systems have multiple kinds of points, such as such as regular miles and elite qualifying miles. This is a common point of complexity. If you use an accounting viewpoint, however, you can track these easily as multiple currencies.
An interesting twist on this is potential points. If I book a flight for next month, the airline needs to know that there are miles I will earn when I fly next month (potential miles). These potential miles affect their liabilities. However it's only when I fly that they turn into real miles. Again accounting thinking can help here, we can use multiple currencies again, or use an accounts payable notion. The mechanisms are there and well understood, we just have to apply the model to the situation.
We fleshed this out in practice where we also found it really helpful to use TestDrivenDevelopment. A group of people spent a couple of weeks trying to sort out potential miles with planned design, but the core issue was cracked in a couple of days with TDD. The crucial part of this was focusing on examples to make the problem concrete.
The accounting analogy also applies, although partly less directly, to deciding how to award miles for activity. Any program has activity rules that need to be very flexible and need to cope with constant changes to the loyalty program. We can look at this as following the model of domain events triggering accounting entries through using Agreement Dispatchers. This is a pattern John and I have used lots of times and works well to these kinds of changing rules. Essentially we have agreements that represent the overall program rules for a class of participant. Each agreement consists of a set of posting rules keyed by the type of event and a date range. When an domain event occurs (a hotel stay) we look up the agreement dispatcher for the customer, and use the event to look up the right posting rule. We then run the posting rule to create the appropriate accounting entries to represent the miles for the event. The time dating of the events allow us to change posting rules over time but still be able to handle old events and correctly do automated processing of adjustments. (Some day I'll finish writing up these patterns, but what I have on the web is hopefully enough to give you some ideas.)
The second aspect of a loyalty system is tracking the customer experience. Since the accounting requires the system to record the customer's activity, the loyalty system acts as a natural base to learn from the customer's interactions with the company. Much of this is data mining - looking for patterns in customer behavior which can lead to new products and promotions. You can also use this activity history to assess the success of promotions - if you offer a mileage bonus for flying a route what is the response like?
Like me, John is a strong proponent of using ReportingDatabases, and this is a good fit for this kind of problem. The accounting side needs a very different set of data structures and uses regular updates as activities occur. The customer experience analysis is all read only, so you can use less normalized structures with regular, but not necessarily real time, feeds from the accounting side.
Taking it further, it seems reasonable to completely decouple the accounting and customer experience systems. They are both usually lodged together as a single customer loyalty system because they track the same events. Yet since they differ so much on the inside it may make more sense to treat them as two separate systems that feed off the same event stream (the accounting side would probably generate some events for the customer experience side too).
One of the habits of customer experience tracking is frequent changes to the system to support new kinds of analysis. We speculated that we could try an approach that had a single stored event log of customer activity, and plug in relatively independent 'miners' that would transform selected information from the log into more particular data structures to do different kinds of analysis. The miners could be relatively independent of each other and thus easier to build.
As you can see, our discussion did shift from looking at John's experiences to some of our joint speculations about how a system like this could be built in the future. What's clear to us is that there is a lot of room for exploring new ideas in this space that could introduce a new set of abstractions that would lead to systems that can provide better support to this business activity. More and more attention is being paid to this these days, so this seems like a fruitful territory for us to work in.
Is changing the interface of part of the code a refactoring?
The answer to this question is pretty simple - changing an interface is a refactoring providing you change all the callers too. A great example of this is Rename Method, which is an interface changing refactoring present on pretty much all refactoring tools.
The changing of all the callers is an essential part of this refactoring. Just changing an interface declaration will break the system - and thus isn't a behavior preserving change.
Interface changing refactorings do assume that you can get hold of all the callers, which is why you have to be much more careful with PublishedInterfaces. With a published interface, the interface itself is part of the observable behavior of the system.
Dynamic languages can make these changes much more awkward. Static typing really does help here in pinning down exactly which interface is being called at various points. Reflective calls that can also make it harder to find, either by embedding method names in strings or even composing them at run-time. This is another area where tests are essential even in environments that have refactoring tools.
Should we strive to only have one language in our development efforts?
Throughout much of the last decade the fashion in the enterprise software world has been to focus on one standard language for software development efforts. Many development organizations strive to do all their work in Java (or C#/VB).
The rationale for this is that developers find it hard to be proficient in more than one language. Sticking to a single language lowers the learning burden, particularly when hiring new people.
There's some truth to this, but also much that's missing. The programming environment is partly language, but also about languages and frameworks. Larger frameworks, Hibernate, Struts, ADO, present as much of a challenge to learn as a language even if you program them in a single host language. Often the difficulty of expressing what you need in the host language is sufficiently awkward that many frameworks resort to configuration files, which are effectively external DomainSpecificLanguages written in XML - which adds a jigger of 80 proof ugliness to them.
For many developers, the one-language notion is a sign of lack of professionalism. This is best exemplified by the Pragmatic Programmers' advice to learn a new language every year. The point here is that programming languages do affect the way you think about programming, and learning new languages can do a lot to help you think about solving problems in different ways. (It's important to learn languages that are quite different in order to get the benefit of this. Java and C# are too similar to count.)
I agree with prags' advice here, as in most things. But I also sympathize with the overhead of learning a new language. My personal scripting is pretty much all done with Ruby and I've been loath to any more than play with reasonable alternatives like Python, Groovy, or PowerShell. It's not that it's hard to use the alternatives, but with Ruby I know too much that I'd have to look up with the alternatives.
The important point here is that when I'm writing these scripts I'm not manipulating new abstractions. Much of what I do is fiddling with text, the file system, and hunks of XML or YAML. If I need to take on a sizable new abstraction, the cost of learning it as a library isn't really much less than the cost of learning a language to manipulate it. If I want to specify a directed graph structure for display, learning Graphviz's Dot language is hardly more work than learning a new Ruby library.
Using a DSL instead of a library can offer us better ways of manipulating our abstractions. This makes it easier to see what we've written and to reveal our intentions. An API is like declaring a vocabulary, a DSL adds a grammar which allows you to write coherent sentences.
This argument is strong for DSLs, but does it also apply to general purpose languages? If you are working in Java (or soon C#) does it make sense to use Ruby now it's available on your platform?
The last decade has seen the rise of memory managed C-based languages. People saw that, despite many years of skepticism, memory management makes life sufficiently better that it was worth stepping away from C and C++ in the enterprise world. A common platform and language also pulled people away from proprietary walled gardens like Powerbuilder and Delphi.
Now there is a similar question. Are modern scripting languages another step forwards? Do we prefer their well-chosen terseness? Time and time again I hear experienced Java and C# developers report they are more effective in Ruby - which is why I've been encouraging Ruby. It wouldn't surprise me if similar reports appear in the next few years about other languages too.
A decade ago I was talking to my old friend Tom Hadfield at OOPSLA 96. Java's rise was apparent and it was clear that Smalltalk's future was doomed. Despite my love of Smalltalk I was pretty sanguine. I felt that Java gave people enough of what they needed; while it wasn't quite as nice as Smalltalk it was enough of an improvement over C++, particularly with memory management, for me to be happy with it. Tom disagreed, he felt there was something fundamentally different about the expressiveness of Smalltalk, the way you could better capture the intention of what you were doing directly in your code - closing the gap between domain knowledge and programming.
In the intervening years I've come to the view that Tom was right after all. After several years in curly brace land, Ruby reminded me of what I was missing. There's a clarity to reading Ruby code that just makes it an easier medium to work with, despite the inferior tooling. I'm way more sympathetic to the Smalltalk holdouts than I felt then, even though I haven't felt inclined to open an image in anger for a long time.
So are we returning to the language cacophony of the late 80's and early 90's? I think we will see multiple languages blathering away, but there will be an important difference. In the late 80's it was hard to get languages to inter-operate closely. These days there's a lot of attention to making environments that allow different language to co-exist closely. Scripting languages have traditionally had an intimate relationship with C. There's much effort to inter-operation on the JVM and CLR platforms. Too much has been invested in libraries for a language to ignore them.
So my sense is that we will see multiple languages used in projects with people choosing a language for what it can do in the same way that people choose frameworks now. I agree with Neal that we are entering a period of Polyglot Programming.
There are few things more frustrating than spending hours trying to install a piece of software and then having to delete everything and start again. Today at 9.30 I began installing openArchitectureWare, I finally had it installed (I think) at 15.30. So I thought I'd write this to help someone else do it more quickly.
OpenArchitectureWare is a set of tools, based on Eclipse, to support Model Driven Development. I'm interested in exploring some of its tools that are oriented towards DomainSpecificLanguages. (Xtext - which helps you develop textual languages - is something that's specifically been pointed out to me as worth looking at.) I don't know how worthwhile these tools are yet, after all it took me most of the day just to install the dratted thing, but we'll see.
One of my problems with the installation was that I'm not an Eclipse user - my usual Java IDE is IntelliJ. To install openArchitectureWare you need to know how to deal with the plugin system in Eclipse - and I'd never done anything with Eclipse before so that was new to me.
The first step was the easiest one - install Eclipse. I installed it on my Ubuntu machine, so all I had to do was wajig install eclipse (wajig is a unified command-line for various debian packaging and sysadmin tools). Then all hell broke loose. Rather than go through my miserable morning, I'll explain what I would do now.
The trouble with OpenArchitectureWare is that it has dependencies, other eclipse plugins that need to be installed before it can work. As anyone with experience in these things knows, sorting out dependencies can be a right pain without a good tool. apt-get for Debian and gem for ruby are examples of a good tool that resolves dependencies. When I installed eclipse, apt-get knew it had to pull down a whole host of dependencies and installed them for me. The situation in Eclipse is not so good.
To install openArchitectureWare you need a bunch of plugins: EMF, UML2, ATL, and GMF. I couldn't see from the web pages exactly how to get these things, or if they had their own dependencies.
There are several ways of installing plugins in Eclipse, although I had to hunt a bit for instructions. The easiest way is a menu option in Eclipse itself. In the menus pick [Help -> Software Updates -> Find and Install] (no I don't know why it's on the help menu). With a bit of button pushing you can get it to download a list of packages - the relevant source is the Callisto Discovery Site. Once you have that list downloaded look in the Models and Model Development section and select Eclipse Modeling Framework (EMF) and Graphical Modeling Framework (GMF). You'll get an error message saying that these have an unresolved dependency. Take note of the button on the right that says 'select required'. Hit it and it will find the dependency to GEF and its dependency on Batik. If you don't see that button and hit it you'll have a frustrating time trying to find them (believe me, I know).
That gets two of openArchitectureWare's dependencies. The others, and openArchitectureWare itself need to be done the harder way. Digging around the eclipse site I found the relevant web pages for UML2 and ATL. These need to be downloaded as zip files as does openArchitectureWare itself.
When you unzip the UML2 and openArchitectureWare folders they unzip into a folder called eclipse that contains subfolders for plugins and features. You can take the contents of these folders and put them into corresponding folders on your load environment (in my case /usr/local/lib/eclipse). As that didn't work for me when I tried it first, I found another way.
The way to tell if stuff has installed properly is to go to [Help -> Software Updates -> Manage Configuration]. When you open that you have the option of "Add an Extension Location". An extension location is (almost) any directory that contains an eclipse folder with subfolder for plugins and features. I say almost because the eclipse folder also needs a file called .eclipseextension. This is just an empty file so you can create it with touch .eclipseextension. What I did is created folders in /usr/local/lib for openArchitectureWare and uml2-eclipse, moved the unzipped eclipse folders in there, did touch .eclipseextension inside each of them and then added them using "Add an Extension Location". ATL just produces a plugin directory so I copied the contents of it into the plugin directory for openArchitectureWare.
It's important that you do this after you use the Find and Install tool because if you do it first, the Find and Install tool will tell you have an unresolved dependency and refuse to do anything until you fix it. When I was all installed it tells me "UML2 End-User Features (2.1.1.v200707181556) requires plug-in "org.eclipse.emf.ecore.xmi (2.3.0)". I don't know how to fix this and I have a bunch of emf.ecore jars present in EMF. However the rest of eclipse seems to work so far, so I'm carrying on regardless.
In the summer of 2006 I did a major chunk of work on UI patterns. Since then they've been very much on ice as my primary writing focus has shifted (although not very visibly) to DomainSpecificLanguages. On this page I'll keep a note of links to writings I've liked that are connected with that work.
Over the last few weeks Jeremy Miller has been writing an excellent series of posts on UI design in C#.
(See my note on DomainSpecificLanguage for a quick intro to this topic and my terminology on it.)
Update:David Laribee has written a post contrasting what he calls ordered and unordered fluent interfaces. The distinction is that ordered fluent interfaces force a particular flow on how you compose your DSL sentence. He provides an example where he uses multiple interfaces on a single ExpressionBuilder - the same technique that's used by JMock.
Anders Norås has written two interesting articles on writing internal DSLs in C#. The first article gives a sample of the DSL and a discussion against Chromatic's cynical check-list. The second article goes into details about its implementation.
Piers Cawley makes the point that a key characteristic of DSLs is their narrow focus on a domain.
Is it worth the effort to design software well?
From time to time I have indirect conversations about whether good software design is a worthwhile activity. I say these conversations are indirect because I don't think I've ever come across someone saying that software design is pointless. Usually it's expressed in a form like "we really need to move fast to make our target next year so we are reducing ".
In there is a notion that design is something you can trade off for greater speed. Indeed I've come across the impression a couple of times that design effort is tolerated to keep the programmers happy even though it reduces speed.
If it were the case that putting effort into design reduced the effectiveness of programming I would be against it. In fact I think most software developers would be against design if that were the case. Developers may disagree on what exactly is good design, but they are in favor of whatever brand of good design they favor because they believe it improves productivity. (And by "design" here I mean either up-front design or agile's approach, ie planned or evolutionary design.)
Design activities certainly do take up time and effort, but they payoff because they make it easier to evolve the software into the future. You can save short-term time by neglecting design, but this accumulates TechnicalDebt which will slow your productivity later. Putting effort into to the design of your software improves the stamina of your project, allowing you to go faster for longer.
One way of visualizing this is the following pseudo-graph.
The pseudo-graph plots delivered functionality (cumulative) versus time for two imaginary stereotypical projects: one with good design and one with no design. The project that does no design expends no effort on design activities, whether they be up front design or agile techniques. Because there's no effort spent on these activities this project produces function faster initially.
The problem with no-design, is that by not putting effort into the design, the code base deteriorates and becomes harder to modify, which lowers the productivity, which is the gradient of the line. Good design keeps its productivity more constant so at some point (the design payoff line) it overtakes the cumulative functionality of the no-design project and will continue to do better.
I call this a hypothesis because it is a conjecture, there is no objective proof that this phenomenon actually occurs. In scientific terms it's not a very good hypothesis because it's hard to test. We CannotMeasureProductivity nor can we measure design quality.
But despite it being only a hypothesis, it's also an axiom for many people, including myself. We may not have objective proof that this effect occurs but many of us feel that this explains what we see we see qualitatively in the field. It's an axiom for me as it's the assumption that underpins my entire career as a writer about software design. If design doesn't actually improve productivity in some way, most of my writings are worthless.
I'm sure it sounds strange to many people to treat a hypothesis as an axiom, but it's a common thing to do. I look at it that I use my judgment to assess that the hypothesis is true, but can do so without ignoring the objective weakness of the hypothesis. I'd love to find a way to to prove it and almost as much to refute it.
The hypothesis has a corollary, which comes from the the design payoff line. If the functionality for your initial release is below the design payoff line, then it may be worth trading off design quality for speed; but if it's above the line then the trade-off is illusory. When your delivery is above the design payoff line neglecting design always makes you ship later. In technical debt terms it's like taking out a loan but not using the principal for so long that by the time you use it you've paid out more in interest payments.
This raises the question of where that line is. Even with people who accept the design stamina hypothesis there is substantial, and important, differences over where the payoff line sits. I take the view that it's much lower than most people think: usually weeks not months. But again this can only be a judgment call.
This leads to a consequence for Technical Debt. Technical Debt is a fantastic analogy and I use it frequently. But the design payoff line reminds us that taking out a Technical Debt is only worth doing up to a certain point. Not just do we have to consider whether delivered value is greater than the interest payments, we also have to judge whether the delivery is above the payoff line in the first place.
Last week I got the newest book in my signature series: xUnit Test Patterns by Gerard Meszaros. I've been working with Gerard with it on and off for a couple of years, so I'm fairly familiar with its contents, but somehow seeing the physical copy gave me quite a shock. Somehow it hadn't dawned on me how big the book was - 883 pages, easily the biggest book in my series.
On the whole I'm not keen on big books, I was very proud of keeping UML Distilled so small. A book this big scares me, how will I ever get time to read it?
But xUnit Test Patters isn't as scary as it looks, because it's actually two books in one set of covers. In this it follows a style I also used in P of EAA. The first book is a narrative book, designed to be read "cover to cover". The narrative book is something small enough to be digestible, in xUnit Test Patterns it's 181 pages, 106 in P of EAA. The second book is reference material, which is designed not be be read cover to cover (although some people do) but instead to be dipped into when needed. The idea is that you read the narrative book to get a good understanding of the field, then put it on your shelf so it's handy to grab when you need to delve into the reference section.
I read a lot of history books and I've often wished that the author had written a duplex book. History books often need detail to expand upon a particular topic or to describe the evidence for a point of view. The result, however can be a long book. One example is a favorite book of mine: Guns, Germs and Steel. I'm really glad I read it, and I do recommend it. But it did feel long, and I wonder how it would have worked in duplex.
How to Read a Book suggests that when you read a book for the first time you should deliberately skip-read it, unhesitatingly skipping over detail sections. I read very fast, which helps me, but although fast or skip reading is a plus, I'd rather the book was designed to help. My personal rule of thumb is that 200 pages is the limit for a narrative cover-to-cover technical book. If it goes over that you need some way to allow people to get the core information with less bulk. A duplex book isn't the only way to it, such as selected bolded paragraphs in the Timeless Way of Building, which worked pretty well for me. I'm sure there are other techniques that I haven't run into yet, but the duplex is currently top of my list.
A duplex book is really a specific case of a more general principle to organize a book in gradually increasing sections. The Pick Axe is a good example of this. The first two chapters are a quick overview of ruby in 24 pages. Then you have 280 pages of tutorial, followed by 500 odd pages of reference material. The first few chapters of the Enterprise Integration Patterns book is an overview (95 pages) with rest (~550 pages) reference. Books like this often don't make these successive layers of revelation as clear as they might, however.
An interesting question is whether we can do more with the duplex book format. A lot people won't realize that xUnit Test Patterns is a duplex book because it looks like one thick book (the back cover actually claims it's three books by considering the reference section as two separate sections). Could we physically split the books, perhaps by packaging them as two volumes in a slip case? I don't think actually selling the books separately would make sense as they are too closely coupled together.
Naturally we think about online access. Reference material often works better on the web, so perhaps we could make the reference book web only (maybe with a CD or something in the physical book) and sell only the narrative book. What would that structure do to sales?
There's interesting questions here, but the final message is that I'd urge authors of big books to think more about how the book is structured in a way that helps someone who wants something within that 200 page limit.
Updates: I'll be in Europe for the two weeks before JAOO and will be around a few conferency like events.
I'll be giving a couple of talks with my regular co-talker Dan North. Both are part of ThoughtWorks's technology briefing series. Our topic this time is "selling agile to your organization". As I detest selling anything to anyone it will be interesting to see how this talk works out.
I'll also be attending RailsConf Europe. I'm not speaking (thank goodness) but I will be around.
After I return to the US I shall be at the ALT.NET conference. As indicated in a previous post, I'm a keen advocate of a broader based .NET community. Lots of people at this conference are people whose blogs I regularly read, so it will be fun to meet them face to face.
I shall be taking my regular trip out to JAOO again. As regular readers know, this has become my favorite conference and about the only one that I make a regular effort to get to. I'll be doing a tutorial on Test Driven Development with my colleague Erik Doernenberg. We did this one together at Øredev last year and would have done it at QCon London this year had I not gone down with lurgy before my flight out. I'll also be taking part the closing panel.
In October I'll be back at OOPSLA - another long running favorite conference. I'll be on a panel "No Silver Bullet Reloaded", which looks back at twenty years since Fred Brooks's famous paper. The panel line up includes The Man himself, which is rather intimidating, although nowhere near as intimidating as being on the same panel as 'Bedarra' Dave Thomas.
JAOO and InfoQ are collaborating on another QCon conference, this time in San Francisco. For this show I'm intending to do my first shot at a half-day tutorial on the material I've been working on for Domain Specific Languages. I'll be ably assisted by Neal Ford.
One thing I'm hoping won't repeat is a further fare-up of my back problems. I've gone twice thought the cycle of seeing the back problem go and come back with a vengeance (the last time wrecking my vacation). So I'm being cautious about making too many promises.
Update: Recently we've seen some very encourging signs coming out of Redmond. In particular, IronRuby will be hosted on RubyForge with what looks like a very permissive licence - Ola approves.
At RailsConf2007 there was a lot of excitement over JRuby. This small team had taken a moribund project and turned it into what looks like a first class implementation of the Ruby platform on the JVM. They got a lot of cheers and deserved them all.
So with JRuby pretty much here, the spotlight moves onto the other common managed code runtime - .NET. Microsoft's intentions for Ruby are currently much less clear. They have announced Ruby as a language for scripting Silverlight - but that still leaves a lot of open questions. Is this a full implementation of the ruby language, or is some form of Ruby++ - an enhanced subset of Ruby?
JRuby serves two distinct but complementary purposes. On the one hand it's a powerful scripting language for the JVM, allowing you to weave a dynamic language into a Java application. It's second purpose is an implementation of the Ruby platform in the JVM, which allows a Ruby application, in particular a Ruby on Rails application, to run on the JVM just as well as it runs on the MRI (Matz's Ruby Interpreter, the current C runtime).
The big question for Microsoft's "Iron Ruby" is how compatible will it be? Will it be a full implementation on the CLR? All the signals I hear tell me that John Lam, the main force behind Iron Ruby, is determined to get a fully compatible implementation. However this may be very difficult to do as things stand. Soon-to-be-ThoughtWorker Ola Bini, a JRuby committer, reckons that it's almost impossible to figure out how to implement a Ruby runtime without looking at source code of the MRI - but Microsoft imposes drastic limitations on its employees' ability to download open source software, let alone look at the source. The open-source community does much of its communication through source code - so this makes collaboration with an open-source community very difficult.
Overshadowing this, of course, is Microsoft's historically difficult relationship with the open source world. In the past Microsoft has gone out of its way to vilify and threaten the open source community. In recent years things have improved, but there's a real question about Microsoft's core intentions. The recent patent threats are seen by many as proof that Microsoft is still intent on fighting open source to the death.
Unlike most other technology companies, Microsoft has struggled to find a way to co-exist with the open source world. It is harder for Microsoft - unlike Sun, Apple, or IBM they are overwhelmingly a software company. Open source projects like Linux, GNU, and Open Office are directly competing with Microsoft's crown jewels. However I've never felt that declaring war on open-source, trying to stamp it out, was a viable long-term solution. Open-source is here to stay, the question is how to accommodate it.
With Ruby Microsoft is in a different position to the more visible death match. Ruby doesn't compete with the core revenue generators in Microsoft's line up. What's more there is a real desire in the Ruby community to co-operate with Microsoft. Most people I talked to at RailsConf were very keen to see a full range of support for Ruby on Microsoft - and there were a lot of creative ideas floating around on how we could try to come with an approach that could make it work. The overwhelming sense I heard in the community was not "Ruby will kill evil Microsoft" but "how can we overcome the problems to get Ruby on Microsoft."
As Chris Sells pointed out, we do have to consider the question "what's in this for Microsoft". I see a couple of reasons. First off is the role of .NET and Windows in the data center. If Microsoft doesn't support the Ruby platform, it runs the risk of people moving away from .NET (and Windows) on server farms if Ruby on Rails becomes successful.
Another reason is people. Microsoft doesn't like to acknowledge this in public, but there is a real concern that AlphaGeeks are moving away from the Microsoft platform. There's a growing sense that Microsoft's vision is armies of Morts in command-and-control organizations. There often seems to be outright discouragement of tools to enable talented enterprise developers, or of agile development processes.
A few years ago my (limited) contacts in Redmond told me that they were seeing a real drift of technical leaders away from the Windows platform. More recently these signs seem to be increasing. Reading the 'softie part of my blogroll I got a sense of real disillusionment amongst people who have been long-time Microsoft supporters. Agile-oriented developers have been frustrated with the direction of Microsoft tools. Microsoft conferences barely mentioning agile processes, leaning much more to waterfall approaches. The tools, with their rigid role-separations, actively discourage the blurry boundaries that agilists prefer.
At RailsConf, Tim Bray contended that the key decisions on technology are made by the programming community. I partly agree with this. The reason we have so much bloatware in IT is because IT purchasing decisions are usually made on golf courses by people who have lost meaningful contact with the realities of software development. However golf-course decisions may dominate the short-term, but as time rolls on I think Tim's contention is true. So losing the alpha geeks may not matter this year or next, but will inexorably harm Microsoft over time.
Indeed it's already past next year for Microsoft. We've seen a noticeable drop-off in interest from our clients for Microsoft projects, particularly in the US. In Australia, .NET hasn't got any foothold at all amongst our clients. I'm not sure what to make of this data. We aren't so big to be a statistically valid sample on our own. But it's a useful data point nonetheless particularly since we like to think our clients are the "alpha IT shops".
Perhaps more significant is the story within ThoughtWorks. When .NET appeared there was a lot of interest in the platform. Many people were pleased to see a strong competitor to the Java platform and were keen to get involved on .NET projects. Yet over the last year or so there's been a big turn away from .NET. This is despite the fact that there is some really interesting things coming out of Redmond. Mike Two is very keen on the windows workflow tools, I've been very impressed by Linq and other language developments. But the general view of Microsoft technologies is a loud yawn. This is important because, as Tim O'Reilly believes, the alpha geeks point to what everyone else will be doing in a few years time. And the crucial point is that the attitude to Microsoft isn't hatred (a common attitude amongst many geeks) but boredom. This is what Paul Graham means when he says that Microsoft is dead because it's no longer dangerous.
The attitude to open-source is a large part of this problem. When Java appeared there were yawning gaps in its portfolio, and worse some dreadful standard tools in its API (visions of Entity Beans spring to mind). Those gaps and bad ideas were fixed by the open-source community. Ant gave us a build tool, EJB was displaced by Spring and Hibernate. .NET has also got its gaps, and again the open source community has stepped up to fill them. Yet Microsoft refuses to collaborate with these efforts, even seems to go out of its way to undermine them. I was particularly sickened by Microsoft's reaction to NUnit - an excellent XUnit testing tool, elements of whose design were lauded by Anders Hejlsberg at OOPSLA. Microsoft ended not just bringing out a competitive library, but deliberately making it incompatible. That's not the kind of reaction that encourages people to invest their time in the platform.
To be fair, that debacle was a couple of years ago. Actions like hiring Jim Hugunin and John Lam have helped counter that impression. Technologists like Chris Sells, Don Box, and Jim Newkirk are working to make Microsoft are a more open environment. But like any large organization, Microsoft is full of contradictory forces and we don't know which ones will prevail.
My colleague John Kordyback pointed out that at the heart of all this is realizing that Ruby is not Yet Another .NET Language but a whole community and attitude to software development. Ruby is a community where open source, agile thinking, and lightweight solutions are deeply ingrained values. He says a common problem in Redmond is that "They ask me 'Why is this language important' rather than 'Why is this thinking important?'"
So what I see for Ruby and Microsoft is an opportunity. The Ruby community seems eager to work with Microsoft. This provides an opportunity for Redmond to figure out how to deal with the problems of working with open source and for this effort to serve as an exemplar for future collaboration. A first class implementation of the full Ruby platform on .NET would be a wonderful product of this collaboration. Perhaps an even better result would be for this work to serve as an example of how Microsoft can collaborate with a community that's centered on openness and agility; an example that can be a springboard for further spreading of attitudes that can further help programmers and their customers throughout the Microsoft world.
There's been quite a few reactions to this (see Technorati for a full list). Particularly worth reading are the ones from: Sam Gentile, Cory Foy, Luke Melia, Jeremy Miller, Rockford Lhotka, John Lam, Evan Hoff, Karl Seguin, Ola Bini, Miro Adamy, Charles Nutter, Peter Laudati, Nick Malik
When I said HelloCup I was looking at a yacc based parser in a language that didn't require me to handle my dirty pointers. Another alternative to play with is Ruby which now has a yaccish parser built in to the standard library - inevitably called racc.
Racc has an interesting interplay between ruby and grammar syntax. You define the grammar with a racc file which will generate a parser class.
Again I'll do my simple hello world case. The input text is
item camera item laserI'll populate item objects inside a catalog, using the following model classes.
class Item attr_reader :name def initialize name @name = name end end class Catalog extend Forwardable def initialize @items = [] end def_delegators :@items, :size, : 'item.y.rb' do sh 'racc item.y.rb' end task :test => 'item.tab.rb' do require 'rake/runtest' Rake.run_tests 'test.rb' endThe racc command needs to be installed on your system. I did it the easy way on Ubuntu with apt-get. It takes the input file and creates one named inputFileName.tab.rb.
The parser grammar class is a special format, but one that's pretty familiar to yaccish people. For this simple example it looks like this:
#file item.y.rb... class ItemParser token 'item' WORD rule catalog: item | item catalog; item: 'item' WORD {@resultI don't go to as many conferences as I used to, but the advantage of that is that I have time to go to ones that take my fancy. I've long had a particular fondness for the Ruby community, so I turned up as an attendee at this year's RailsConf.
Chad Fowler and Rich Kilmer introduced the conference. Chad shares my name but I don't share his ukulele skills.
With a young technology there's lots of new and important pieces appearing, but for me the most important of these is JRuby. Now in its final cycle of release candidates, JRuby offers a full implementation of the Ruby Platform on the Java JVM, as well as providing a scripting language for the JVM. For what we do at ThoughtWorks, and for many Ruby/Rails developers this matters a great deal even if you never "include java".
One of the biggest issues our Ruby teams run into is deployment. Getting a ruby app into production involves a whole bunch of new technologies, and data centers tend to be conservative on this kind of thing. Our RubyWorks stack tries to simplify this, but JRuby offers the choice of just deploying into a Java container, turning a Rails app into an easily deployed war file. I think this will make Ruby on Rails a much more viable choice in lots of enterprise environments.
JRuby's maturity on the JDK raised the obvious question of what will happen with the other popular managed runtime. Signals from Microsoft were faint and unclear. It was good to see notable Microsoft-oriented geeks Scott Hanselman and Chris Sells at the show, even if when I saw Chris Sells it felt like being in an arena. What I like is that there is a real desire within the ruby community to collaborate with Microsoft rather than the usual competition and Redmond-bashing.
Indeed the overwhelming impression I got at RailsConf was a sense of industry-changing mission. At last year's conference my sense was that of a community delighted and rather stunned by its success. My iconic image was DHH as Neo of the Matrix, breaking the rules of enterprise software and reveling in the success.
This year my sense is that this morphed into something much more important. Success is not just being an outbreak of sanity in the IT world, but actually leading that world. There seems an excellent chance that Ruby and Rails could become a significant platform for IT develop over the next few years. We're already seeing signs of this at ThoughtWorks - 40% of our new business this year in the US is Ruby work.
Unlike previous platforms, this one isn't controlled or even dominated by vendors. Ruby is a community effort. I've been hoping for a long time that one of the LAMP technologies would break through into corporate IT, and it looks like Ruby could well do it.
Corporate IT is dominated by bloatware. Time and time again we have to deal with expensive software purchased on golf-courses that just get in our way while sucking funds and development hours. Technologies that make it harder for programmers to do what they like to do best - make a difference for the businesses they are supporting. (This week's memorable tale was of a large company that spent eight million dollars on an enterprise-wide version control system that couldn't branch properly.)
My hope that with a platform that's controlled by a community we'll see a platform that focuses on simplicity - finding the key things that need to be done and doing them in a way enables programmers to provide their best.
Michael Koziarski and Jamis Buck describing the Rails Way - establishing the good style which lies deep in the Ruby community.
It's helped by the fact that the ruby community has formed around the best ideas of the OO and Extreme Programming communities. Listening to the keynote of Jamis Buck and Michael Koziarski I was delighted to reflect on the thought that they were right there in the values of Ward, Kent, and all the other people who've been advocating clean code, well-factored object-oriented design, and testability. These ideas have made a great impact on many other technological communities, but in Ruby-land they are the orthodoxy.
Throughout the conference there was a sense that we are at an inflection-point in our industry, a key technological shift that promotes a new major platform. Listening and reading about JavaOne I got the sense that a big shift had happened there, people no longer focusing on Java the language but on Java the JVM, replacing one language with multiple languages closely collaborating.
A particularly fascinating thing about the Ruby community is the diversity of ages of people in it. Not just do you have the young paradigm breakers like DHH and the core team, you also have the, ahem, rather more seasoned campaigners like PragDave and the RubyCentral triumvirate. The important thing is that there's a lot of respect and collaboration across this generation gap. There isn't the wall between the old farts and the loud teenagers that you often get, instead a real appreciation for what both groups have to offer.
Not that all is good, however. There is a marked lack of women in the ruby community which, apart from the fact that we're missing out on some serious talent, probably is a sign of other problems within the community. The DevChix group was pretty active in looking for ways to try to correct this and I was pleased to see several conversations sprout up during the conference on finding ways to improve our Diversity problems.
A couple of years ago I wrote about the impressions people had that the RubyPeople were notably more friendly than most software communities. Talking to people at RailsConf I got the message that that had changed for the worse. Both the ruby lists and particularly the rails lists had tended more towards the sad Internet mean. In his opening keynote Chad Fowler said that the rails community had a reputation as "a bunch of arrogant bastards" and I cringed as an, admittedly small, group raised a triumphant cheer.
What pleased me was, that starting with Chad, there was a strong push from the ruby leadership to try and change this. A theme that came from several talks was that our community had an opportunity to try to lead the software profession away from this tar-pit of NetNastiness and lack of diversity, to create a community that really welcomed many types of people into a nurturing and pleasant environment. I've been sick of the tone of geek discussion for many years and if the Ruby community could really show a direction to lift us out of this quagmire, this would delight me even more than all of the other prizes that glitter in front of us.
(Photos courtesy James Duncan Davidson.)