Schrade.Blog

The hardest thing to see is what is in front of your eyes. -Goethe

It's been about 6 or 7 months since I've done any Rails work.

I'm in a post-Rails world.

After working in Rails for 3 years, I was burnt out. I took some time off to try to learn the finer points of marketing and running a business, and then attempted to start a new business just before the market cratered. Not a great idea.

Now I'm back writing code all day and it's great. My days now consist of coming up with test cases to solve problems and then solving them.

Something is different though.

I used to write a ton of test cases in the context of my Rails apps.

Now I write high level test cases and then write code and pick and choose technologies to solve them.

I feel like this is a natural progression for those of us that practice TDD.

Take one more step back, use something at an even higher level to write tests (I'm using some Cucumber and some rSpec), and then implement a bunch of little apps to make what you want to happen, happen.

It keeps things simple, and keeps the problem from growing too large.

In my current project I've got a cluster of Nanite agents up and running with a couple of Merb apps talking to them on the front end. It's something that I never could have done if I was trapped inside of the Rails box.

Hell, it's probably something that I never even would have thought of.

I think that the Ruby world is eventually going to end up in a model like this, writing small simple apps that all talk to each other, and can be replaced or upgraded at any time.

By writing a lot of tiny apps, I've been able to solve problems now in days that used to take weeks or months.

Plus I'm not getting locked in to anything. If I want to replace my Merb app with a Sinatra app next month, it's not a huge project. If I want to swap in some code running on the Maglev Alpha that I'm playing with or deploy some components written in Clojure, I can do that too, without being intimately tied to anything.

All of my hard/long running logic is well tested, encapsulated, and most likely running in little agents on the wire.

I guess what I'm saying is that if you've been writing Rails code for a long time, perhaps now is a good time to step outside of the rules that Rails imposes upon you and look at things from a different angle.

You might find it refreshing and helpful.

I certainly did.

As some of you know, we're building our new startup on top of Merb. One of my favorite things about Merb is how much easier it is to understand than Rails is when you need to hack on something that's outside the realm of what the framework expects you to use it for. I think that Ezra Zygmuntowicz put it best in his recent tech talk on Merb:

A lot of people tend to end up treating Rails as a blackbox because it is eighty thousand or so lines of code and it's a little bit hard to step into and figure out what's going on because there's so much advanced Ruby meta-programming going on inside of there. I kind of feel like the place for meta-programming and that kind of stuff is in your own application code. I'd rather have the foundation that you're building on top of be as simple and as easy to understand as possible so that when you do hit the wall and you need to look inside the framework you don't get grossed out and you can get in there and actually do stuff with it.

Amen.

My friend Jay wrote a blog post today that says that you can use 'svn patch' as poor man's replacement for git-stash. Fair enough, I've heard of places that use patch as full-on replacement for source control and integration as well. :-)

First of all, I want to point out that we still use Subversion as our main code ghetto at work.

Git fully supports updating from svn and pushing changes back, so even if you have to use Subversion at work, you don't have to use it on your local machine.

The difference is that I have our entire history of source code commits on my machine in not even twice the space than it takes for my Subversion checkout (969M vs 578M). I also can do fast local branching and checkins, without worrying about being connected to the network.

Here's a real example of how we used git today at work that I never would have done with Subversion.

I started working on a feature yesterday on my laptop that is going to take a few days to finish. When I got to work today I decided to pair with my coworker Krishna to continue working on it.

In the pre-git world we would have had to either huddle around my laptop, check in the broken code to Subversion to check it out on our other machine, or do a patch of just the changes related to that feature and transfer it to and apply it on the pairing machine.

Not today though. Today I just fired up git-daemon on my machine and did a 'git pull' of the feature branch onto the pairing machine. At the end of the day I just did a 'git push' back to my machine to get all of the changes we made today back so that I could continue working on it tonight.

Even better than that though, I can not continue working on it tonight.

Git makes branching and merging so fast and easy that I can rewind all of the changes that I've made to the code base to support that feature, fix a showstopper bug, check that change into the subversion repo, and then merge the feature branch back on top without even worrying about it.

Git is starting to change the way that I do development. I can finally use branches the way that I've always wanted to. I can finally have several features in development at once on different branches, because it's so easy to switch and merge between them.

Even if you're stuck with Subversion as your backbone, you can still grow outside of it.

I converted my Subversion repository at work to a git repository recently, and today I had my first real "aha" moment.

I was working on a feature this morning when I realized that some code I checked in about an hour before had broken the build. In the old Subversion world, I would have needed to keep track of what I was changing to fix the bug, and then make sure to just check that in to ensure that none of my half-finished new feature got into the trunk.

Not anymore. Today I just did a:

git-stash

and it stashed away all of my changes in a temporary branch. Then I fixed the build, checked in my fix out main Subversion repo (using git-svn), and did a:

git-stash apply

to unroll my earlier changes back on top of the now fixed code. It's not the world's biggest saving of time, but it's one less thing that I needed to think about during development, and using git I'm seeing more and more little things that are starting to add up to a big change in the way I'm doing development.

My friend Jay wrote a few days ago:

I bet of you looked at the majority of Rails applications you would find empty test folders (or only the generated tests, which are never run). I'm quite sure that's true because I expect the conferences to attract the best of the Ruby developers, and several of the people I talk to at those conferences "simply don't have time" to write tests.

These people are idiots. Either that, or they don't have any sort of complexity in their apps and they're writing them in a far too simplistic and elementary way.

Quite often, doing a little meta-programming magic in Ruby can make you massively more productive and, in my experience, the only way to find the side effects of doing a lot of meta-programming is to have a comprehensive test suite. As this magic-level in your app increases, the number of side-effects of that magic will also inevitably increase.

Eventually, you will get burned by something if you're not ready for it.

Tests are your flame-proof suit in this case. They're not going to protect you against everything, but most of the time they do a damn good job of keeping you alive.

DHH wrote the other day:

We try to do a fairly good job at keeping our test suites current and exhaustive as well. Basecamp has a 1:1.2 ratio of test code (thanks to the persistence of Jamis!), Highrise has a ratio of 1:0.8 (bad me!).

At CDD, our current code to test ratio is 1:4.2.

Overtested? Undoubtedly.

However, we don't want to ever have to go in and fix the damage after someone tries to import 1,000,000 molecules to our system and something goes wrong. The repair work can take days in some cases, so we err on the side of caution.

I'll wrap this up with the following three easy points:

1. If you're writing a Rails app, write tests for it.
2. If someone who works for you "simply doesn't have the time" to write tests, fire them.
3. If a consultant tries to give a Rails app without a test suite, fire them (and don't pay them, if at all possible, what they've given you is pretty much worthless, and is going to be a nightmare to maintain).

Selenium is a great tool. We have a bunch of Selenium tests that run with every one of our builds. Unfortunately, they contain a bunch of XPath in them to verify elements, and the XPath engine in IE6, to put things lightly, is a piece of garbage. It takes a huge amount of time to verify an element in a page.

As an example, here is the amount of time that our Selenium test run used to take on Firefox:

Finished in 506.108548 seconds

And here's how long it took on IE:

Finished in 4567.47592 seconds

That's 1 hour and 16 minutes to the same set of tests that it takes less than 9 minutes to run in Firefox. Clearly this is unacceptable, and, from watching the tests run, it's clear that the IE XPath engine is to blame.

We decided to attack this problem by using Hpricot to verify the elements in a page instead of the built in XPath engine. This allows us to use Selenium for navigation between pages in our site and Hpricot to do the verification.

It's simple enough to grab the HTML from the page at any point during a Selenium test by using

@browser.get_html_source

(as long as you're using Ruby Selenium Driver) and then you can do all of the verification that you want on it, in far less time. Plus, the number of changes that you have to make to your Specs/Tests are minimal. As an example,

readouts_table = table('id=readouts') readouts_table.cell(0,0).should contain_text("Molecule")

in our old code became

doc = Hpricot(@browser.get_html_source) readouts_table = doc/"//table[@id='readouts']" readouts_table/"thead/tr[1]".should == "Molecule"

in our updated code.

As always though, the proof is in the pudding. I've updated 2 out of 19 of our specs, and our time on Firefox has dropped to:

Finished in 488.753052 seconds

But more importantly, out IE run time is now:

Finished in 3171.07507 seconds

That's a 24 minute difference already and we can already fit in a couple of extra CI builds a day because of that.

We're planning on adopting this strategy for all of our Selenium tests moving forward. Selenium will be used only for navigation, and Hpricot will be used for verifying the page content. I strongly suggest that you consider it instead of using the built in verification.

My friend Jay published a blog post today entitled Despised Software Luminaries and, for once, he got it all wrong. I know Martin and Obie , his examples in the post, from my time at Thoughtworks, and they're not software luminaries. They're software pundits.

Both of them spend a lot of time telling people how they think the software world should operate. Whenever you tell people how they should do something, it naturally leads to disagreement. It's that simple. People despise them because they disagree with them.

When I think software luminary, I think of people like Alan Kay, Fred Brooks, and Edsger Dijkstra. People that made fundamental changes to the way that we write software. People like Obie and Martin are smart, talkative guys, but they're not quite in the same league as those guys.

(See also: http://en.wikipedia.org/wiki/Turing_Award)

Seaside, my other favorite framework, released version 2.8 today. This release has been optimized and optimized some more, as it doubles the average page rendering speed, and now uses up to 4 times less memory. Plus it supports Gemstone, which is the feature that I'm most excited about playing with.

The official release page is here, and if you still haven't tried Squeak, a great new tutorial can be found here.

Congrats to the entire Squeak team on this one.

(Seriously though, who does a release on a Sunday? You guys are nuts.)

Yesterday's blog post certainly generated a boat load of feedback, which I'm going to group into a few different tongue-in-cheek categories:

• Refactoring isn't a problem, just refactor everything yourself by hand, or write a refactoring tool yourself.

If I had the time, I would. Refactoring by hand is what I'm doing now on a daily basis, because I don't really have any other choice. Thus the request for better tools.

Which brings up a tangentially related point. Why is it that whenever someone asks for better tools it's because they're lazy? I think that we can all agree that it sucks to use an axe to chop down trees when you've used a chainsaw before. From now on, comments of the form "just use grep to do it" will only be accepted from those of you who program on mainframes and walk everywhere, simply because you remember the good old days before we had these fancy cars and trains to move us around.

• You must be coding Ruby incorrectly.

When making this comment, please point out the web page that shows how to write Ruby correctly. I have multiple Ruby webapps in production, I worked at Thoughtworks and I'm currently working with Pivotal Labs, two of the foremost Ruby consultancies, and no one has told me or my team that we're writing things incorrectly. However, I have an open mind and I'm willing to learn more about this topic.

• You can't just throw together Ruby code without tests, that's why you're running into problems.

Aren't you going to have this problem with any language? Where did I imply that this was something that we were doing? Anyway, 'rake stats' says:

Code to Test Ratio: 1:2.8

So I don't think that's the problem.

Seriously though, It's all a bit academic. If you asked me what framework I would use if I was going to start building a new webapp tomorrow (and told me that I couldn't use Seaside) I would still choose Rails. It's still the best tool for lots of jobs out there. Pointing out some of it's deficiencies doesn't make that any less true, despite what some people seem to think.

Seaside does not have a marketing problem!

Everyone I know in this business knows about Seaside and its super productive powers. We've all seen the demos and the screencasts and the awesomeness that is DabbleDB. For god's sake, Avi gave a keynote at RailsConf this year.

I think it's safe to say that people know about Seaside.

The problem at this point isn't a marketing one. It's the same problem that Squeak has always had. Namely that when you download Squeak and start it up, you're presented with this:

When most programmers I know see this, they have the same reaction, "What the fuck?"

This is usually followed quickly by the question of, "Can I edit this using Vi or Emacs?"

Let me answer this now. No and No.

If you're the type of person that can't imagine giving up writing code at the command line, then Squeak isn't for you. (Although there are Vi and Emacs keybindings.)

And that's usually the end of it. People play with Squeak for a few minutes and then go back to doing their real work in Ruby or Python or Java or C#. A lot of the power and magic of using Seaside is directly derived from the power and magic of using Smalltalk, which unfortunately (or fortunately, depending on your point of view) means that you're going to have to learn Smalltalk if you want to use it .

And Smalltalk is, at least in the case of Squeak, a whole different world. You have to unlearn a lot of what you know in order to use it. Editing is different, class creation is different, version control is different. Basically everything you know as a programmer gets thrown out the window.

But you know what? At some point, once you do that and really start to immerse yourself in Smalltalk, you'll suddenly realize that it's hard to go back to the old way of doing things.

You'll realize how much easier it is to just ask the system what method you should use to get something done instead of going to a PDF document somewhere to look it up. (Yes, in Squeak you can just type "'class'. 3. 'cla'" into the the method finder and tells you that "contractTo" and "truncateTo" are the two messages in the system that give you that result.)

You'll suddenly come to the realization that this is simply better.

That this is how things are supposed to work.

Unfortunately for most of us at this point, we'll have to go back to our real jobs, (I asked Avi after his keynote how many people in the world are actually making money using Seaside and he said that he figures about 20) but even if you get this far you'll learn enough to to have a whole new outlook on how coding should work.

It's not a marketing problem, it's the the same old problem that people are scared of things that are different then what they're used to. Hopefully someday we will be able to overcome this fear, and then Seaside and Smalltalk will finally take over the world. But for now this new outlook on development will have to be good enough.