Entries Tagged 'Tech' ↓

I’ve been working at Intent Media for a while now and I’m just getting some time to come up for air to talk about what we’ve been doing while building our software platform.

This will be the first in a series of articles about what I’ve learned over the last year.

For a little background, our engineering team is currently 9 people strong (in a 20 person company). When I joined the company to help build the team, there wasn’t a line of code written and I was the only technical member of the staff.

Over the last 9 months we’ve built what I consider one of the finest pieces of software that I’ve ever worked on. It’s scalable, it uses the right languages for the right jobs, and it does exactly what it needs to do.

It’s also remarkably bug free.

I credit a large part of that (on the tools and good decisions side, that is; I credit most of it to our team) to Cucumber.

One of the first decisions we made (after writing a unit test and our first bit of code) was to write our first Cucumber test. One of the things that I learned while working at Collaborative Drug Discovery was the importance of automated functional tests.

I, for one, will never do another project without them.

At Intent Media we use Cucumber to drive Celerity across our entire software stack. Currently, this consists of 3 Java applications and a Rails application that interact with MySQL and a HDFS data store. Every single time we check in new code we build the entire stack in TeamCity and run our entire suite of functional tests against it.

As of today, I’m currently greeted with this on our TeamCity server:

Each of those “Tests passed” is a step in a Cucumber scenario. For example:

Given I login as “admin@intentmedia.com”

Over the last 9 months we’ve slowly built up a library of steps that encapsulate everything that we need to do with our apps so far. We’ve also found that we’ve gotten a huge amount of reuse out of our existing steps as we’ve grown the app. So far, there haven’t been any scenarios that we couldn’t tackle in an automated manner.

This suite of tests gives us the best safety net I’ve ever had on a project to try large refactorings. In fact, we often do refactorings across the data model that affect several pieces of software in the stack, often in unexpected ways. Cucumber (along with our unit tests) lets us immediately surface any problems in both our applications themselves and the communication between the apps.

If you haven’t checked out Cucumber yet, you owe it to yourself to do so. It’s one of those life-changing pieces of software that’s amazingly simple to get started with, but quickly becomes one of the most important tools in your toolbox.

Quick update on the last 6 months:

• Moved back across the country to NYC
• Took a job as Head of Engineering at Intent Media (We’re Hiring! We’re funded!)
• Been working in Java and Rails

I’ve got plenty of things to write about, so I’ll try to get back on the horse soon.

The hardest thing to see is what is in front of your eyes. -Goethe

It’s been about 6 or 7 months since I’ve done any Rails work.

I’m in a post-Rails world.

After working in Rails for 3 years, I was burnt out. I took some time off to try to learn the finer points of marketing and running a business, and then attempted to start a new business just before the market cratered. Not a great idea.

Now I’m back writing code all day and it’s great. My days now consist of coming up with test cases to solve problems and then solving them.

Something is different though.

I used to write a ton of test cases in the context of my Rails apps.

Now I write high level test cases and then write code and pick and choose technologies to solve them.

I feel like this is a natural progression for those of us that practice TDD.

Take one more step back, use something at an even higher level to write tests (I’m using some Cucumber and some rSpec), and then implement a bunch of little apps to make what you want to happen, happen.

It keeps things simple, and keeps the problem from growing too large.

In my current project I’ve got a cluster of Nanite agents up and running with a couple of Merb apps talking to them on the front end. It’s something that I never could have done if I was trapped inside of the Rails box.

Hell, it’s probably something that I never even would have thought of.

I think that the Ruby world is eventually going to end up in a model like this, writing small simple apps that all talk to each other, and can be replaced or upgraded at any time.

By writing a lot of tiny apps, I’ve been able to solve problems now in days that used to take weeks or months.

Plus I’m not getting locked in to anything. If I want to replace my Merb app with a Sinatra app next month, it’s not a huge project. If I want to swap in some code running on the Maglev Alpha that I’m playing with or deploy some components written in Clojure, I can do that too, without being intimately tied to anything.

All of my hard/long running logic is well tested, encapsulated, and most likely running in little agents on the wire.

I guess what I’m saying is that if you’ve been writing Rails code for a long time, perhaps now is a good time to step outside of the rules that Rails imposes upon you and look at things from a different angle.

You might find it refreshing and helpful.

I certainly did.

If you haven’t already heard, Rails is now Merb, and Merb is now Rails.

I’m a bit torn about how to feel about this, having dedicated a lot of time lately telling people to use Merb and why I think that it’s superior to Rails.

On the one hand, it’ll be good for Merb usage, as people using Rails 3 in big projects will now essentially be using what Merb 2 would have been.

On the other hand, I’m concerned with the technical hurdles that the core teams are going to have to deal with in order to make this actually work. There’s a lot of old crufty code in Rails that needs to go, although I’m not sure what removing it would do as far as backwards compatibility is concerned.

I’ve been talking in IRC all morning about this, and I’m sure I’ll have more thoughts as things move forward, but for now I’ll just post some of my favorite quotes from today’s IRC session here:

  <devjj> Modular meaning, what, not included by default, gem install rails-bad-ideas
  <gruntruk> this is like biggy n' pac hugging each other
  <bitbckt> so, I presume the quality of conversation in #merb will disintegrate into #rubyonrails :-/

Certainly should be an exciting couple of months as this merge happens…

(Credit goes to an IM from my friend Will for the title of this post.)

Just a quick note to let people know that the Merb community is finally starting to address the lack of good documentation on Merb. I’m personally helping out with the open source Merb Book.

A preview version can be found here: Open Source Merb Book

The source can be found here: GitHub

We’ve got 20 plus people working on it, and considering that it’s only been up on GitHub for 12 days it’s pretty amazing what we’ve already accomplished.

If you want to learn more about Merb you should definitely check it out.

A long time ago, I did a more than a little bit of open source work. Unfortunately, I always found the process of being a part of an open source project to be a pain.

The process usually went something like this:

• Fix a bug in the code.
• Post patches to the dev list.
• Bug people to get your patches accepted and committed.
• By the time they are accepted, someone else has committed something that breaks them.
• Wash, rinse, repeat

Once you do this enough times, you get commit rights and you don’t have to deal with the process anymore.

I’ve never enjoyed it. Far too much overhead and bullshit.¹

So for the last (more than a) couple of years I’ve taken to forking code when I needed to and then never bothering to give the patches that I wrote back to the maintainers.

Clearly not the best situation, but it worked for me.

GitHub changes all of that.

On GitHub, I can fork anyone’s code, patch it, push my changes back, send a pull request, and then not worry about it.

I can share my code with no overhead.

Example: I fixed a bug in the merb cucumber bindings last week. I pushed my changes to GitHub and sent a pull request to the maintainer. My changes haven’t been pulled into the main tree yet, but I don’t really care.

One look at the GitHub merb cucumber network graph should show any prospective users of the library that development has happened on my tree beyond that of the original tree. Hopefully they’ll take the time to look at it and use the most up-to-date code.

Either way, I don’t have to worry about it anymore.

Also, I love that GitHub lets me share my half finished projects.

I recently posted a partially done Merb based blog called Gutenberg on GitHub.

Will I ever finish it? Who knows.

At the very least it’s a simple Merb example project for people to learn from, and I already know people who have looked at it to learn more about how Merb works. That already makes it worthwhile to me.

Again, no overhead.

Just push code onto GitHub and it’s live to the world.

No pain. No overhead. Just code.

1. As an aside, I really like commit policies like one that Rubinius has, namely, “anyone who submits a patch for some part of the Rubinius project and has the patch accepted will be given commit rights to the mainline repository.” We need more projects with commit policies like that.

As some of you know, we’re building our new startup on top of Merb. One of my favorite things about Merb is how much easier it is to understand than Rails is when you need to hack on something that’s outside the realm of what the framework expects you to use it for. I think that Ezra Zygmuntowicz put it best in his recent tech talk on Merb:

A lot of people tend to end up treating Rails as a blackbox because it is eighty thousand or so lines of code and it’s a little bit hard to step into and figure out what’s going on because there’s so much advanced Ruby meta-programming going on inside of there. I kind of feel like the place for meta-programming and that kind of stuff is in your own application code. I’d rather have the foundation that you’re building on top of be as simple and as easy to understand as possible so that when you do hit the wall and you need to look inside the framework you don’t get grossed out and you can get in there and actually do stuff with it.

Amen.

The easiest way to run Merb from edge:

$ sudo gem install sake
$ sake -i http://github.com/wycats/merb-more/tree/master%2Ftools%2Fmerb-dev.rake?raw=true
$ sake merb:clone
$ sake merb:install:all

This will install the edge versions of all of the merb gems.

To keep things up to date, navigate to the directory that you ran the commands in and do:

$ sake merb:update
$ sake merb:install:all

Then no software would get written, but at least we wouldn’t have any crap software…

My friend Jay posted two blog entries yesterday on the subject of raising the standard for programmers in our industry. I left a comment about it on his blog and talked with him about it on IM, but I figured I’d expand a bit on my thoughts on it here.

First things first, I completely disagree with him.

I don’t think that 50% of programmers should leave the profession. I’ve worked on a lot of projects with a lot of different teams and I can’t ever remember thinking “man, this project would be a lot easier if I got rid of half of this team.”

I can also only think of few Net Negative Producing Programmers that I’ve worked with and most of them usually leave pretty quickly.

Most of the time, I’ve found that when a person isn’t performing up to par, it’s because of a management failure. I think that it’s up to the development leads to analyze their teams and figure out how everyone can best contribute and make sure that people are performing at the top of their game.

Too often I’ve seen people fall into the trap of always being assigned to, or always choosing to work on, the easiest features in the project. They then do a half-assed job because they’re always working on the boring part of the project and don’t feel like a valued part of the team.

Maybe, for a change, we should assign those people the hardest part of the project for a few weeks. Sure, they might not so the best job, but they will work harder because, for once, what they’re working on is challenging and exciting. Plus they know that the entire team is depending on them and is there to help them. That’s how you bring people back into the fold and make them a productive, happy member of the team.

The point that’s neglected in Jay’s arguments is that experienced programmers shouldn’t just be responsible for writing code. It should also be the job of experienced programmers to take those who are less experienced or interested and pull them up to our level.

Forcing them out of the profession is just as much of a failure on our part as it is on theirs.

Things sure have been quiet around here…

As some of you may already know, I’ve left the relative safety of my job at Collaborative Drug Discovery to have another go at starting a company.

I’m currently working with a friend to once again create something out of nothing and bring yet another crazy idea into the world. Therefore, if you see me and I look like I just got finished working for 18 hours straight, please buy me a beer and put me to bed.

As for the new company, I’m not ready to talk about what we’re doing yet, but I’m sure that I’ll be writing about it here as it comes together.