The Scotty Effect

raph @ 2011-10-27 10:33

This post was migrated from my old blog which used to be hosted on Blogger. As a result, some links might be broken.

There’s a common problem among software engineers. Whenever someone asks us for an estimation on how long task X is going to take, it happens. “Ah, not a big deal, half a day, max.” Is what we initially want to say. It happens to most people I know, and it happens to me.

It seems like there’s a little geek inside us that wants to impress our customers by making it look like we’re wizards. Somebody is asking you to do something, and obviously it’s a problem complex enough that an engineer has to do it, now.

And it’s all because of Montgomery Scott, aka “Scotty” from the Original Star Trek ensemble. Whenever Kirk asks Scotty to fix something, it goes somewhere along these lines:

Kirk: “Scotty, can you get the warp drive online in 2 hours?”

Scotty: “Lol captain, that’s impossible. Do you have any idea how complex and fragile a matter/anti-matter propulsion engine is?? We’ll have to…”

Kirk: “Scotty, it’s crucial that we prevent the universe from imploding so I can get into this alien chick’s pants.”

Scotty: “Meh, I’ll see what I can do…”

No surprises so far - just a normal day in the life of an engineer having to deal with unrealistic expectations, right? Wrong! In the end, Scotty pulls it off! He not only fixes it, but apparently the warp engine hasn’t went supernova in the next episode, either. So he probably did a good job, too!

We grew up with this idea, that if you really want your boss to get into that alien chick’s pants, you need to be a wizard. You get the point. That’s why I believe it’s Scotty’s fault. Don’t get me wrong, Scotty rocks (rest in peace, Mr. Doohan). But that inner geek inside us, it’s probably a little Scotty.

duct tape and wd40 can fix anything

I couldn't find DuctTape or WD40 on GitHub

Back to reality

Spoiler alert! Star Trek is just a TV show. In the real world, it’s not going to happen. And this is why.

It’s not the same thing

If you do a 10 day job in just 1 day, then you’re not doing the 10 day job. Claiming otherwise is lying to yourself and your customer. You’re really doing the ugly 1 day version, involving lots of duct tape and WD-40. We like to call ourselves “hackers” - just like the good, old-school hacker. But people forget that a “hack” back in the good old days was nothing bad or malicious. It was something neat, elegant. It was “the right thing to do”. Read Steven Levy’s Hackers. If you’re hotshot enough to pull off the 10 day job in 1 day, fair enough. It’s most likely though, even though you might be awesome, that you are just taking shortcuts.

Sooner or later, it will come back to bite you

This can work in emergency scenarios, and over short periods of time. But by doing so, you’ll build up more and more technical debt. We all know what happens if you run a never-ending tab without bothering to repay your debt. One day, your warp engine will not work in the next episode. You might eventually have to eject it.

Also, be aware that making it look like you can make the impossible happen will raise your customers’ expectations - putting you under pressure to uphold the illusion.

Life’s not a TV show

Scotty is usually pessimistic at first, but then finds some magical way to do it within the impossible time frame. In reality, things are the other way around. They seem simple, but when you take a closer look, they turn out to be more complicated. Knowing this makes the difference between guessing and estimating.

What to do about it

You’re better off to resist that initial urge to fix the warp drive/deflector shield/ in “2 days” (or whatever ambitious answer your gut gives you). Take one of the many great pieces of advice from The Pragmatic Programmer, and just say “I’ll get back to you”. Do some realistic estimating (not guessing!), and then tell them how long it will take.

And guess what, you’ll get better with your estimations. You’ll end up doing things the right way, and within realistic time.

And every once in a while, they’ll still think what you do is magic.

UPDATE:

My fellow trekkers have not dissapointed and spotted what can appear to be a contradiction to my descriptions of Scotty (which are based on the “old” series): in the great espisode 130 of “Star Trek: The Next Generation”, “Relics”, Scotty returns to the latest USS Enterprise D. He then hints at his successor, Chief Engineer Geordi LaForge, that Geordi should always exaggerate his estimations to make himself look like a miracle worker (quote). Since I’m claiming that Scotty always managed to do miracles in an impossible time frame, it looks like Scotty did not only deceive Kirk, but me as well. Or did he? While thinking about a response to this, I find that I am torn. I have two separate responses, each from a different perspective. Yet none of them affect the point of this post.

I, Hacker

Senior engineers often tweak their previous achievements to look good in front of the n00bs by giving advice that they not necessarily have never adhered to themselves. But who’s going to prove the opposite? In this episode, being in the 24th century with all the shiny strange technology, Scott feels outdated and useless. This could just be a senior engineer trying to say something smart to impress his younger, more advanced successor.

I, Trekker

This espisode was produced 23 years after The Original Series, approx. 1 year after Gene Rodenberry’s passing. Star Trek is full of inconsistencies. It’s hard to keep continuity perfect over so many episodes and movies, and it’s Sci-Fi after all. I seriously believe this was something the authors wanted Scotty to say in order to be funny, nothing more and nothing less. It is funny because it’s a direct contradiction to what we all know from previous events. This was a great espisode, and this scene in particular was a memorable one - but I refuse to acknowledge this as evidence that Scotty wasn’t simply an extraordinary engineer - a wizard.


This post received some attention over at Hacker News - check out the discussions.