The Magical Build Machine

Evidently Jerry Dennany is a member of the build machine cult:


This is a companion discussion topic for the original blog entry at: http://www.codinghorror.com/blog/2004/12/the-magical-build-machine.html

“MS office automation, anyone?”

cheese.

There’s just not enough chicken sacrificing in software dev these days. Why, I remember when…

Seriously tho, nice post!

One of the points of a “magic build machine” you missed is reproducibility. Using a build machine, especially an automated one, allows very easy reproducibility: you know that all the code used was checked in, you know when it was checked out (at the start of the build), and you can reproduce it locally. You can’t do that from a developer’s desktop machine… no matter how many chickens you sacrifice.

It seems like some important points are being missed here.

I was brought up on the build machine methodology 10 years ago. The only reason anyone cared: something called “escrow”.

We were a small underfunded “startup” type company with a single fortune 500 customer. That customer wanted to be able to do something (it’s not necessarily important what) with our software if our company died. That’s what escrow is for. After every release, we were required to basically package up our build machine and send it to a third party company for them to hold onto. If our little company went out of business, the third party escrow company would deliver the “build machine package” to the fortune 500 company.

This basically creates most of the rules Jerry Dennany is talking about. You must know how to package up the build machine.

I think escrow is the majority use case here. I think there are other user scenarios where packing up a build machine are useful, but I would hardly call it a golden rule.

I always strive to have a packagable build machine, because I have personally experienced many of the downsides of not having one. I don’t make it a requirement or complain too loudly if it doesn’t happen though. It’s just one of many (probably more important) risks that every software project bears.

I think both Jerry and Jeff have extreme (and opposite) viewpoints on this issue that are too far outside the mainstream. Frankly, given my limited experience with the joys of VB6, I’m not even sure I understand most of Jeff’s post/point, which is unusual.

xcopy your files, hah! That’s fine, unless you are using Serviced Components (not on XP, where you can create them without Enterprise Services) or any sort of Interop.

Even in those cases though, we’d get it down to a simple msi installer file. I’ve had to use classes that Microsoft didn’t even document to get a serviced component installed correctly through an MSI file. I guess I should write up a post on that.

By the way, when I’m typing in this here textbox for comments, and then press tab to get to either the preview or post button, I tab up to the Coding Horror link at the top. That’s like a usability issue, mmmmkay? :slight_smile:

One of the points of a “magic build machine” you missed is reproducibility. Using a build machine, especially an automated one, allows very easy reproducibility

I’m all for a build process and a build script. Just not a fan of the build machine for the reasons listed.

That’s what escrow is for. After every release, we were required to basically package up our build machine and send it to a third party company for them to hold onto.

Isn’t a windows installer version of the app, plus the source tree, good enough for escrow? Say on a ZIP disk?

I think both Jerry and Jeff have extreme (and opposite) viewpoints on this issue that are too far outside the mainstream.

Actually, aside from his “you MUST have a magical build machine” focus, we actually agree on a lot of points. The build process should be well understood and documented, and probably automated. I’ve seen a build machine lead to lazy developers who don’t understand deployment like they should. Every piece of code should be written by a developer who understands the deployment implications of that code… and that implies all developers know how to get the app to a user’s desktop.

xcopy your files, hah! That’s fine, unless you are using Serviced Components (not on XP, where you can create them without Enterprise Services) or any sort of Interop.

True, true… but that gets back to my “correctly architected .NET project” argument. In some cases these situations are necessary and unavoidable. In other cases, weird interop-y decisions (MS office automation, anyone?) are made by developers who are not thinking about the deployment headache they might be creating.

A virtual build machine is a definite improvement over a physical build machine. I still would prefer not to have one unless it’s absolutely necessary.

I was just thinking that a decent midpoint would be Virtual PC images and differencing disk images.

I agree that each developer should be able to do the build on his machine, etc., and I also agree that the build machine should be easy to recreate.

So, do your builds inside of a Virtual PC image. Create your image, set up differencing disks, and then archive the image. Once the build is done, reset the image. You can even include your automated tests in another image should the need be.

If a developer needs to reproduce an issue on the build image, he doesn’t have to commandeer the build machine. He just has to copy down the image.

Admittedly, using VPC will slow down the build, but the benefits could very easily outweigh the negatives.

Obviously, you guys have been dealing with this perfect build quest for a while now. I, on the other hand, have a different perspective. I’m fairly new to the “real world”, and have only worked for a company that had the isolated build machine. I never know much at all about what went into deployment; only that it was complicated. Too complicated for a neophyte like me. However, it worked and it worked well. QA built. Anyway, now I’m here at my second place of employment and lo and behold, they don’t have a build machine. Heck, they don’t even know who built the last release. Or, what went into it. OR, what version of Delphi the code was built with, and in some cases, where that code is. (We currently have 4 programmers and three different versions of Delphi.) Also, VSS was apparently abandoned long ago because basically nothing ever builds straight from “get latest”. I always have to ask someone, “Hey, what files do I need?” which is followed with "Oh, here, you need this. I didn’t add it to sourcesafe yet. " Jeesh. And, I’ve been given the wonderful opportunity to develop SOP’s for our little shop, including pulling everyone together and defining some sort of work flows. Basically, my boss wants me to get us all on the same page. Honestly, I’m not sure where to start. I figured I’d start with standardizing the build process. If anyone has any insights, I’m all ears. For now, I’ll keep trying to talk everyone into at least using their shiny new DElphi 2005 that they’ve had for about 5 months.

Anyway, now I’m here at my second place of employment and lo and behold, they don’t have a build machine. Heck, they don’t even know who built the last release. Or, what went into it. OR, what version of Delphi the code was built with, and in some cases, where that code is

LOL! Sounds like you have your work cut out for you…

My general advice is, for what it’s worth, is to make things simple. Remove any barriers or dependencies that prevent the build from being dead easy. XCOPY deployment is the holy grail:

http://www.codinghorror.com/blog/archives/000243.html

Is XCOPY for .NET technology only?

Pretty much, since the .NET framework is the only significant dependency.

I’ve ordered the Microsoft Virtual PC software

Sounds like you’re on the right track…

Where can I find out how to write a build script without using Ant??

Ant is java-oriented, just like MSBuild (soon to be released with VS.NET 2005) is .NET and C++ oriented. I am not sure if there is a Delphi specific build script language or not… I found this thread for Delphi, though:

http://www.cmcrossroads.com/ubbthreads/showflat.php?Cat=0Number=47826page=0view=collapsedsb=5o=fpart=1

Is XCOPY for .NET technology only? Our products are Win32 projects.
I have the network guy setting up a machine with XP and I’ve ordered the Microsoft Virtual PC software - should be here tomorrow. I’ll set up a couple different build environments (one module is built in delphi 6 and another in delphi 7) and apparently Delphi has some differences there, per my boss. As for eliminating dependencies, I have a big one! Microsoft Office - we use Access for our database. Ugg. I’ve suggested XML but that was quickly shot down with complaints of poor security. I’ll have to research that more. I can’t imagine that MS Access is so secure.

Where can I find out how to write a build script without using Ant??

Nice essay Jeff. I would have trackbacked it, but didn’t see here the url was. Looks like I’m facing the same thing others have in this comment trail. I found a Delphi Ant clone called WANT on sourceforge, but noticed it appears to be abandoned :(. I guess I’ll leave a note here if I come up with a build script. I’m hoping to get my team off VSS and onto CVS at the same time too. Wonder if this task will doom me.

here’s a link to my trackback since I couldn’t find the url to use: http://patf.net/blogs/index.php?blog=2title=magic_buildsmore=1c=1tb=1pb=1

While LinkedIn is ostensibly a office automation social network, it does offer an easy way to keep track of your business contacts. Because most people take care to keep their LinkedIn profile up to date, especially when they change jobs, it provides the easiest way to make sure that you have your contacts’ most recent contact information.

Excellent points. One problem with a build machine is that it can make the build process “fragile.” This is the idea that a build happens only in one single environment, you will have many hidden dependencies upon that one environment. The only way to know if you have such dependencies, is to regularly build in many (slightly) different environments. This does not matter so much right now while you are developing, but years from now, when you need to pull that software out to make a critical patch for a customer, having a robust, resilient build script that is not dependent upon a very very specific environment will be a huge benefit. I linked this article from my blog post: Don’t Baby Your Builds