@Ben Moxon my two main languages are PHP and C++ both of which I don’t want syntax errors in when they are in source control. They might not always be tested, but having code that compiles makes merging conflicts a lot less painful, it also makes deployment to the test environment less painful (no more, “what was the latest revision that did work?”) and it’s easier to add new people.
Some of the comments on here so far are really scary. Having a build script should be one of the very first things on a project. Admittedly, it is much more work to build one for an existing project, but it is absolutely well worth the time to do so for the reasons Jeff mentioned in the article - getting a new developer going and getting a CI build going.
People who say that CI servers (and build scripts) add no value (or worse - reduce value) must not have worked on a project where they have been used properly.
The project I’m working on at the moment has had a build script and CI build from the start. Our checkin process is:
- developer gets new functionality going (with appropriate tests) in their ide
- they run a pre-checkin target in the build script which runs all of the unit tests (around 2500) and selected integration and distribution tests
- if these all pass, they checkin and the CI server does a full build which runs all the tests and publishes the generated artifacts.
Granted, keeping the build scripts up to date requires more work than pushing F5, but having a repeatable build process more than makes up for it.
I get the feeling that some languages provide better tools than others for writing/maintaining these scripts, but that shouldn’t be a reason not to use them.
(This feels like a bit of a rant - but I know my life is much easier with all these things set up)
Hi,
Build process is not just about F5. I totally agree. And I cannot rant more about an automated build process and continuous integration and build scripts. I agree to that also completely. But can the new employee ramp up be resolved by continuous integration?
So what I am saying is a new employee gets a new laptop. He gets a URL to his source code repository. He fires up his source code repository client, links to the URL, gets the latest source code, opens it up in Studio and starts enhancing. the moment he checks in, the code is re-built and all the integration procedures of running unit tests, applying standards and coding rules, checking for code coverage, etc is triggered. does this happen?
There are a no of other parameters. A codebase might fail because of a digital certificate not installed properly, or proper authorization not given to the database server. We tried to tackle this problem using Virtualization. The code base is installed on a Virtual PC and is given away to a new developer with all environment variables set. The only thing that remains is to give him proper authorization. Even after all this effort we are struggling with the standardization procedures. And still the ramp up procedure takes over a week.
Of course there are soft reasons like non-availability of the right people to help you with it, etc. but even so…
We are not planning to prepare a webcast specifically designed to ramp up the new intern. And a no. of such initiatives.
Do let me know what you think…
AUGH! Jeff, you just hit my Big Issue At Work.
I agree with you wholeheartedly. I’d been working with a client whose build and SCC process is SourceSafe, F5, and ZIP files for packages (all manual, mind you). And there was talk of “evaluating TFS” for its “Project Management features”.
My response to that:
“FUCK NO”.
Why? Because you’re not there yet. Before you can/should work on creating a heavyhanded TFS development “process” (not that I believe you even should), you need to be a couple steps higher in maturity:
Level 1: SOURCE CODE CONTROL
You can’t do JACK unless your projects are ACTUALLY in SCC. That means not in VSS. Don’t have any money? Learn and use Subversion. Wait 2 months for the “new” SVN way of doing SCC to sink in for your development team. Make sure everyone on the team really knows the ins and outs of SVN. Don’t proceed to any other step until you’ve done this. This is the bedrock of all non-joke software development.
Level 2: A BUILD FILE
Make a build file. Have it MSBuild your .sln file initially, so it works. Choose MSBuild or nAnt. They’re both good. nAnt is a bit more mature, MSBuild is getting there, but if you’re a pure MS shop it’s probably the one for you, since you’ll be able to learn some of the ins and outs of MSBuild by studying your .csproj and .sln files. Put your documentation, deployments, everything you need in the MSBuild file. Make sure it builds on couple of peoples’ machines.
Level 3: A BUILD SERVER
Use CruiseControl/CC.NET or equivalent. Get your build server up. If you can convince the Man, put it on a REAL server with a LOT of CPU, disk, and memory. It should reference your SCC provider and make your builds. You’re now at the level of maturity Jeff’s talking about. Wait 2 months. Get the whole team up and on the build server. You’ll never go back.
Level 4: TESTS! (Yes, this is Level 4).
If you’ve been using F5 and VSS, I’m willing to bet a $20 that you don’t have any unit tests written. Work on them. Add the tests to your build process, and make sure your build server runs them with each build. Soon, maybe in a few months, you’ll discover that you need to write your tests FIRST. Start going the way of TDD.
Level 5: POLISHING UP
Since you’re using VSS and F5, you don’t have a serious deployment plan. Refine your build file. Cut out as many manual steps as possible to DEPLOY your app. Use WiX. Use InstallShield, I don’t care. But XCOPY deployment is not a real, viable deployment strategy. (Does it create your event logs for you? Does it upgrade your databases? Does it roll back in the middle of an install if something goes wrong? Didn’t think so)
Level 6: MORE ADVANCED PROJECT MANAGEMENT
Now you’re ready to add tools that integrate estimation techniques into your project. Your project build and deployment process is now repeatable enough that your estimates are no longer a joke.
Don’t have a big project to do this on? A lot of “one man” projects? Guess what. You need this even more. What happens when you go on vacation and there’s a critical bug? What happens when you quit? How do you know that someone has all of the files they need to completely deploy your app when you’re gone?
It is real shame the coding horror author always gives advice about how to things, but he never gives any advice on tools. I would really like it if he would once on a while blog about essential tools which everyone is on the look-out for.
At the end of the day every software development team needs a way to produce “official” builds for regression testing, deployments to Test group, and deployments to manufacturing/GA release team.
Official builds are “official” since they are stamped with a version number, compiled with fixed build/configuration flags, tag/checkpoint via source control with a unique build tag, and always compiled with fixed external library versions.
So it’s not important whether you use “make”, a batch script, or a GUI button, what’s important is that you need the PROCESS of developing a official build with a CONTROLLED build environment.
Because if you leave it to some “developer” who hits “F5” to generate a new build and put it on the “file server”, there is no way you can accurately verify that the “developer” is using pristine source code from source control (e.g. he could have modified a file and no one knows about it), no way to verify if he is building with the right library versions, no way to verify what build flags are being used.
I’d really like to see IDE packages (I’m talking to YOU, Eclipse) be able to use a build file as the build process instead of it’s own build process. You’d be guaranteed that the build-script that is part of the solution/project/whatever is the same in the IDE as on the build server.
Eclipse/ANT would be good.
Visual Studio/Whatever Scripting Language would be good, too.
g
While we use a build script to handle all builds for deployment to our test or production servers, we don’t use the script to handle builds on our local development machine.
We are building a very large ASP.NET application where relying on a build script to do a complete build every time we want to test changes to a single page would be a major headache. Luckily for us Visual Studio lets us build just that one page and the assemblies that have been modified by our changes. This results in a much shorter build time to do our testing and a lot less wasted time during our development.
I hope you don’t mean you should never use your “F5” key to run in a development environment. I would think it would be enough to just have the build script be used for deploys. This will help try to keep the process cleaner and more reliable. I am not a fan of the often practiced Visual Studio build then xcopy the results deploy method.
We use f5 to build, and have ccnet do an automatic build on every checkin using a giant solution file which includes every project we have. We unload projects we are not working on as needed so f5 is still zippy. Running the whole build script would take far too long to do on each build, so f5 suits us perfectly - in fact I would adovocate its use as that was VS provides a convenient way to select only the projects you are working on.
This article is falsely accusing the f5 key, when the problem is people not running a ccnet instance.
One thing overlooked in the article yet painfully noticed by some commenters: a build script requires a build script maintainer. A build server requires a build server administrator.
I have been that guy for several years in several companies. Observations from the trenches:
- Doing this kind of thing in a simple manner is relatively straightforward. At the first sign of scaling up, issues become rather more involved, quickly surpassing the amount of time any individual developer can reasonably be expected to spend ‘not developing’.
Therefore: don’t keep it under the radar. Make a case for it and get it into whatever process you have that decides on effort allocation. Preferably (imho) argue for someone to gain centralised expertise to avoid wheel re-invention.
-
Many competent developers don’t get source control. Or branching. Or deployment to anything beyond ‘another directory on the localhost server’. I find that this is disturbingly common amongst even otherwise rather capable smart people. In other words: one can follow a learning path towards Capable Developer without ever noticing the side street labeled ‘development infrastructure’.
-
Build test environment deployment etc activities do not exists within the development team perse. In many environments, this is where you will need to get to know the sysadmin folks beyond getting your desktop working and telling them to punch a hole in the firewall for your Voip toy. Seriously, there are considerations in deployment to live environments that are alien to many a dev team that has led a sheltered life.
Point to make: if you create a position (even part time) with responsibility for the development/deployment infrastructure, be aware that it needs to be within throwing distance of both dev and sysadmin people.
- Once you have infrastructure to build automatically, deploy automatically, run a test suite, etc, you might find that there are more processes that might be extracted (or even consolidated) from other places to be handled centrally, by the same folks. Examples:
= coordinate documents produced by technical writers and bundle those into a documentation product to go with the builds. Automate that.
= align the requirements of projects’ building steps and test suite (with their release frequency) with provisioning of server environments. (rather than sysadmin working on their own schedule and budget). Incorporate virtualisation in this concept. Let sysadmin folks worry about network, wider hardware issues, whatever, rather than installing identical stuff over and over again.
= provide a whole new dimension of capabilities for your QA people by giving them the opportunity to order testable (virtual) machines with very specific code content loaded at specific times.
(mind you: these are applicable to larger teams, rather than three-man shops)
Your nightly build has to be complete. This means not just the source code but any custom tools that are used in the build process. As said in other posts, the build process’ output should be your installation package and any documentation that you build.
If you are using a good source control system than use a private branch if you are going to work on something that will take you days or weeks and that is incompatible with the rest of the project.
We have a product with around 100 VS projects. We have a tool that steps through a list of projects and spawns off a task to build each project from the command line. This calls devenv directly. This way we are using the setting in the .vsproj and .csproj files and we get the same build from the script as we get from the IDE.
Our nightly build script starts by deleting all of the files that are not in source control (in ClearCase terms view private files). It updates the view, builds the tools, builds the product, and creates the installation package. Our .vsproj files have 3 configurations: debug, release and non-optimized release. The non-optimized release is the same as the optimized release but with optimization turned off. We use non-optimized release when we have a crash that we can duplicate because it gives us performance closer to the product and can be debugged because the source code matches the machine code.
We have two branches: A release branch and a working branch. This minimizes the chances of a file check in breaking the build.
We use the ‘team project set’ within Eclipse to get a new developer started. You can export this file from Eclipse, give it to the new developer, and have him import the project set. Eclipse will then pull all the projects in the set out of CVS for him. It’s really handy and can be done within minutes.
I think focusing on the build step is a mistake. If you are really looking at what it takes to make a developer productive there are many more steps:
- installation of the dev environment (IDE, runtimes)
- installation of dependent libraries and packages
- installation and configuration of source control client
- how to check out the repository
- installation and configuration of other subsystems (database, web server, etc.)
- local configuration (DNS, hostnames, logins)
- building the system
- running the system to verify everything worked
In our world - Python web applications - the build step doesn’t exist. But all the others do, as they likely do in every project. Creating a checklist on a wiki that walks someone through the drudgery of getting everything setup properly is a big win in my eyes.
At the project I am currently working on, we do have a server that every night checks out all code from CVS, does a full build, sets up the test databases and runs the complete set of unit tests with code coverage measurement. Every morning someone looks at the report from the nightly build to see if there were any problems (compile errors or if there are failed tests).
Having a nightly build process on a server is important, and it should also include automatically running unit tests.
We’re using mainly Java.
Garret said: I’d really like to see IDE packages (I’m talking to YOU, Eclipse) be able to use a build file as the build process instead of it’s own build process. You’d be guaranteed that the build-script that is part of the solution/project/whatever is the same in the IDE as on the build server. Eclipse/ANT would be good.
- There has been Ant integration in Eclipse for a long time, didn’t you know that?
Someone asked about tools to do builds. In the .NET world you could use NAnt.
As I read “The F5 Key Is Not a Build Process” … I thought this post would be about usability … because in most applications (at least on Windows) F5 is the refresh key.
Who put that thing on “Start Debugging” …
Your post basically says: Build outside the IDE, Build outside the IDE, Build outside the IDE, Build outside the IDE, Build outside the IDE
Please give some good reasons too. You know, one could just run the build from command line even if the configuration is made in the IDE.
I’ve never worked on a project with daily builds but I’ve always worried about the idea.
Afaik, in most places there isn’t a “daily build”, there’s a build server and it starts a build as soon as there’s a commit.
It may take me several days to make the changes and get them all working. With daily builds I have to keep these changes checked out so they don’t break the build
Or you make your changes incrementally so that at each step the whole system compiles cleanly (and passes the test).
The other alternative is to use DVCS or patch-handling tools (see Quilt) to have a local history on top of the remote one
I would much prefer to check in my non-working code as regularly as I can (hopefully several times a day) so that I can see at a lower level how my changes happened and where a problem might have occurred.
Does this mean you have no problem with breaking the build for all your coworkers? Whoa…
A build server is merely another coworker’s working copy. If you break the build, you break it not only for you and for the server, but for every coworker who’s had the misfortune to update recently.
Is there a way round this problem when running a daily build?
Only commit working code.
adding proper build process into your project will require at least one more new dependency (http://www.codinghorror.com/blog/archives/000497.html) and add some more new code (http://www.codinghorror.com/blog/archives/000878.html) which have to be maintained and that’s not a good thing. so really, we’re talking here more about trade-off than requirement.
Don’t you think a dependancy on the closed inscrutable build artifacts of an IDE is a bit more worrying than on a plaintext script of a (usually) open-source build tool?
Within our company, most projects are a one-person deal. So, the F5 key very much becomes the build process.
yeah but… what happens if the dev in charge of a project leaves and someone else has to take over?
But can the new employee ramp up be resolved by continuous integration?
I think jeff’s point was more along the lines of: an external build tool/script gives you a central place where everything necessary to build a project is/should be visible in plain text.
Thus, a new employee wastes no time trying to track down deps and maybe-working code, check out, build, done.
And if he wants/needs more knowledge of the organization of the code and build process, he can just open the build file (and, often, learn the build tool’s syntax)
There are a no of other parameters. A codebase might fail because of a digital certificate not installed properly, or proper authorization not given to the database server.
This should be handled by one of the build’s targets.
A complete build system/script should allow you to take any clean machine (with all the pre-requirements, usually the build tool itself and the compilers/runtimes your language-of-choice uses), checkout the code, launch the build, launch the application.
If there’s any other step involved, then the build system is incomplete.
It is real shame the coding horror author always gives advice about how to things, but he never gives any advice on tools. I would really like it if he would once on a while blog about essential tools which everyone is on the look-out for.
That’s because there are dozens of tools for each task. Furthermore, Jeff is a windows developper, so the tools he suggests usually wouldn’t work for people e.g. developping Erlang software on Solaris.
I’d really like to see IDE packages (I’m talking to YOU, Eclipse) be able to use a build file as the build process instead of it’s own build process. You’d be guaranteed that the build-script that is part of the solution/project/whatever is the same in the IDE as on the build server.
Are you sure Eclipse can’t yet? I’m pretty sure IntelliJ allows you to specify Ant or Maven files to build the project through, so if it isn’t in base eclipse I bet there’s a plugin that adds that.
We are building a very large ASP.NET application where relying on a build script to do a complete build every time we want to test changes to a single page would be a major headache.
No build script should rebuild everything unless you explicitely ask it to.
The partial rebuild features has been available for years, make had it before I was born or something.
Many competent developers don’t get source control. Or branching. Or deployment to anything beyond ‘another directory on the localhost server’.
Then they’re not competent.
I find that this is disturbingly common amongst even otherwise rather capable smart people.
and they’re not smart either, sorry.
installation of the dev environment (IDE, runtimes)
usually manual. Better, as a new employee your machine should be preconfigured with everything you’ll need the moment you sit at your new desk.
installation of dependent libraries and packages
the dependant libraries are in the source tree, thus part of the build system
installation and configuration of source control client
it’s part of the dev environment, and was thus preinstalled at step 1
how to check out the repository
get the repository URL, checkout the project, done.
installation and configuration of other subsystems (database, web server, etc.)
should be configurationless or handled by e.g. an “install” target in the build file
local configuration (DNS, hostnames, logins)
there is no reason why a dev should have to do that, it’s part of the dev environment and should thus have been done at step 1
building the system
and running the test, all handled by the build system
running the system to verify everything worked
yep.
In our world - Python web applications - the build step doesn’t exist.
That’s because you restrict the word “build” to “compile”, which Python doesn’t need.
The concept of building goes far beyond the mere act of compilation, or it would be called “compile”.
Hey Now Jeff,
Good Post. After reading this I understand that a build script is much better practice than F5. You’ve taught me when building a doghouse the F5 key may work fine, however on a skyscraper a script is needed if not a dedicated server.
Coding Horror Fan,
Catto
Day one, line one, must compile.
How does it compile ?
Write a one or two line script…
Continue to develop, make as few assumptions on build environment
If a developer wants to use an IDE, hotwire a key to call the script from within that IDE or Editor.
Jeff is 101% right, F5 is not a makefile.