To:AaronG
I worked in a shop where if you broke the build, you were in charge of the build server. Attaching a “punishment” to breaking the build gently encourages that people not break the build.
Now that I’m working somewhere else, I wish that I did break that build so I would have learned how to babysit the build server. My current shop does not have a build server; and I’m thinking of ways to propose bringing one in.
Excellent post as usual, Jeff! I almost always learn something from your entries.
We’re slowly moving toward a better build process here. I’m loving FinalBuilder + FinalBuilder Server for avoiding the angle-bracket tax in MSBuild, and for those special touches like emails and deploying to a client “demo” site.
Thank you Aaron G. You just saved me having to type that myself. Jeff asks “What are you waiting for?”
At my gig we have a system that has not been able to be fully rebuilt for about 2 years. And, true to the corporate IT Dilbertism, the management hasn’t allowed us to work on resolving that issue.
Having said that: one of the first utilites I had our team work on (okay: sort of a side project, as IT-mgmt would never let us spend good time on things like that) was a dedicated build capability that can roll-out deployables to both windows and unix. Sweet capability.
I hope we get a chance to use it when/if we are allowed to fix the software-based Sword-of-Damocles we find ourselves living with.
Any professional software developer who is not:
1.) starting a software project based upon a design.
2.) developing software using source code control system
3.) using some form of built-in unit testing
4.) commenting in source files
5.) relying upon an IDE for anything other than code editing, GUI building and debugging
is simply not acting in a professional manner. These are not niceties, they are basic principles that anyone who claims to be a professional should be following.
Nice idea Smitty…
Pray you work for a company that has an IT and management team that recognizes that those “basic principles” that you mention are worth spending time and money on.
The sad truth is that many professional (and not-so-) software developers work in places that that is not the case.
Most of the time we were just invited to the dance, and others pick the music. And yes: this means that sometimes you just need to go find another dance.
I don’t really like the one-shoe-fits-all approach a lot of developers take. Yes, certain things may be necessary for certain people in certain situations. However, you can’t prove that your solution is going to work for everyone every time. There are almost surely edge cases that you haven’t considered. If you think you’ve found an exception to this rule, you should seriously doubt yourself.
(The above is applicable for (almost) every all-sweeping argument.)
Visual Studio actually lets you build a SLN or CSPROJ (or VBPROJ of course) file from the command-line, just run “devenv.exe /?” to see how. So there really isn’t any excuse! We used this in Paint.NET for awhile, along with an intricate perl script, to do builds and send e-mails to volunteer testers on a daily basis whenever there were any changes (no changes, no e-mail). And of course to e-mail us if the build broke. Sadly I have not set this up again, but mainly because I’m the only developer now so there’s no need for this type of coordination.
You might also add that automation of generating the installer is also super important (if relevant – any client app will need this). Visual Studio makes it easy to generate an MSI, but if you use anything more complicated than what it allows and have an NSIS or other custom EXE wrapper, it’s imperative that you automate it. I rolled the first few installer EXE’s by hand and it’s way too easy to fumble it.
Well I could not resist giving my 2 cents worth here. For starters, I just want to say it is an eye opening article for some, and some of the comments will definitely be appreciated by some.
I do agree with John Chapman’s response and just wanted to elaborate on Schmolle’s response.
For large projects with multiple development teams there is actually a professional CM (Configuration Management) Engineer. I actually managed a number of such teams, for a premier IBM partner building large bank systems that had 20+ developers broken down into different teams by layers, like the Presentation Layer, the Business/Data Layer, Integration Layer and the Database Layer etc.
Working with the IBM tools, we had the luxury of working with IBM Rational ClearCase, which solves all the problems people did not really find elegant and straight forward solutions for. ClearCase has been reviewed as the most sophisticated tool of its kind, much more advanced that VSS and CVS. I loved using these other tools in the past, espacially CVS, and would use again for small to medium sized projects meaning less than 10 developers.
Well besides nicely integrating with ClearQuest for automatic bug code tracking and WebSphere IDE integration, it has a solution designed specifically for the issues much talked about here, which is promoting partially functional code. Well it introduces Views, so a developer sees a particular view with certain files and commits to that view partially completed code as often as he/she wants. Then the next level up would be the streams, there can be multiple streams one for individual teams say per layer and unit testing and then an integration stream for when the different teams want to combine code, and run functional tests, be it automated or manual tests. What is great about ClearCase is that it allows every project to do its own CM design to fit the prject team and size’s needs as well as gives you out of the box solutions called UCM. It also allows you to to do baselines from which different teams can start of, besides branching management which is really for the purpose of have having multiple projects off of a product line. I just want to say I actually never owned IBM stock, and really am a big fan of open source and use it every chance I have, and hence in the Java and Linux/Unix world. I use Eclipse most of the time in the past few years, and yes Eclipse has had a CVS and Ant plugin come standard with it for the past 5 years.
Definitely have to agree. I created the build process at my work. My coworkers were a bit concerned over it, but they have all come around now.
This post inspired me to finally write my own post on the subject (I promised it in my blog awhile ago). http://brewder.blogspot.com/2007/11/creating-build-process.html
As with testing, or even source control, people that haven’t tried it judge it to be more complicated or time consuming that it really is.
In practice I found that the ability to check out a project and build it with a single command is invaluable. And I work with a small team, with 5 developers, normally distributed by 2/3 projects.
The time to get someone to compile one of our projects, from scratch? 1 hour! There are only 2 dependencies, java and ant (we are mostly a Java shop). Everything else is part of the build process.
And the time to maintain it? Well, the initial version is simple enough, when the project starts, and every time some of the devs adds a new dependency, it’s his responsibility to include it in the build. Who better to do it? He/She has just added it to his/hers environment. Every step is fresh and clear.
“The value of a build script is manifold”
Would that be the intake manifold, or the exhaust manifold?
I strongly disagree. Your build process should be F5! It justs needs to be done properly. I have personally designed systems with several types of clients and several types of servers, plus a dozen shared libraries. All you need to do is check out the tree from SVN, open Visual Studio, pick the project (think executable) you need to run and run it. Two more clicks and you can instantly push a new build to all users and servers. It just works.
Furthermore F5 is a build script. Eclipse creates an Ant file for you, Visual Studio uses MSBuild, which is comparable to Ant. In both cases, the IDE writes the build script for you and gives you enough options to tweak it (including a text editor), if you want to (but ideally you don’t). Therefore, I can take any VS or Eclipse project (probably the same for other IDEs) and compile it from the command line.
Although writing your own build scripts might give you geek credit, it is in no way effective, it’s just an example of hubris overtaking reasoning.
When I worked at IBM in 2006, it took over two weeks to get a new employee access to five different AFS cells spread across three different sites. This was required to compile our application. The good news is, they had a build process for that project. The bad news: it took over seven hours to compile on 500MHz AIX machines.
Ah thank you Jeff… one of my many battles… At least we’re using the leading edge .NET tech at my current office, but this whole Build Server, TDD, Multiple Deployment environments, etc. stuff is still lost on my 12-man outfit.
Mind you, we only have 2-3 on any given project, so it doesn’t take more than a day to get anyone on-line. But we learn so many new practices on each of our projects that we can’t really seem to make a set stick.
sigh we’ll get there.
I don’t understand. If you automate the effects of pressing F5 (or whatever) by using “devenv [solution] /Rebuild” and specify the environment, is that a build process?
Or are you arguing that it’s not a build process unless it involves a separate set of files? No, obviously not, that would be stupid.
I’m sorry, but your concentration on the keyboard shortcut completely hides any point that you’re trying to make.
Regarding the “only check in code that compiles” discussion: I prefer to check in code that has only a relatively small number of changes from the previous version. There are at least two benefits: 1) you can roll back to any point without undoing as many “good” changes, and 2) you can accurately describe the changes in a short change comment, and build a good change log. So while it’s nice if the code compiles, I think the “small number of changes” rule is more important. In a group environment, you can use labels or promotion groups to identify the last compilable version.
“The value of a build script is manifold”
Would that be the intake manifold, or the exhaust manifold?
Eric on November 1, 2007 11:35 AM
No… just manifold. Buy a dictionary and try again.
"I use PHP most of the time. What build? :)"
To this and other similar comments about interpreted languages: building is not (just) compiling. In the (only) slightly advanced example that you are not developing PHP with Vi on the web server itself, but rather on a different machine, possibly using a localhost webserver of sorts, your build process might serve no other function than making sure you haven’t forgotten about dependencies when deploying to another box.
I’ve defended somewhat involved build processes (with seemingly frivolous bells and whistles) on this basis: it allows you to do the work involved in deploying at the start of your project, rather than at the point where people are putting up the office decoration for the Grand Going Live of your precious app (when you find out that the live box for some obscure security-related reason does not allow the file system permissions you founded your architecture on).
Anecdotal counter-evidence: I once found myself (as the CM guy/deployment mule) painstakingly explaining to a Perl web app developer that ‘no, really, even though it works happily on his local machine, it is not a good design to have his app write state stuff to the file system on a 15 machine distributed load-balanced server herd. If only because it will functionally break within seconds.’
Usable-towards-management-point in a nutshell: a seperate build/deployment process is a test in itself (of the infrastructural capabilities) which allows for significant end-of-project risk reduction.