Hardware is Cheap, Programmers are Expensive

I must disagree with you today. You should optimize code, but not why you think you should not. There are three reasons:

  1. Someone else will have old hardware, and it needs to run well for them too.
  2. Scalability. You need it to be efficient, because although it might be OK to throw hardware at it and double the capacity when it is small, it becomes less reasonable when you become large.
  3. Optimized code should be simpler to read and maintain (optimize by simplifying, not by clever hacks) saving future programmer time.

Your attitude is why Vista stinks as bad as it does. There is no reason that what it is doing should require the hardware specs it does. Other OS’s do the the same with so much less hardware.

Throwing hardware at a problem, or benchmarking and refactoring the code, are no substitutes for having an experienced programmer design it right the first time. Demand may increase faster than hardware scalability. A good programmer can get it right early on in the development cycle, and will identity most problem areas before they become a problem, and will have a good idea about how to mitigate these in the future.

I see your point… and I disagree.

If Windows Vista was written as optimized as these demo programmers make their code, it would probably still run on hardware (without quality loss) that was already considered too weak to run XP.

If a process spends 5% more CPU time on doing something than it could, you will say, hey, who cares? But if you my system is running at 100% CPU and I have 10 apps that do something. As I said, all could use 5% less, that means 50% less for all apps together.

The inefficiency in your code sums up to the inefficiency of all the other code running on that system. It is a little bit like: Getting one cent from 100 people makes you just earn a dollar. Getting one cent from every human being on this plant makes you a rich man, but for every person involved, it is just one cent.

I agree with DMB. Throwing more hardware at the problem only masks the problem and it will rear its ugly head again, only this time with a lot more difficulty in making the true fix that should have happened in the first place.

We ship a complete solution (hardware and software), so qualifying and upgrading hardware is a difficult process and causes an increase in COGS. We are in a performance-sensitive industry where you are never allowed to slow down, even when you add new features.

@Mecki good point. Generally sloppy practices don’t show up as a single line item in the profiler. The code is too slow, but there’s no nice pareto chart showing the one thing you need to fix.

I am expensive, therefore my salary is too low.

It seems like you cannot count on Moore’s law anymore.

Until a few years ago, the CPU clock frequency increased all the time, but that development seems to have stuck at about 3.2 GHz. It’s hard to find a CPU faster than about 3.2 GHz today, and that hasn’t changed in over a year. And it seems like we have to wait for a while (for a technological breakthrough) before we can get faster CPU:s again.

Now we get more cores instead. But more cores doesn’t automatically make your program run faster, it has to take advantage of the parallelism, and you may need to hire some expensive programmers to fix that.

Throwing more hardware doesn’t quite work when you’re talking about client side of things (e.g. desktop software or JavaScript)…

This is all well and good advice for server side stuff where the code is only running on a few machines. However, for client side code, where you have to multiply the inefficiency by a million, a few optimizations go a long way.

I’ll also say that good programs tend to write better code to begin with, and that beginner programmers need to learn some of the basic optimizations and bottlenecks of programming. So my advice would be for beginners to spend a bit more time learning where their programs are bad so that they can become better.

@Dom - You ever hear of Vista?

Jeff,

You’re arguing against yourself here. You start out by saying that the best gains are in optimization, then you say not to optimize, then you say to optimize only when your code isn’t good enough. This hasn’t worked for Microsoft and it sure as hell won’t work for you.

Also, when I was looking at going into programming in 2002, according to the school counselor, the national average salary for a programmer was $50K. The average salary for a software developer was $55K. (The difference between a BS and a Master.)

I agree with your article Jeff, but you also have to be careful. In your case, adding hardware is an obvious choice. But what happens when you have multiple servers already running. Doubling your hardware power suddenly starts to become more costly…

For example, if you have 10-20 machines, each say at $5k. That’s anywhere from $50k-$100k. It’s now close to 1 year’s worth of salary. And if your system is growing in traffic, well…

At some point, and it doesn’t take too long, it becomes more interesting to look at software optimizations. For example a 1% optimization for Google in software will cost dramatically less than adding 1% of hardware.

Especially when you factor in the person time to build the machines, order them, maintain/update them, power them, etc.

You are in a sweet spot where it’s much better to buy more hardware. Enjoy it while you can, it’s a great place to be! And a fun one too :slight_smile: But once you reach about a dozen or so boxes, you’ll notice your metrics will flip on you.

Btw, I absolutely agree with getting the right hardware for developers. There’s no nice way to put it, it’s just plain stupid not to!

The world unfortunately is changing. Intel/AMD have both hit a wall on single core speed. Instead of getting about 2x performance every 18 months its about 20% in the same period and falling in growth every release. Processors are now caches with logic as they try desperately to increase performance with more transistors and many cores.

So I think the old rule of no optimisation is wrong, I think we need to make sure our applications will scale across multiple CPUs and in todays web applications that means being kind with the number of database hits we do and decent amounts of caching (that is until the database dies and is replaced with a better more scalable solution). That means giving developers multiple core boxes so they can test that.

One of the few applications capable of using the multiple CPUs efficiently is the application server. Generating web pages is easy to do in parallel (ignoring the database problem). Even this free lunch will soon come to an end. Consider what would happen if the CPUs in 10 years time had the same single thread performance as they do today. We face that very real possibility now and it means we’re going to hit a new limit where adding new hardware won’t help with increasingly complex and powerful applications.

Because single thread performance will remain the same and we currently in web apps marry one CPU to a user the maximum processing time per user is going to be 2 seconds. Users don’t want to wait for a page much longer than that including transmission time so lets call that the maximum. Today we probably aim to deal with a user in much less than that, say 200ms purely because we run more than one user per CPU at any time. But with the apps of tomorrow we’ll want to do more. Excpet we’ll hit a wall at 2000ms when single thread performance won’t increase further. At that point adding new hardware won’t help, its all algorithms from that point on.

Scared yet? I know I am because most algorithms do not scale even remotely as well as we would like. Even an algorithm which is 99% parallel can only ever get 100x faster regardless of how many cores you throw at it. Something is going to have to change in the way we generate pages, the way we write GUIs if we are to use all these CPUs in a single application.

As the great wise man once said:

We should forget about small efficiencies, say about 97% of the time: premature optimization is the root of all evil. (Knuth, Donald. Structured Programming with go to Statements, ACM Journal Computing Surveys, Vol 6, No. 4, Dec. 1974. p.268.)

but no-one advertises for $100,000 jobs for PhDs with two years’ commercial experience

I would NEVER hire stupid university people for IT jobs. Their knowledge and experience is nowhere compared to people who started programming as a hobby back in the 90’s/80’s and don’t have a single certificate.

Look at the many middle eastern, Israeli monkeys with their CCNA, CISSP can administrate their 10 routers in their country lawlz.

Paying 100K for someone just to sit there every day with his 0 knowledge and phd is a waste. This is a profession what you don’t learn in schools, but we all know that already.

Jeff, you comment is written from the point of view of a user. If you’re creating an application for your own company, sure, hardware is much less expensive than programmers.

This situation is different from the point of view of a vendor. if you’re creating an application which thousands of customers are going to use, your programmers are much less expensive than their hardware. You really don’t want to make customers pay more for your application because it requires more hardware. (Vista is an extreme example, as an OS, but this is doubly true for vertical applications.)

For example my company Aperio makes digital pathology systems. These manage lots of really big images. If we didn’t have efficient code our customers would have to buy more hardware - CPUs, disk, bandwidth, etc - and it would make our applications more expensive. We’d be less competitive and deliver less value. So for us spending programmer resources to reduce application hardware resources is the right trade-off.

BTW this discussion bears on the choice of language, too. If you’re creating applications for yourself, sure, go ahead and use C#. If you’re creating performance critical applications to sell to others, C++ might make more sense.

Jeff says, If a one-time investment of $4,000 on each programmer makes them merely 5% more productive, you’ll break even after the first year.

This isn’t actually true for small consulting firms that bill large companies an hourly rate for programmer services. If my boss spent an extra $4,000 on me, he wouldn’t see any of that money back, because every hour that I spend is simply money that he makes from our huge corporate clients. Being able to finish a project early is nice, but when you have clients who think nothing of projects going over time and over budget and will just keep paying as long as it takes, then there’s little incentive to make sure things don’t slip.

Your point about making developers want to keep working for you is good, but as long as the hardware I’ve got is reasonably decent, it’s not going to make a difference as to whether I switch jobs.

i wish i could get a new pc at work. i’m running a pentium 4 with 2 gb of ram. it’s painful to use.

Don’t forget the cost of server licenses.

Microsoft’s SQL Server licence policy is getting much tougher, to the stage where a busy top-range 8-way server can cost $80,000 or more. Per year. In license fees alone.

When you start to include the cost of a DBA to administer the hardware then alternatives involving more efficient code become very attractive.

My current employers are seriously considering throwing out their SQL Server installation and going with a /more/ code-intensive solution because of this alone.

Sure hardware is cheap, but sysadmins arn’t.

I worked on a large project where the hardware is cheap card was played early on. We ended up with a system with 20 servers…

Oh but the system was replicated across two sites, we had a qa system, the customer had a qa system and there was a development system…so actually 100 servers got bought.

Then they needed setting up and running. We had a team of 10-15 people racking hardware, wiring and installing OS and configuring the system for well over a year.

In all honesty I think we could of put the system on 3 servers and saved a lot of time and money.