I’ve come to realise that there is always a cost with solving a problem, regardless of how you solve it. That means that if you have a number of different ways to solve the problem, then the best solution is the one that best solves it, and that solution is the one that best matches the requirements. Different problems might have different requirements - e.g. least time, least cost, most scalable etc.
For example, if solving the problem in the least amount of time is a high priority requirment, then something like throwing hardware at the problem is one of the better solutions. Changing the algorithm to a better one might be another option, and as long as the time invested in doing that is cheaper than other options then it might be the best way to solve the problem.
A great paper I’ve just read, which Programming Pearls referenced, is Hints for Computer System Design, by Butler Lampson. It goes throught the various ways of solving performance problems e.g. use cacheing, using hints, and, of course, using brute force a.k.a throwing hardware at it. Programming Pearls and The Practice of Programming also have good sections on performance improvement techniques, that are also applicable to outside of programming.
Given the rapid advance of Moore’s Law, when does it make sense to throw hardware at a programming problem? As a general rule, I’d say almost always.
No, not even close to almost. Your neglecting the fact that most computer systems out there are embedded systems - where the rules change significantly. Desktop systems may be the most visible computers, yet I bet that if you take a close look at your household you’d find more microprocessors running your microwave, toaster, remote-control etc. than you have desktop/server systems. Now do the calculation again with 5 million units where the difference between a uC with 16K Flash and 32K flash may be as small as $0.02, gaining a possible cost reduction of $500,000. According to your chart that’s the salary of five senior software engineers working one year around the clock tweaking the code size from 21K down to 14K to make it fit into the smaller uC with a couple of bytes to spare.
What about cases where the same software is deployed over large number of hardware? In this case it would be justified to spend more on the programmer rather than upgrading all the hardware.
As a performance tester of 10 years. Tuning gains the most benefit. Cheap or Expensive hardware doesn’t solve the problem. I seen a maxed out Sun E10000 thrown at problems (2.5mil) (5+ years ago) the application still performed terribly few hundred users terrible times. The application couldn’t scale at all. Distributed or parallel applications need lots of customization. = programmers. Typical performance problems in order are: 1. Load balancer setup 2. Network setup 3. OS/DB/Machine setup (hardware and software). 4. Layer communication (RPC). 5. Easy Application fixes. (SQL, searches, data, cache). After those in depth optimization has little effect the problem is architectural and the whole thing needs to scrapped. I seen this too.
What about the performance cost of band-aiding a poorly chosen architecture?
Most of us don’t make money by writing demo programs. I’d rank architecture choice as far more important than simply algorithm choices.
The problem is most enterprise architects seem to be people pulling a specific vendor’s framework, etc. out of their hats. It seems that most people have forgotten what architecture really means.
This sounds like the typical excuse to write bad code quickly and then expect the systems guys to make it run faster with new hardware. Write good, fast code to begin with, and always having the newest, fastest hardware isn’t necessary. The fact is, there are lots of crappy programmers out there.
For badly structured code no amount of hardware and power use will break even with a GOOD Developer.
You must think that only BAD Developers are in play.
In the late 70’s / early 80’s when run size changed from 4M to 16M, that programmers to use on mainframes, bad developers started to read whole tables into memory to run faster. Yes that single job did run faster, but the whole system ran slower and more conflicts of sector in quue showed up.
Lastly with these faster computer are not faster, because most have bloat of the OS that take up that extra speed. Add to it bloat that developers leaving in their programs…
There are situations where you cannot throw hardware at it. Take for example a portable device - throwing hardware means you have to throw batteries at it, which means the person using the device has to carry a heavier and bulkier machine. Sometimes this is unrealistic - these people may be mobile 8 hours a day, and never near a source of power.
Embedded devices is another area - throw in another DSP and CPU, and it can push up the price of a consumer device beyond what people are willing to pay, or reduce margins to make the product not worthwhile. (Also, programmer time scaled across many devices may be cheaper than the hardware - take devices like iPods which can sell millions - a beefier processor may cost more than the complete RD effort - $10/part versus $1/device for development).
Also, throwing hardware at a problem caused by idiotic programmers is completely nonsensical. Sites like TheDailyWTF.com have plenty of listings where some programmer iterates through a database manually, making a simple update or query take 1000 times as long as it has to because the program is doing work the database can do faster, and the program is doing it 100 times in order to generate say, a web page. Or by a programmer using bubblesort instead of quicksort - proper algorithm choice can have far better ROI than throwing exponentially more hardware at the problem.
Your article assumes that the choice is between hardware and developers, and steadfastly ignores the non-linear costs of purchasing, installing, and maintaining both the servers and the data center space to house them. Running hardware is NOT cheap, it’s merely wholly externalized to the developer.
It’s often easier to believe hardware is cheap – and create an entire team’s worth of grunt work – than it is to studiously approach the problem at hand.
Optimized code should be simpler to read and maintain (optimize by simplifying, not by clever hacks) saving future programmer time.
You are essentially right, but that is Refactoring not optimizing. Refactor your code to make it simpler, remove redundancies, call out patterns that you see and put tests around your code.
Once you have refactored your code, it is much safer to optimize… if you still need to. Now with some proper tests around it, you are at much less of a risk when optimizing.
I avoid theoretical rules of thumb like this article. You have to look at your problem and find what works.
That said, when there is a real issue to solve in terms of optimizing, I’ve not found hardware to be the answer in the majority of cases.
Perhaps all you mean is when you know hardware will solve it, it’s cheaper. Maybe, but the costs of that isn’t just slapping a box on a rack, often your dev costs will move over to your infrastructure engineering costs. And as things scale you’re going to spend more on supporting infrastructure hardware and space.
This article also must be assuming a web type app, not something like a shrink wrap, where you can’t control the system requirements (marketing does that)
There is a point of no return (or at least diminishing returns) on hardware as a performance solution. Once you scale beyond one box architecture (or one web server to one database server), the complexity of the system spikes as locking and synchronization become fault points in the application. Ironically, the expensive programmers you skimped on are now the ones you need to design a system that doesn’t deadlock a clustered web farm connecting to a clustered database farm and you have both - expensive hardware and expensive programmers. The expensive programmers, if they are really worth their salary, will also be saying that the system is overly complex, and fixing the fundamental design problems will remove the need for all the extra hardware developers are spending 90% of their time on, instead of adding features.
fire the bad programmers!
otherwise in 5 years you will only be able to hire bad, overpaid programmers
to support the problem with faster, better hardware doesn’t solve it …
My first reaction when I read a post like this is that such effects are local.
My second reaction is that American programmers won’t be making that kind of money much longer, hardware costs notwithstanding.
I spent a year managing a web dev team in China. I’d estimate they were 1/4 - 1/5 as efficient as European/American programmers in the same tech zone (I’m basing efficiency here on the number of revs we had to do AFTER beta.)
Most of my employees were making USD$400-600/mo. No benefits, no withholdings. That was their take home pay and the cost to the company. This was a generous middle-class salary in Xiamen China. My Sr. developer – who spoke four human languages and easily ranked with most western programmers skillwise – made the luxurious sum of USD$1200/mo.
So these guys were 1/4 to 1/5 as good, but made 1/10 as much as an American programmer. You do the math.
There are places in the world where it DOES pay to throw people at a problem.
BTW I definitely think you get better product from a highly paid team in a country like the U.S. But just because you spend twice as much on your product doesn’t mean it’s twice as good.
It’s not premature optimization, it’s short-sighted optimization that’s evil. If you optimize the first piece you start writing, especially if it’s done in a way that reduces flexibility and generality, you can end up with a legacy code mess during initial development.
But some early global thinking about performance, before you’ve frozen the organization and algorithms, can give you asymptotic speedups, and that’s well worth it.
Micro-optimization can generally wait, unless you’re positive that a particular inner loop is both required and performance-critical.
I believe you have been poisoned by management though. Throwing hardware at most performance problems is ridiculous. If the application isn’t design from the begining to scale it will never work.
The proper solution is to design for performance, create an architecture that will be able to spread to as much hardware as possible, and then program for such architecture.
After all this is done I agree with you that programing optimizations in algorithms is mostly a non issue - compilers and frameworks usually take care of those.
I simply don’t get the point of this comparison. It’s like I’d say that it is cheaper to buy a fridge than a year’s worth of meat for the family.
Buying hardware is one thing and really does not cost much (I’d gess you need to be a software developer to a) think that it is news and b) stop thinking there). According to most studies it covers only 10% of the costs for the lifespan of that hardware. And these studies don’t take into account the personnel operate and administer the hardware.
Doubling HW rarely doubles performance or throughput. However, doubling production server hw usually requires similar purchases in development, test, stress, uat environments too. In addition excessive disk requirements can quickly lead to huge backup storage requirements. Figure a copy for everynight of the last month (30) plus a monthly for the last 7 years comes to a multiplier of 110 copies of that data for compliance purposes plus whatever copies are in the non-production environments.
HW is a small fraction of overall server costs.
I saw a vendor’s suggested €250K hw purchase add up to €6m when software licencing was added (ftp software, clustering sw, db software, middleware, app server s/w, backups, etc). Their ‘HW is cheap’ attitude was working out so expensive that we were willing to send them packing.
I do databases. I get fed up when database developers say that the database infrastructure is performing poorly when new code is releases into the stress environment. When we investigate, we discover basic errors such as table scans.
In a recent scenario, we were asked to find out why the hw was performing slowly. We discovered a missing index that boosted performance on a key user action by 8000. I would hate to know how much hw would be required to provide the equivalent HW performance.
In summary, good HW can double / treble performance. But spending some developer time can improve performance by thousands. But it does cost time. The main question is, is it fast ENOUGH for the purpose?