Take a look at CPU vendors. Years ago they were scaling up. CPUs got more and GHz of processing power. Now they are scaling out, instead of making a single core faster, they add more and more cores.
Have system gotten faster because of this? Yes, if the application can make decent usage of all these cores they got much faster, even though we are back at around 2 GHz and were far beyond 3 GHz before. A current 2 GHz Dual Core CPU can beat the crap out of an overcloked 4 GHz CPU with a single core.
As you pointed out, the advantage of scaling up is that the software doesn’t have to be optimized for anything in particular, while scaling out only works well if the software can really operate that way. However, pretty much every software can. Just take a simple LAMP project (Linux, Apache, MySQL, PHP). You can have a single Apache server in front that gets all requests and spread these across a whole server farm of Apache servers, totally transparent. Each of these run only Apache and PHP. The MySQL Databases can be scaled out to different servers again. You can go as far as having one server not for every database but actually for every single table of a database.
Managing 84 servers is not that nice, and you should expect an increased % of HW failures. And since I work in a farm with 30 “barebone” servers and some blade servers I can tell the difference
Additionally 84 servers mean clustering or redesign of the applications running on them, and more maintenance, so additional time that hasn’t been counted in the calculations:
I don’t think this comparison is really fair.
you don’t need 83 1U servers to compete with the power of the 7U server.
So probably you might end up buying just 10 1U server to reach the same amount of CPU power.
This will also reflect on the licenses, power and hosting costs, which will be just 1/8th of what you write.
And the big monster is not enough… if you are talking about DB servers, than you’d need a cluster, which might cost even more (external disk unit, 2 physical machines) and will have higher costs attached (if I’m not wrong SQL Server Datacenter edition, the only one that run on clusters, costs 30K).
This also must be evaluated based on the kind of software you are running on:
DB servers are most likely to scale up, with redundancy
Web Farms are better suited to scale out, same with application servers.
Another option you have to consider is: putting everything up in the cloud.
A client of mine recently got their main DB turned into a clustered DB server at a major hosting company. Clustering (for them at least) basically means you get some amount of fault tolerance at the ‘mere’ cost of 2x the main processing boxes (i.e. 2x Windows Clustered servers) and some extremely expensive SAN storage that is used by both machines.
You, however, will quite possibly have the option of splitting your DB’s into 3 (I’m assuming one for StackOverflow, one for ServerFault and one for SuperUser). Depending on your willingness to lose ‘minuts’ of data, you could then vary the sizes of these servers by necessity, have them at different sites, and use a circular roubd-robin approach to log-shipping. In this way, each ‘DB’ machine would need to be able to store it’s own DB and the one it was receiving log-ships from, and computationally be able to ‘muddle through’ if it had to actually process two DB’s at once due to one server failing.
This is not true fault-tolerance, but may offer a pragmatic intermediate approach to the big iron and clustering and so on.
Another fun possibility is to rent instances on the Amazon cloud.
Prices run as low as $428 /year for a reserved instance or can be billed by the hour if your computing demands change frequently. If you build a good AMI image, you can automatically create and destroy instances as needed for load handling.
Of course if you “brick” an instance with valuable data on it, you’re very hosed. There’s no option to drive/fly to the data center to hook up a monitor and keyboard to the thing for the purposes of stepping past a boot time kernel panic. (this has been my headache this week)
I don’t mean the costs of scaling up vs scaling out in hardware, that’s pretty old news. But I was never fully aware of how hard shops using proprietary software were getting screwed until I read this…
In my experience, very few problems scale out easily. Scaling out always looks good on paper and never goes nearly as well. Having a lot of small things is almost always more painful and expensive than having a couple of big things in practice. Even if the big things cost 5 or 10 times as much. Even with outsourcing, people are expensive. Even if you aren’t paying them much you still have to manage them effectively. Effective management of people is the hardest thing to do in business, IMO. And the more “things” you have the more people you need to manage them.
Now I won’t deny that there are some very specialized, large-scale cases where scaling out is the only option you have. But before you go that route, you better triple check your plan and consult some people that have real-world experience with more than a calculator.
Having fewer, more powerful processors actully provides you more flexibility for handling intensive, linear tasks while still being able to “scale-out” virtually. If you currently have a process like that and you could spend 50K to bring a single new server in to make it run in 1/4th the time or you could spend 50K on a project to re-architect the process to be non-linear (assuming that’s even possible), going the hardware route is a no-brainer.
Markus obviously wants to be a one man shop so one big server makes sense. Clustering, replication, SANs etc. Too much work, too much to worry about with diminishing returns. You also do not want to rewrite your application ground up to support some kind of scaling out strategy. I don’t see Markus touching what is working already.
Bigger operations are a sprawling jungle of complicated and confusing software with piles of servers not doing a whole lot of useful things. That would be just about every company on the planet. That’s why Markus should be a scaling hero.
Scaling out a database is certainly possible when you want to do data ware housing/BI or MapReduce like stuff but scaling out your database when you want to do online transaction processing is hard.
I’ve had jobs ranging from embedded development on hardware that would fit in your shirt pocket to several years spent at a national lab with a floor full of supercomputers. It’s interesting to see that these scalability issues crop up in a similar fashion at all parts of this spectrum: “bigger” versus “more”.
But in many ways, as the manycore folks suggest, “bigger” has become “more”, as the way things have gotten bigger is to have more cores. Licensing issues aside, we are faced with transitioning our production systems from one or two core platforms to dozens of cores. This transition presents similar issues (differing perhaps only in scale or maybe not at all), whether those cores are all in one chassis or spread across a campus.
I’m guessing that we’ll all have to starting thinking about the memory architectures and processor interconnects a lot more carefully, much like the supercomputer and MPP folks have always had to do. Suddenly we’re faced with deskside MPP systems that aren’t the exotic hardware we once may have known and loved (or hated) but are the latest commodity whitebox we bought over the internet.
I believe that embedded developers and supercomputer developers may have a leg up on everyone else because they’ve been dealing with these issues for decades. The manycore folks at Berkeley seem to also have the opinion that the embedded area has a lot to contribute in this domain. I think product development organizations should consider leveraging any developers they have in the embedded domain to multi-core/manycore server-side development since (I hope) the skill sets are transferable.
I would suggest outsourcing the hardware infrastructure to Amazon Web Services. They have brought transparency to many of the hidden costs associated with scaling.
For instance, they have figured out the pricing of Microsoft SQL and present it to you in a way that is predictable, i.e. 12.5 cents per hour.
Obviously, a linear model doesn’t work for everyone financially – but it does take the guess work out of it.
We’re a Microsoft shop (and a gold partner, too, so our cost for OS/SQLServer software for internal development use is $0). We also use virtuals for development. So for us, the key feature of a server is the amount of RAM one can jam into it, as each virtual is going to be sucking up 1G of RAM (more or less, but it is a reasonable rule of thumb). So that honking great beast of a machine will be able to host close to 500 virtuals for development, while the 83x 1U servers are probably going to be close: about 6-7 virtuals per server. On the down side, 83x 1U servers = 2 full racks of servers with a bit left over, while the big beast is less than 1/4 of a rack.
Our current AC infrastructure has a hard time handling what we’ve already got. It is a 20-ton unit and needs maintenance several times per year (especially when the cottonwood trees are shedding and the roof heat exchangers start looking like cubical sheep).
We’re further constrained by corporate policy that states we are forbidden from obtaining “used” stuff, so the ebay items are way off limits. This was comically absurd when I was looking into an itanium server to develop against (we can check x64 builds with 64-bit vista/server 2003/2008, but for ia64, we need totally different hardware). The cheapest new itanium box comes from HP and starts around $15k, while used 1U itanium servers are in the $200 range (with RAM and HD, you’re still looking at maybe $1k per).
A lot of excellent discussion on this post - it’s been an interesting read. A few more things that are particular to my background…
As for software licensing - bbviously scale out will incur more costs in that arena than scale up (unless you’re 100% free/open source); how badly that affects you is extremely application and vendor dependent.
Scaling out is very hard to do if you didn’t design for it in the beginning. Let’s face it - most homegrown stuff isn’t designed up front for issues like this, and so it’s going to be difficult/impossible to scale many such systems horizontally.
Horizontal scaling requires a much more dedicated approach to system management - you’ll need to be able to automatically build servers, apply patches en masse, backup/restore, push content updates, etc. Content management systems, reporting and alerts - just more of everything. Moving from 20 to 80 servers isn’t hard, but the move from 4 or so to 20 is. This stuff is not trivial, but depending on how much value is in your infrastructure (is this your core business, what’s the size of your business, etc) it may be warranted.
Scaling out does have other hidden costs. You’ll need external load balancers, as I mentioned. You also may need a beefier network, better/more switches, NIC teaming, etc. Centralized storage like SAN or NAS - while I highly recommend it - can be complex and costly.
And of course, you can meld the two. Not only by separating your application and database servers and scaling independently, but by splitting the database into multiple servers. This may not always be possible, but at least we’re back to standard software engineering techniques of factoring and database design.
Virtualization makes a lot of these issues even trickier, but offer the hope of major gains if done right. You gain a huge amount of flexibility in adjusting workloads, CPU/memory allocations, or migrating systems to better servers. You also have an additional layer to manage, and have to be very careful balancing resources between virtual machines. This is especially difficult when all systems are the same, as you can’t use the differences in application design to your advantage (for instance, placing a CPU-intensive VM on the same system as an I/O-intensive VM, etc). However, don’t expect it to be cheaper - far from it. Hardware cost tends to scale exponentially once you pass 4CPU (16 core) systems, so partitioning a 32CPU server into 32 x 1CPU servers won’t likely be cost effective.
Beyond virtualization there are interesting technologies like Solaris Containers and other lightweight partitioning schemes to consider. Not even going there - this is long enough.
Finally, planning for failure. You definitely don’t want a single huge machine, as everything is in one basket. Many folks will then move to clustering for redundancy, but then you have a new set of trade-offs: do you go with Active-Active clustering, and watch your ability to handle load drop by 50% when a node dies, or go with Active-Passive clustering and have an expensive paperweight 90% of the time? It may hurt to have a passive server sitting there, but the last thing you want to see is one server die, immediately followed by your second server cratering under the load.
Scaling out offers built-in N+1 redundancy, in that a single node failing takes out very little of your capacity. This makes hardware management somewhat less critical, as you haven’t lost much capacity (relatively speaking), and you’re not sitting vulnerable on a single point of failure until it’s fixed.
So… from my perspective, scaling out is the preferred method, because of the usual cost savings, flexibility, simplified growth, and redundancy. However, it’s not for everyone. Your applications may not be written this way, your databases may be tied to closely to the applications, or you may not have the necessary support infrastructure for lots of smaller machines. Like so many things, it’s a set of tradeoffs - but that’s what keeps our field interesting.
Jeff, you may want to read the “Special Considerations” (http://www.microsoft.com/Sqlserver/2005/en/us/special-considerations.aspx) page about licensing on SQL Server. Basically it says that you need a CAL for each computer (or user) accessing the database, even if they access it through some other middleware application such as a web server. So if you have a public website, the only way to go for licensing is per processor. With the big 7u server with 32 processors, that would require SQL Server Enterprise edition (standard only runs on a maximum of 4 processors), which runs about $25,000 per physical processor, which would cost you $200,000 just for SQL Server licenses. On the scale out solution, you could run SQL Server standard edition, which is still $6000 per processor, which would run you $498000.
For someone who insisted that he build his own servers from scratch, you seem to know very little about the licensing of the software you installed on them. Which makes me wonder if you even have them licensed correctly.
This comparison is wrong because the Microsoft stack - Windows and SQL server - is not built (or priced) to scale out cheaply.
Systems that are built for large scale using a different set of technologies (i.e not relational database) to enable this kind of growth - map-reduce frameworks like Hadoop, column base databases, distributed data warehouses.
Scaling out is not a matter of choosing between 1 bigass server and 83 small ones. Software has to be built to scale out, and when it does costs should be cheaper…
It’s something of a compromise (albeit with all free software)—they scale up in some places, out in others, depending on the function of the servers in question (and take care to divide up the functions a bit so that they can do this). On the whole it runs rather smoothly given the size of the databases, the number of constant I/O operations, and the number of visitors.
As others have said, the management cost of 80 servers is non-trivial. Once you have that many, you’ll probably want to know if one of them has a hdd warning, an ecc memory warning, or a fan failure. For one server that’s not so bad, but for 80 you’re going to want to deploy one of the vendor’s hardware management solutions (which are free, iirc) but non-zero to maintain. Oh, and I suspect you’ll want to patch and deploy app updates to those servers, that’s non-trivial and non-zero cost also.
One other difference is that the larger servers from the major vendors build redundency into the server to avoid problems like a power supply failure taking out the box. The cheaper servers don’t employ those things to keep costs down and exactly because it is assumed you’ll deploy them in a way that one going down doesn’t take down the app. So while you have more of your eggs in one basket when it comes to the DL785, it is a more resilient system (and one big reason the chassis cost is $16k alone.) The cheaper servers are more cost effective but less reliable. It’s a reasonable tradeoff but one that has to be factored in.
Have you considered what the price point would be for say a setup using shinny new supermicro systems? They’ve got a 4 board in 2U setup that packs in 8 CPU’s (2 per board, so a maximum of 48 cores with the new 6 core I7’s) and 24 gigs of memory per system (assuming you want 1333 DDR3). They also use 2 power supplies to provide load balanced redundant power to the 4 computers (which are hot swappable as well). That’s packing some pretty significant power into 2U, never mind 6U:
2U
96 GB memory
48 cores, each capable of 2 threads
6U
288 GB memory
144 cores
now you could scale back your core count to 4 cores and the system would actually be affordable and you’d get more ram / core. These boards are also capable of taking more ram than 24, but only if your willing to give up the 1333 DDR3 speed.
For full disclosure, we are currently building a cluster out of these. The dell blade systems also provide an option that is somewhat comparable, and if you combine a set of blades together to act as one system using an infiniban back end and some clever bios level software(allowing you 2 sets of 8 blades acting as semi monsters) in one 10 U space. Each node having comparable stats to the super micro systems above.
