Caching needs to be considered, if it’s not already done, at every level: web server, php, database, etc.
Does the network, at every network interface, have traffic shaping/TOS enabled? There should be no possibility of an external influence “taking down” the internal network.
I would also suggest Joyent’s BingoDisk (http://www.bingodisk.com/) for your list. Low annual fees and no bandwidth charges or limitations. (Although they do have a somewhat vague warning against abusing the bandwidth.) They specifically mention that it is useful for hosting podcasts, images, etc. though.
Isn’t just buying a virtual host with a 1000 gigs of Transfer (or even more) at a hosting provider better? You say - switch to an external image provider. I say - why not just switch to a hosting provider?
Are there any advantages of hosting the blog yourself specially when hosting providers are offering competitive rates and giving you Remote Desktop access to your virtual servers with 1000 gigs of data transfer or even more at very reasonable rates (some of them are less than 50 bucks a month)?
I’ve never hosted out my house so I wouldn’t know but I’ve never had problems with a lot of good hosting providers. But then again, I’ve never hit crazy traffic like your site does 
So why do you host the site yourself? Are there any added advantages of doing that?
I’d just like to put a little warning here for people that are planning to mess with the IIS metabase. If you’re not careful with it, Notepad will change every X hundreth character to a “?” totally corrupting the file. Saving it as UTF-8 (which isn’t the default, nice.) may help, but using WordPad instead will definitely work.
Goodness, that is a fierce amount of traffic for a blog!
Maybe you could further limit the bandwidth use by requiring people to solve FizzBuzz before they’re allowed subscribe to RSS? /smartass
If you use PHP’s Smarty Template manager (http://smarty.php.net) it has a built in “strip” function that can strip white space from your HTML.
It should be noted that Flickr is a photo sharing site and not an image hosting site. There is a big difference in what those words mean and what Flickr’s purpose is.
DreamHost offers Files Forever if you have an account with them.
interesting observations… for those of us who don’t consume the amount of hosting bandwidth of this magnitude… what’s the dollar impact of a 9gb day?
(oh yeah… good move disabling comments at the height of traffic on that day!)
Very useful and in-depth. Thank you very much for sharing this! I would say that optimizing CSS/JS would be very minimum if HTTP compression is enabled, but I’m not sure (would really appreciate some statistics).
Excellent information Jeff! Thanks for sharing. -RR
One simple optimisation, that most people forget: Make sure to set the Cache-control HTP header on all static content - typically .js, .css, .gif etc.
A response with the header
Cache-control: public,max-age=7200
lets the browser cache content locally for 2 hours - it doesn’t even need to send a request with If-modified-since. It can cache it, even if it normally wouldn’t (for example if using https). Incidently, this also makes your web site much more responsive for people on high-latency connections.
Don’t overlook the cacheability of your site as a way to improve performance. Take a look at this excellent article and the cacheability engine that can test your web site for cacheability.
Thanks for the tips! This is a very useful guide you’ve put out.
First of all, kudos for having this problem to begin with. Great content here that’s highly deserving of the attention.
That said, you missed one. Most blogging software will already create well structured HTML that utilizes CSS for positioning and decoration, but not all of them do. If you are still using tables for positioning and have inline application of style elements (which, in really poorly formatted sites are repeated over and over instead of relying on styles created in CSS), that will cause your HTML size to be much bigger than necessary too.
The best argument I’ve seen in awhile for well structured HTML that relies on CSS for the positioning and decoration is this:
http://accessites.org/site/2007/02/graceful-degradation-progressive-enhancement/1/
It goes a step further and addresses accessibility, but the general rules for HTML structure stated here are a great guideline.
Thanks Jeff for this helpful information, especially for the CSS compressor and HTML compression hints; I never heard about them before reading this post. Great writing!
I started page-making for the web back when we were all using dial-up, and not only dial-up, we were using 14.4 at the highest speed, and I personally was using 2400 baud. I had an AOL connection and maintaining a website that was easy to use and small in footprint was highly desirable. Most of my images clocked in at around 4kb, no more than five total per page at the most. The page formatting and design were specially crafted for 640x480 resolution, and I managed to keep things small. I, of course, had a little-visited site, but I never exceeded my 2MB file-space capacity given to me by AOL’s FTP space, and I managed to have close to 20 pages.
The newer, younger generation doesn’t know about optimization. I was also a web monkey for a company for a long time and they could not fathom why I’d delay a page’s launch. I’d explain carefully, in 1996, “We can’t let a 50 MB page go through. I won’t even let a 1 MB page go through. Give me time to re-tool it so we’re not making visitors wait a week for the page to load.”
Ultimately, that company failed spectacularly. Small wonder. Anyhow…
Optimization GOOD. 15 MB JPEGs BAAAAD.
For compressing PNG images, PNGOUT (a href="http://advsys.net/ken/utils.htm)"http://advsys.net/ken/utils.htm)/a is the best utility I’ve found. One sometimes has to fiddle with the (many) command line parameters, but since it only overwrites the file if it can make it smaller, you can make a script that runs a number of common combinations to find the best one.
“pngout /c3 /f0” compressed the images in this article as follows:
bandwidth-usage 5593 - 4295 (77% of original)
official-logo 1973 - 1867 (94% of original)
text 1463 - 1294 (88% of original)
I was hoping you’d have some more tips for this, 200gb just isn’t what it used to be. I need as many tips as possible, I’ve cut load my remote hosting my images on a paid image service but I’m stilling getting killed here.
Based on how many comments are on that linked article you might want to limit comments to 100 or something. Or even better, if an article is being requested more than X times per second, don’t display any comments and link to another page with them.
I also remember reading an article years ago about yahoo changing their images directory to be named ‘i’ instead of ‘images’, this alone samed them gigabytes in bandwidth because they serve up so many pages, crazy but true.
Other tips:
use relative links
shorten massively used directory and file names