“…completely saturate two T1 lines-- nearly 300 KB/sec-- for most of the day…”
A T1 is 1.544 mbps or ~193 kBps , two of them = 386 kBps. Transactional TCP headers were really 86 kB/s or nearly 22% of your bandwidth ?! That seems high even for small pages.
Also I imagine you’re paying a small fortune for a real T1 from Telepacific Communications to your location in Richmond, CA ( Vertigo Software ).
Shop around for a cheap hosting solution, I suspect it’ll be safer to put the page on a hosting site rather than deal with it on the company network. Or buy a Fiber link that Verizon is offering out here on the east coast : verizonfios.com , the $50/month offering gives 5 mbps upstream. The $189/mo offering gives 30 mbps down / 5 mbps upstream.
As a precursor to outsourcing your image serving, I would have thought that you could do much to reduce the size of your image files. That 46,300 Bytes for the image seemed a lot to me. Clearly, outsourcing your images altogether is a total fix, but one day those nasty TC may come back to bite you. Plus, smaller images will be served faster wherever they are retrieved from.
I would strongly recommend Macromedia Fireworks (now owned by Adobe, I think) and Adobe ImageReady to allow you to see the same image with different image compression levels side by side. Mind you, from past posts, I realise that you know all about image compression…
Presumably, you now have an extra step in your blogging process in terms of uploading images to your image server. I’d be interested how this effects your blogging…
One thing you forgot to mention is that while compression does greatly increase speed of download for a page; if the user has a crappy proxy or has a not-so-perfect link in general, pages will inevitably drop key bytes. Not so much of an issue when it’s actual HTML or CSS but it’s quite a big deal when it’s 30 bytes of compressed data.
These are all very good suggestions, most of which I have been coding into a CMS I’m working on. One thing that you have not mentioned though is to enable content caching by making sure that all of your page elements are served with ‘Last-Modified’ and ‘ETag’ headers, and then responding with an HTTP ‘304 Not Modified’ when the client is smart enough to do a conditional get – that brings the bandwidth for page refreshes as close to zero as possible. You can also add an ‘Expires’ header with a time value in the future and get the clients (and/or intermediate caches) to not even go back to the server for fresh copies of content. ‘Expires’ is most appropriate for images, but also for javascript and css files. If you add a future Expires header to the content pages themselves, then people will not see comments right away, but you could use a five minute offset or some other very low value for content that is routinely modified.
As a precursor to outsourcing your image serving, I would have thought that you could do much to reduce the size of your image files. That 46,300 Bytes for the image seemed a lot to me.
Mind you, from past posts, I realise that you know all about image compression…
He could be much better though. E.g. the codinghorror-bandwidth-usage.png image in this post is 5593 bytes.
Opening it in Photoshop and then saving it for the web by reducing the number of indexed colors from 64 (!) to 8 brings it down to 3633 bytes, and running the output PNG into PngOptimizer (as simple as dragging dropping it) further reduces it to 2992 bytes.
Personally, I love flickr, for photos. Using flickr to store random charts and graphs feels like mudding the waters so to speak. It also means that if enough sites are using flickr for off-site storage and even a few of them are dugg hard enough the entire flickr user-base will have to suffer. Flickr also is known to go down from time to time as well so that’s another reason not to use it for site-image linking.
On the other hand we’ve done some research as well and S3 comes out on top. It’s a perfect fit for what you’re proposing and it’s also the most cost effective.
Using flickr is like using a big rock to pound a nail, sure, it works but it’s not really what it’s for, while S3 is looking to be a rather nice hammer to pound the storage nail.
We hit a huge bandwidth bottleneck at my previous employer, and the first thing we did was turn on IIS compression. That worked pretty well, but we noticed significant improvements when we switched to x-compress (www.xcompress.com). It caused lower CPU usage on the server, and seemed to compress the pages more.
It’s pretty common for shared hosting providers to offer more than a terabyte of bandwidth a month for ~$7.00. That’s no excuse to slack on the optimizations but you can serve 10 gigabytes every single day for a month and still not even use a third of your allowance.
Just stick with one of the big providers that don’t particularly care that your site just got on Digg or Reddit or Del.icio.us as long as you’re within the bandwidth limit as long as your site isn’t using 100% of the cpu on their multi-processor-multi-core enterprise blade systems. (Smaller hosting companies will shut you down a dime because of “cpu usage”, larger hosting companies give you a lot of latitude).
I suppose this does not matter in some situations, and especially if the in/out stats are gathered from the router, but…
If you want accurate numbers make sure to use mod_logio, as the bytes out field under the regular mod_log_config can report figures that can vary 10%-50% on successful, complete, requests.
I would also update your ISAPIRewrite to include feedvalidator user-agent.
FeedBurner’s FeedMedic “Validate your Source Feed” links points to feedvalidator, so in case you would like to check your feed for errors, feedvalidator should get the real source, not the feedburner’s version.
An alternative javascript compressor that I’ve used is Dean Edward’s Packer(http://dean.edwards.name/packer/). On the pre-filled example from the JS Minifier page the result was worse, but I just tested some code from a personal project and the results are as follows:
Pre-Compression Size: 5257
JS Minifier Size: 2695
Packer Size: 2134
I just wanted to throw another tool out there in our quest for smaller bandwidth bills.
I’m also a fan of Photobucket. I’m a little intrigued about Imageshack, especially after looking at the “common questions” to which you linked and spotting your error. According to that, their hourly limit is 100GB, not 10GB.
