There is also the ETag (If-None-Match) and Last-Modified headers, but again this depends on the feed readers being aware of them. I think setting up Apache to handle Etag is particularly difficult when on an NFS filesystem, alas. At least it was when I last looked, or I misunderstood something.
I’m using a Pro Photobucket account for the simple reason that the interface for uploading/editing images is so much better than what comes with Wordpress. Pro was well worth not having to worry about months like February when I hit digg/reddit/stumbleupon/lifehacker/problogger multiple times.
I also have a Pro Flickr account but that just used for photography. I don’t find that the tools are as good as photobucket for blogging.
I use a free FeedBurner account for RSS. It’s great. FeedBurner is such a good product. Feed flares, reader chicklets, email subscriptions, good stats. Amazing tool.
especially after looking at the “common questions” to which you linked and spotting your error. According to that, their hourly limit is 100GB, not 10GB
I did notice that all of the images on the post I read yesterday were ugly “this user has exceeded bandwith” images
That’s because I exceeded the 100GB/hour limit on imageshack.us a number of times.
It’s why I switched from imageshack.us to a paid Flickr account…
Oh, and the “worrying about handing over my feed to FeedBurner” issue can be solved with buying a paid upgrade that let’s you use your own domain name for the feed.
This is a slightly different issue, and I’m not sure what blog software you’re using, but a lot of people recommend wp-cache for Wordpress blogs that are getting hit by huge traffic spikes.
http://mnm.uib.es/gallir/wp-cache-2/
Makes Wordpress as efficient as a static site.
Have you looked into AllYouCanUpload for your image hosting?
This service looked good until it forcefully converted the 5kb PNG in this post into a 22kb JPG… (!) WTF? Most photo sharing sites respect the original file format.
Also, Masklinn, this PNG requires more than 16 colors due to the ClearType font aliasing. We’ve covered this topic before…
http://www.codinghorror.com/blog/archives/000464.html
But yes, OptiPNG is pretty cool, and I do periodically run it on my /images folder.
Great article.
I’ve toyed with the idea of outsourcing the image hosting on my site, but I’m concerned that if anything should happen to the hosting service (going out of business, change of TOS, etc.) it would be a nightmare to find another host, re-upload years worth of images, and change all the links on my site. Yes, Flickr seems like a pretty safe bet as far as Web 2.0 goes, but who can really guarantee it will be around in 2010?
Call me a control freak, but I’m keeping my images on my own server for the time being. It’s worth the extra few dollars per month to me.
Wouldn’t it be great to just have a faster/bigger pipe and not care about bandwidth? I hope that bandwidth issues die like modems issues died sometime before I die…
This causes a problem with software such as DotNetNuke in which downloads (such as a zip download) is piped to the response through the ASP.NET runtime.
For example, using Response.WriteFile.
Phil
The only problem with using Flickr as your image server is that more and more corporate filters are blocking it. I see nothing but red x’s.
I’ve gotten an email about the flickr block, too. For those of you behind a corporate filter, does ANY of this stuff work?
http://www.flickr.com/
http://www.imageshack.us/
http://www.photobucket.com/
http://www.webshots.com/
[update from reader: ALL of these are blocked, with a weird oversight in the case of imageshack: they don’t block the imgxx.imageshack.us servers ]
Looks like Amazon S3 is the only viable choice now … I’ve updated the post to reflect that choice.
Thanks Jeff, good ideas. I just added some new TODO items to the web publishing system I’m working on – access policies that restrict too-frequent requests from the same source, and automatic whitespace stripping when you edit a node.
I would guess most RSS readers also are sending proper headers with their requests so the website only has to send them the RSS document if it changed-- are they?
Reed
Hosting images on a different server might have a minor performance hit on the client side due to the additional domain reference.
I would think that you could improve much further simply by restructuring your blog. There is no need to post several blogs on one page. Why show all of the images and everything for blogs that are several days old? Just show a header if you need to. Or add a table of contents. Also, as someone else suggested, paging the comments would help tremendously (but at the cost of usability). Finally, I come to your blog often but don’t always read the article itself if it doesn’t interest me. Having a table of contents and forcing people to actually navigate to the blog would save bandwidth from people like me who only need to see a header and a short description to know that I am not interested. Why send me pictures, article text, and several old blogs if I’m not going to even read it?
Kudos on switching to S3. I hear it is a great service. I do have a comment though:
“the ironic thing about this is that the viral post in question was completely HTTP compressed text content anyway.”
Since you are now using Amazon’s S3 have you thought about putting your semi-permanent static files up there (eg. css/javascript)? IIRC they set HTTP headers to suggest the data remain cached forever, so both you and your readers win.
Cheers!
Say, what about coral cache?
Linking through Coral: http://www.codinghorror.com.nyud.net:8080/blog/images/coding-horror-official-logo-small.png
All you do is use their domain. After a few requests it ought to cache it for you. Downsides are that it is slower than a direct request, and like the other offloading solutions, you are dependent on their DNS system to work.
i’m going to second (or third) some of the commenters here and say you should also explore caching if you’re concerned about rapid-fire access. if the bottleneck is processor speed you can just do internal caching, but if the bottleneck is purely the number of http requests you should check out external caching. external might be pretty pricey though… haven’t used it personally, just for work.
Huh, I didn’t even notice the feed switchover until I checked the logs of the feed client I’ve been writing in my free time:
I, [2007-03-05T17:01:07.650092 #30592] INFO – : fetching feed http://www.codinghorror.com/blog/index.xml
I, [2007-03-05T17:01:08.479549 #30592] INFO – : Feed http://www.codinghorror.com/blog/index.xml had status code 304
I, [2007-03-05T19:09:09.354200 #30592] INFO – : fetching feed http://www.codinghorror.com/blog/index.xml
I, [2007-03-05T19:09:09.920126 #30592] INFO – : Feed http://www.codinghorror.com/blog/index.xml has permanently moved to http://feeds.feedburner.com/codinghorror/
I, [2007-03-05T19:09:09.933376 #30592] INFO – : fetching feed http://feeds.feedburner.com/codinghorror/
I, [2007-03-05T19:09:11.381360 #30592] INFO – : Got http://feeds.feedburner.com/codinghorror/ (id 210) with status OK
I, [2007-03-05T19:09:11.381550 #30592] INFO – : Parsing fetched feed http://feeds.feedburner.com/codinghorror/ (id: 210)
I, [2007-03-05T19:09:12.351278 #30592] INFO – : Updating http://feeds.feedburner.com/codinghorror/
I, [2007-03-05T21:10:59.955918 #30592] INFO – : fetching feed http://feeds.feedburner.com/codinghorror/
I, [2007-03-05T21:11:00.132586 #30592] INFO – : Feed http://feeds.feedburner.com/codinghorror/ had status code 304
I suppose enabling compression would also be a good thing to do (it’s on my todo list). With the checking for whether a feed’s updated though, it hasn’t been as high a priority.
If you’re worried about feedburner, one thing I’ve seen a lot of sites do while tailing my logs is to just do a temporary redirect for feeds, either a 302 Found or a 307 Temporary Redirect. While it won’t completely eliminate your bandwidth usage, it should allow you to switch underlying external providers without having to pay for any “professional” accounts on any single provider’s service.