Hard Drives -- breaking the Terabyte Barrier

I recently upgraded my home system with one of the 750 gigabyte Seagate perpendicular drives in order to consolidate a number of hard drives I had on my server. 750 gigabytes is a tremendous amount of storage space in a single drive-- but it doesn't quite get us across the magical terabyte threshold. It's looking more and more like the first terabyte desktop hard drive will arrive sometime in 2007.


This is a companion discussion topic for the original blog entry at: http://www.codinghorror.com/blog/2006/09/hard-drives----breaking-the-terabyte-barrier.html

Dell already has a 1TB HD listed under one of their XPS bundles, it is the high end one i think.

I have a very strictly structured music directory, all done automagically based on tags and a small script I wrote. I never use it, because like Foxyshadis the search capabilities in my media players make it far easier.

A student in my Intro into computer science class has had a 1TB harddrive for 2 years. What amazed me is that he filled it already.

Jeff,

What about performance and seek time? Have you looked at the difference between your perpendicular 750 Gb drive vs a striped array of the same size? I am also leary of keeping everything on just one drive, having recently lost two drives in two machines in about as many weeks. In both cases I was very grateful that the machines were an array.

I’m in the process of upgrading a 4x250GB RAID5 (850GB usable capacity) to 4x750GB on the same controller. The problem is that I bought 1x750G as an interim step and it’s filling up faster than the price of more 750GB drives is falling.

Joop, WinXP and many programs slow down horribly if there’s too many entries in a directory. With XP that number seems to be about 500 (much smaller for image software that generates “folder thumbs” and doesn’t cache (like, say, windows explorer), so I’ve gone to YYYY/MM/YYYYMMDD-event for photos, and I’m about to go music/a-f/artist/album for music for the same reason (over 1000 artists, windows explorer is becoming unusable - if I accidentially click the “music” folder its ~10s before I get the machine back).

I remember when doubling my storage space meant purchasing another cassette tape (TRS 80 Baby!).

And then for my Amiga, upgrading it to a full megabyte of RAM cost me $99.00 (look who’s screaming fast now!).

I’m actually in dire need of extra drive space for my laptop. How naive of me to think 100gigs was enough :slight_smile:

Finding things should be really simple. Just keep things organized from the start.

./audio/artist_name/album_name/XX - track.mp3|ogg
./audio/artist_name/album_name/folder.jpg
./audio/artist_name/album_name/back.jpg
etc

./movies/movie_name (year)/movie_name (year).avi
./mobies/movie_name (year)/folder.jpg
./mobies/movie_name (year)/notes_if_any.txt

./photos/YYYY/YYYY-MM-DD_EVENT/00001.jpg

This is the basic file structure. Once you have that set up you can go OCD like me and then sort music by year, record label, style etc. All this can be done using symlinks. You can do it manually or through scripts. The same can be done for movies. Create dirs such as an actors name then symlink his/her movies in there.

Most important is to keep a directory called incoming or something to that effect and put all incoming goods in there. Only move them to the correct directory once they are complete and they are correctly tagged/labeled.

I remember spending $500 for a 420 MByte Hard drive in 92 (I think) so I could install Borland C++. Then everything was still easy.

Now, 100 GBytes is easy and cheap, and finding things is still not too hard. We’ve learned a lot about storage paradigms in the past decade, which helps a lot. Of course, if you’re just dumping things in your home directory, you’ll still have trouble.

1 terabyte? Bring it on!

RE finding stuff: there’s a free program online called Flexable Renamer. It’s (obviously) supposed to be a batch file renamer. However, I use it for a search engine also as it is WAY faster than windows’s build in search. I can search 100 gb in far under a minute. And unlike many desktop searches (Google, Copernic) Flexable Renamer doesn’t need constant caching to be this good. Oh yeah, and no install necessary. Completely green.

off topic – Jeff, could you make the colored background of the posts (which is a verrrry light blue) just a little darker? It would be way easier to distinguish posts (at least on my monitor).

Another data point:

Corvus Concept (IIRC) 5M and then 10M, big shoebox thing with multiple logic boards, hooked up to an Apple ][ - that’d be around 1980.

I saw tb harddrives a year ago. They are not new, they are just now entering and “afforable,” if you wish to call it that, range.

Mike/Andy – I am referring to off-the-shelf desktop hard drives… stuff you could potentially buy from an online or physical retail vendor.

I’m sure custom and expensive 1/10/100/1000gb drives came much earlier.

What about two 500Gb HD on RAID+0? :wink:

I remember getting annoyed trying to purchase dedicated 40Mb SCSI drives for developer workstations in 1991. I HAD to buy larger drives because the vendors weren’t making the small ones anymore.

My how things have changed!

Once paid $40K for 24Gb of Raid storage (circa mid-1990s).
Now my laptop has more space than that…

Oh the clutter I could have saved! (oh wait, squash that thought!)

-Eric

Joop, I used to be that nuts. Then overload an laziness led to losing lots of stuff, until winamp/foobar’s libraries showed up - now the organization or lack thereof doesn’t matter one bit. It’s all there after typing a couple letters. (Not everything fits into winamp’s view of the world, but minor changes to tagging or naming fixed that.)

My problem now is trying to evaluate corporate search solutions - since I rarely search for documents, I’m not sure what people really want, but we do need one.

Jeff,

Interested to know your thoughts on having just one hard drive these days. My paranoia now means that I run RAID1 on all my essential computers that contain data that I cannot lose.

I’d like a whacking greating 1TB drive, but just don’t feel right putting all my data on one drive - all the eggs in one basket, no?

So, ‘to RAID, or not to RAID’, is the question.

I am also leary of keeping everything on just one drive, having recently lost two drives in two machines in about as many weeks. In both cases I was very grateful that the machines were an array.

My recommendation right now is to get a 10k Raptor as your boot/system drive, and a larger, less expensive secondary SATA drive to store your data, applicatons, etc. If you’re worried about data loss, make that two secondary SATA drives in RAID 0 mirroring.

Striping is definitely not worth the effort on the desktop:

http://www.codinghorror.com/blog/archives/000335.html