The Great MP3 Bitrate Experiment

Interesting test! I used AKG K240 monitor headphones on a mobo sound card and could pretty clearly pick out the 2 best and 2 worst sounding encodings. The real kicker will be whether what sounded good to me was the highest quality encodings.

Like others I ripped all my CDs to FLAC for a lossless copy that can be encoded to any other format without transcoding.

I get the convenience of lower quality encodings, though, and I’ll buy or stream lower quality from time to time for the same reasons I’ll watch a movie on Netflix streaming instead of buying the BluRay.

I whole-heartedly agree with @Bob Meyer. I was very surprised to read that “[t]he point of this exercise is absolutely not piracy,” followed by your plan to donate or sell your entire CD collection, thus giving up your legal rights to the content. I keep my collection of about 1,000 CDs in boxes in my basement so that I can prove I have a right to all of the music I’ve ripped from them. Without those rights, you would be a thief, just like everyone else who downloads or shares music illegally.

Destroying them isn’t a good option either, for two reasons. First, it’s wasteful and clogs up our landfills. More importantly, someday you may want to donate/sell some of your CDs (after deleting the bits from your computer, of course).

I hope you’ll update your post so that it doesn’t encourage piracy.

@Eric Larson: Vinyl sounds the way it does because of the turntable rumble. The low frequencies of the rumble, combined with the music, result in what are called “sum” and “difference” tones. They’re very quiet, but we experience the effect as “richness” or “fullness”.

It’s a pleasant kind of distortion, but it is distortion.

24/192 can be slightliy worse than 16/44 if your system cannot handle high frequencies well and you get nonlinearities anywhere in the signal chain. Then you can get intermodulation distortion from signals in the non audible range which could lie in the audible range. With 16/44 those signals are just not there and distortion disappears. From a theoretical standpoint, the best combination would be around 20 bit/50 kHz, this would be good enough for even very young listeners who might be able to hear up to 22 kHz and the dynamic range would be that of the very best microphones and converters.

I was so excited about this test. Then I heard the song. I’m sorry, but that was probably the worst song choice for this test. The original is highly-compressed, synthesized audio. Converting it to “uncompressed” “raw” audio is like playing a VHS tape on a 60" TV. It still looks like crap.

Furthermore, to add to the complaints, the sample has also ripping errors. At the beginning of every version, in the pauses, you can clearly hear scratching introduced by missed sectors while grabbing. That is imho the worst offense of a lot of DL music.

Hmm, I don’t think so – I used the most severe (accurate) mode of Exact Audio Copy when ripping the track from CD, and it reported a perfect copy.

Your brain, especially your senses, are inaccurate and imprecise sensors. Furthermore, our ability to analyze flaws in audio is even less dependable. Our brain is designed to fill in holes in our senses, and it does an extremely good job of it. Indeed, this is why mp3 and other perceptual audio encoding techniques work so well.

However, one must always keep in mind—you can just as easily fool yourself that you can’t hear any differences, as you can fool yourself that you can.

You’re right in the general case. One cannot hear a difference between the original CD and a bitrate of about 160kbps. But that’s because your ears adapt to it. They readily fill in the gaps; and they want to! But that does not mean there is no difference, and certainly the difference can be heard in untestable ways, not necessarily when focused on the details of short segments of specific tracks, but instead at the overall sound and feeling of the music over extended periods of listening.

ABX is flawed. It’s using the most inaccurate sensor ever designed: the human ear. I don’t trust it, and prefer to go with the highest quality available for that reason. Because I know exactly how the mp3 algorithm works, I know it’s designed to fool me, I know it does it quite well, I know I can’t hear any difference, and I know better than to trust it.

Well I must be a dog. Those five samples were as different as night and day on quality. I listened using my stock EVGA mobo sound card and the only headphones nearby: a crappy skype headset that came with my webcam. No challenge to identify which quality was better or worse.

would like to see you try this again with some symphonic classical, which generally has orders of magnitude more harmonics than typical pop/rock. in my experience, on good speakers, i can absolutely tell the difference between 192 and 320kbps mp3, and in at least one i could even tell the difference between 320kbps and flac.

There are a lot of misconceptions about lossy music, unfortunately. I’ve never done a test like this myself, but I tend to trust the listening tests of the folks over at Hydrogenaudio, who’ve consistently found that people can’t tell the difference between the original track and V2-encoded LAME MP3 (which is ~192kbps VBR), and that many people can’t even tell the difference with 128kbps VBR LAME MP3. (And I believe they have a lot of audio engineers on board, as well as people with really good equipment.) With that said, I keep all my music in FLAC format, just for the archival benefits and the pleasure of having the “definitive” version.

I’m interested to see what the results of your informal poll will show.

“Also: you obviously never listen to opera, or other continuous music split (gaplessly) into tracks. Everyone can hear the glitch at each track change, it’s that obvious. This glitch is an artifact of the mp3 encoding format, and it cannot be corrected.”

This was actually fixed years ago. Modern MP3s have extra metadata to avoid the gap, supported by all but the most obscure music players.

In terms of HD movies, though, I can’t agree with you yet. A Blu-Ray is generally 30GB—40GB whereas a compressed video is often 4GB—8GB, and it’s far easier to see compression artifacts with our eyes than it is to hear them with our ears. In addition, Blu-Rays offer director’s commentary, multiple languages, different audio configurations, bonus snippets, chapter selection, subtitles, etc. — none of which are supported by any of the leading digital video stores. (Of course, the pirates have had this stuff for years.) And what about 4k, when it becomes more mainstream? That’s almost 4x as many pixels! I doubt most people have enough storage for that.

You should have given us an opportunity to say “I have no idea” in the survey. They all sounded basically identical. But I also think that that was a poor choice of song.
So anyway, I didn’t submit the survey.

Brian: if only people who can hear a difference submit the survey, then the results will be inaccurate. It’s important that everyone who takes the survey submits their results, especially if they can’t hear a difference.

Jeff, are you still using Mediamonkey? I’ve been using it for ages, still love it.

Jeff, I’m listening on nice studio monitors (the ones I mixed my last album on) and was really excited about this test until I heard the sampled material. ALL of the files sound compressed, hissy and noisy to me. I then checked my original CD of this song and it sounds the same. The song is just . . . old. Recording technology has become so dramatically better, clearer, and more complex since then.

Complexity within the original file really seems to matter in compression, at least according to my experience. Today’s commercial volume wars have pushed the limits of “real estate” within the sonic spectrum - there’s a lot less “space” available in a modern rock recording as opposed to something like the Starship recording you chose for the test.

I highly recommend using a song recorded in the last 10 years with modern technology and re-issuing the test for more valid results.

The main problem is the white noise that is introduced while reproducing at low bitrate, most audiophiles don’t bother if the original content was crappy that is why they still listen to really old music… any way to troll a bit 24531.

i never thought blog writer like gouda the best, but who knows :stuck_out_tongue:

I would not consider CD to be reference quality, as you point out they are 44.1 and studios work at a multiple of 48. Every time I’ve mastered a track and put it on a CD it’s ended up with aliases because you can’t match the frequency of a CD to studio sound cards.

So it’s pointless worrying about this sort of thing.

I think this runs the risk of “pepsi challenge” issues. The basic premise here is that being able to consciously determine differences between a few samples is an adequate substitute for judging subjective listening experience over a lifetime. I don’t think that premise holds up. I think if you present a handful of songs with different encodings to people and ask them to say which encoding they think is better you’re going to get different results than, say, if you ask people to listen to different songs repeatedly over a long period of time. There are subtle effects that may not be noticeable on a single listen but become quite noticeable on repeated listenings.

As a case in point, have you ever listened to Pink’s “Raise Your Glass”? Have you ever noticed the high pitched beeps in that song? I’ve noticed that most people don’t notice them, but can hear them if they listen carefully. For example, http://www.youtube.com/watch?v=XjVNlG5cZyQ#t=58s “… raise. your. (feep) glass.”

The same thing happens in lots of other music. If you’re not paying attention you may not notice that sibilant s’s are a little distorted or that high-hat hits are muddy. More so if you don’t always listen to music with those sounds. But repeated listenings may make it more and more obvious, and then suddenly you’ll wish you had a better copy.

Encoding can be forever, it’s important to be extra cautious. Personally I think 192kbps VBR is an excellent encoding format, and for the most part it’s indistinguishable from uncompressed audio to my ears. But I’ve noticed that even 128kbps CBR is hard to tell from the original for many songs, though I would be hesitant to pay money to lock myself into such a format for life.

Ok, to all the folks alleging piracy:

Can anyone point me to a EULA from my Best of Berlin CD I bought back in '88 that says I no longer own the rights if I sell the physical media? Because, according to property rights as far as I can tell, just because I made a legal, fair use copy of something I paid for, once the physical media dies, I do not lose the rights that I bought. If I have the receipt, I bought it. Selling it onwards != unbuying. Also, since I bought the CD well before the DMCA, I believe ex post facto law says you can’t go back and change terms of a sale. So, at best, that might apply to CDs purchased after the DMCA in 1998. Might.