Concluding the Great MP3 Bitrate Experiment

blog
61

To Pablomedok above, you see how people can really make things up. TOS-link cables transmit digital signal and it is only a yes/no question (signal/no signal). A good TOS-link vs a bad TOS-link is the amount of signal error. All you would hear is how many glitches and jitter, but never how warm or how sweet or how much bass or treble etc. If someone says he can tell these kind of audio characteristics out of a TOS-link, he is just fooling himself and the readers.

Back to MP3. As I mentioned in an earlier comment here, the industry standards (AAC@256kpbs etc) are developed to be transparent in most cases, and I presume MP3@320kbps can do that too. Of course I don’t rule out exceptions and also people with good ears. If you know the technical details enough you can even generate test tones to trip up these codecs, say a square wave or some harmonic combinations. But most of the time it is very hard to tell the difference to most people.

On the other hand, when Jeff claimed “that difference should be audible in any music track”, it is also flawed. For an extreme case of a pure 1kHz tone. It is accurately reproduced down to 64kbps or even less.

Nevertheless, I still believe someone with good ears and equipment can tell the difference for this bad sample although there is little audio quality. Indeed many here claimed they can, and I don’t doubt that. I use average gears (Macbook + Sennheiser PX200II) and yet I can pick up the worst two. Which sample is better than the other is the big question. But hang on, aren’t we talking about telling the difference? We should not be rating which one sounds better. But we don’t even have a reference, where is the difference??? I think Jeff should think of a more proper methodology for conducting his next experiment!

62

Just another note. Jeff is the lucky one who is satisfied with 192kbps files. I once kept all my music at this bitrate and one day I hated them for I can tell the artifacts. Then I had to redo all of them in 256kbps and that was a lot of work! I don’t keep lossless simply because they are large. I might as well just store all the CDs under my bed!

63

The only sad thing is that most songs in online stores are only available in lossy mp3 format, a step backwards in comparison to CDs. And the more people are convinced that mp3 is enough for all times, the less likely lossless songs will be offered by the stores.
Will be again buy the same songs in 10 years (as we now do for those we buyed on tape or vinyl) when we then discover that after all mp3 is not the best format.

64

With storage being so cheap and music at top quality taking so little bandwidth anyway, I don’t see the point in trying to be efficient. It takes all of 10 min to rip a full CD in full FLAC glory with EAC.I much rather have the lossless version that I can re-encode as I please if there was ever a need.
Similar to diamonds, most people can’t tell apart a great one from a good one, but if you were relatively rich and could easily afford the great one, why settle?

65

thank you to Xentrax, who posted the gizmodo bitrate test music in the previous thread,

http://gizmodo.com/5251247/the-great-mp3-bitrate-test-my-ears-versus-yours

The Carmen and ‘Feel Good’ samples display vivid differences even to my old ears, using a Behringer DAC and Grado headphones. I could not hear anything in the cheesy samples at all, they were indifferently horrid…

66

Fascinating. I had a hard time hearing a difference between any of them. (Or the Gizmodo samples.) I was surprised even though I already knew I wasn’t an audiophile. But that’s good - it means I can spend less money on audio equipment and more on books!

67

I’ve recently gone thru the experience of converting my cd collection to both FLAC and variable bit rate mp3’s. One of the things I learned about (sadly half way thru the process) was Vortex Box (http://vortexbox.org/content/134-About-VortexBox). Basically it’s a headless OS you put discs into(cd,dvd, or blue ray) and it automatically rips and tags them. Music goes to FLAC movies to MKVs. You can also mirror those formats to others. It’s pretty slick, it would have saved me a lot of time.

68

We made a similar test at home, after investing in a nice DAC. We have a decent amp and nice vintage speakers. We used a range of music, but I think most of our test was conducted with a track of Sibellius (Finish classical composer) which has a nice dynamic range, and subtle wind and percussions which can be hard to render (and which we both happen to love). We are both amateur musicians, and love music, but we don’t consider ourselves hardcore audiophiles.

Here’s what we found out, in a blind test:

  • Plugging in an iPod directly in the amp, through the audio jack, was awful. My husband’s laptop, bought for the quality of its soundcard, was only marginally better. But in both cases, that setup was not only distinguishable, but annoying. Ok for party playing, but not for any serious listening. Unfortunately, that’s how 99% of people listen to digital music, no?

  • There was a slight difference between MP3 and CD, both driven through the DAC. In all honestly, I could hear a slight difference, but I couldn’t decide which one I prefered. But my husband systematically distinguished the MP3 and CD (he has better ears than I do, apparently). But we both agreed that for everyday listening, both were acceptable. But we kept the CDs.

My point is that for most people, the quality of their audio system is so poor that trying to optimize the data source is a moot point.

When we were shopping for an iPod dock for our baby’s nursery (just a cheap little something to play lullabies), we were appalled at the quality of what’s offered. Most of those sold in large electronic stores sounded only marginally better than my iPod’s built-in speaker. It took quite a while before I could find something reasonably priced and suitably compact that I wouldn’t cringe listening to.

And this, along with car stereos and cheap earbuds, is what most of the population listens to.

69

I don’t know if you mentioned this earlier, but I think this is more a test of peoples hardware than their ears or the bitrate. I recently bought a pair of AKG headphones and noticed a lot of detail I has missed in the past using my speakers or in the car. So your test is more about do people have good enough audio equipment to hear the difference between bit rates.

70

Great experiment in mp3 bits. I really appreciate you.
live chat software

71

Very cool, now I can tell that I heard correctly but interpreted it incorrectly. I listened to the samples in order (on the first listening) so I unwittingly took Limburger as the baseline. I correctly recognized that Brie and Feta were closest to Limburger, with Feta being worse. I also recognized that Cheddar and Gouda were almost identical but different from the other three.

The compression did noticeably changed the sound, but I interpreted that as a sort of “punchiness” that was supposed to be there (because it was in Limburger) which the Cheddar and Gouda “failed” to capture, as if a bit of the high frequencies was missing. So my ranking was

  1. Limburger
  2. Brie (very close 2nd)
  3. Feta (more distant 3rd)
  4. Cheddar
  5. Gouda (near tie but a hair “worse” than Cheddar)
72

Even knowing that Gouda is best and Feta is worst quality, and having read comments about “listen out for sibilants and hi-hats”, I can’t hear a difference between the two. Maybe it’s time to de-wax my ears.

And to those saying that “storage is cheap, I can fit all my FLACs in a 500Gb hard drive”, I say “show me a portable MP3 player with 500Gb of storage”.

73

The problem I had with the test is that that particular song has lots of audible processing in it normally. What I hear as artifacts in compressed music sometimes sound like the processing. Higher frequencies don’t tend to have a stable stero location I find and dance around in your head. With audio as highly processed as this was, it was hard to tell.

74

The problem that I see with this experiment is that not everybody used speakers/earphones capable of make the difference distinguishable. With some speakers or earphones you can hear the difference, but with others you can’t. I tried using my cheap speakers and I couldn’t hear any difference. Then I tried with my high-quality earphones and the difference was pretty clear.

As an experiment I think is great, but the speaker/earphone quality is an important factor that shouldn’t be ignored. Of course, in an online experiment it’s not really possible to control this.

75

You need to vary your audio samples and use music that has a greater tendency to display artifacts (e.g. Live Music). I’ve personally done extensive audio bitrate tests and I’ve found that I can hear artifacts in live music up to 256K CBR, thus I prefer either 320CBR or V0 VBR.

Other than that. Great choice of topic!

76

+1 for Kyze; the quality of the speakers matters.

77

Re: Aaron Em

You appear to have missed my point entirely.

I suggested electronic music because conducting this test with digitally produced music could potentially eliminate any inaccuracies introduced by recording anything with microphones. You could produce a sample containing any concievable sounds you wanted, and the lossless version would contain completely perfect reproduction, sample rate issues aside.

Perhaps you should stop and think for a minute before replying with such violent gestures. Not all electronic music is Skrillex, nor are all people who listen to electronic music uncultured.

78

Sorry I’m a little late to the party, but this is an example of laypeople skewing the data. I work with audio, and I sharply disagree with everything about this experiment. I mean, I’m a programmer by trade, but music is my life-long hobby, and nobody disses my hobby :smiley:

On earbuds and common “PC speakers”, you can’t hear compression artifacts very well - heck, you can’t hear the music very well either - because the equipment itself is very noisy, underpowered, and non-linear. Did you ever notice, on cheap speakers you tend to turn up the volume a lot more ? You’re subconsciously trying to push over the noise floor! On a fancier system (or good headphones with an amp), there is significantly less noise in the electronics themselves, so you hear more of the signal. In other words, you hear more detail at the same volume level.

If you think about that for a moment, it means high-bitrate, high-fidelity MP3 files are wasted on low-power equipment, because those compression artifacts you’re trying to avoid are almost entirely drowned out by your equipment. It’s like trying to see through a foggy window, doesn’t matter what’s behind it, all you perceive is a blurry mess.

Along that same vein, many people have noted that the particular track you chose was very poorly mixed. You’re feeding crappy audio in, and getting crappy audio out. If the source material had a wider dynamic range, the differences would be far more audible, even to untrained ears. The weakest part of the signal chain dictates the output quality, and in this case it was the original audio itself.

Twelve years ago, when I thought my Altec Lansing ACS54 were the bee’s knees, 160kbps was sufficient. Those things had a constant hiss that I had tuned out of my consciousness. Years later, when I upgraded to proper home studio equipment, all those old MP3s sounded absolutely vile.

On my current setup, in a regular house with typical noise like fridges humming and my PC’s fans, 256kbps is “CD quality” to my ears on most material, but 192kbps is noticeable in all but brickwalled pop or metal (again, crappy source material masks the encoder noise). In a club, on a kilowatt sound system, I can readily hear the muddiness in a 256kbps encode, because those tiny little aberrations and pumping effects are scaled up to a level where they stand out.

The same is true of earbuds vs sealed headphones. Those little white earbuds that come with iPhones ? Yeah, I’d rather sit in silence than use that junk, the stuff they output does not resemble music to me. You could play a 96kbps track through those and be none the wiser. Swap in my 300ohm Beyerdynamic 990 Pros, and you can actually hear the bass again, as well as the fingers barely squeaking down the guitar strings. If those subtle details get smudged by an encoder, you can be sure I’ll notice.

79

When watch movies online you can stay lazy during weekends and holidays. You can stay in bed as long as you wish, keep your pajamas on, and watch your favorite movie on your own computer and with the comfort of your bedroom. It is especially beneficial during long winter evenings or rainy days.

80

Maybe someone here can help me. I am extremely picky about the audio quality of my music. I’m running into the problem lately that when downloading music some of it comes out sounding distorted - very faint popping, crackling noises similar to static. This stays true on every audio device I play these tracks on. Is this simply a bitrate issue or might there be something else involved?

So far I have tried to compare bitrates of the songs that sound distorted to the songs that do not, but I have soooo much music I’d rather not go through all of it! My findings have been that 320kpbs tends to sound fine and the distorted songs range from anywhere below that. My boyfriend doesn’t seem to be able to hear these crackles and pops that drive me crazy and we actually end up having fights about it when trying to listen to music and I finally snap and can’t take it anymore and have to stop listening to that album altogether - since it tends to be an entire album with a few exceptions.

I know I’m not imagining these distortions, they are not on every track I own and like I said it carries over to sound distorted on every device I play them on.