Public MP3 Listening Test @ 128 kbps (October 2008)

A lot of development happened since the last public MP3 listening test @ 128 kbps conducted by Roberto Amorim. All encoders experienced major or minor updates that should improve audio quality or encoding speed and we have a totally new encoder on board.

The purpose of this test is to find out which popular MP3 VBR encoder outputs the best quality on bitrates around 128 kbps.

Warning: As Hydrogenaudio Forums members and I found out, there is a bug in the iTunes MP3 encoder that is present in all iTunes versions since 5.0.1.4 (including the tested version 8.0.1.11). The problem manifests itself only on multi-core CPUs (both PCs and Macs) and leads to a significantly lower bitrate of the encoded samples. Apple engineers confirmed our findings and assured me that the next iTunes release will contain a fix for this problem. Please notice that the only functioning work-around is to disable all but one core in your computer's BIOS or to start Windows with the /numproc=1 parameter. Setting the iTunes process affinity to one core does NOT help.

• Which codecs and settings are tested?

  • LAME 3.98.2
    -V5.7
  • LAME 3.97
    -V5 --vbr-new
  • iTunes 8.0.1.11
    112 kbps, VBR, highest quality, joint stereo, smart coding, filter below 10 Hz
  • Fraunhofer IIS mp3surround CL encoder v1.5
    -br 0 -m 4 -q 1 -vbri -ofl
  • Helix v5.1 2005.08.09
    -X2 -U2 -V60
  • l3enc 0.99a (Low Anchor)
    -br 128000 -mod 1

• Is it normal that the bitrate is very high on some samples (even 228 kbps)?

  Yes, and that is the beauty of VBR encoding - it will simply ignore bitrate limitations whenever possible, using as much bits as needed to encode a problematic sample.

  Although that raises issues of fairness, it is the best way to compare modern codecs that shine most in VBR mode, like MP3 and Vorbis. Trying to force a VBR setting to match a desired bitrate, although fairer, is far from the usual practice of audio encoding, where it's more usual that a user just sticks to a quality setting, not caring much about a specific bitrate.

  The quality settings for the VBR codecs were chosen because they average out to about 128 kbps over a number of encoded albums. It would be unfair to tie the hands of VBR codecs and punish them for being smart about where to spend what turns out to be the same number of bits over the long run.

• Who should take the test?

  Anyone interested in lossy audio quality, or people who have no interest but would like to help making this test better are invited. You don't need excellent hearing, but some good gear is welcome. Headphones are a must-have.

• How do I take the test?

  Download ABC/HR and the readme from the following location:

  • ABC-HR_bin.zip

  If you already have ABC/HR, download the readme from here:

  • readme.txt
  • readme.htm

  Further instructions and links to the sample packages are inside the readme.

  Please note: if you don't hear any sound when using ABC/HR, simply change the device from ABC/HR's settings. In some cases, ABC/HR might not default to the primary audio driver.

• Can I take the test even if I am not running Microsoft Windows?

  Thanks to schnofler, any person running an operating system with Java Runtime Engine 1.5 can participate. Instructions are available inside the ABC/HR package.

  Some users reported that ABC/HR hangs when using the Java Runtime Engine 1.6. In case you are affected, please download and install JRE 1.5 from the following link: http://java.sun.com/products/archive/j2se/5.0_15/index.html. You do NOT have to uninstall JRE 1.6 in order to use JRE 1.5, but please make sure that java.exe from JRE 1.5 is used.

  Since the working directory has to be the location where abchr.jar is stored, the best thing you can do is to open a command prompt window, navigate to the location where you stored abchr.jar and the rest of the files and then call "C:\Program Files\Java\jre1.5.0_15\bin\java.exe -jar abchr.jar". Of course, this is only an example which you have to adapt according to where java.exe is stored on your PC.

• When will the test finish?

  The test is scheduled to end on November 3rd, 2008. This date may be extended if proven to be necessary.

  Update 1: The test was extended to end on November 8th, 2008.

  Update 2: The test was extended to end on November 22nd, 2008.

  Update 3: The partial results are now available (detailed results will be available in a few days).

  Update 4: The final results are now available here.

If you have any questions, feel free to contact me via e-mail: mail@listening-tests....

Best regards,
Sebastian Mares

© Sebastian Mares, 2008 | Layout by Dibrom | Portions of the Text by Roberto Amorim