Mega Lossy v Lossless Test

I doubt I could get it right every time, I genuinely think the result are program dependant, at least thats what my own tests show.

A decent bit of well recorded violin usually sets them apart quite easily.

 

I agree.

I think you have to have a good reason to use AAC or mp3, ...... but if you do, .... then that's ok.

 

Do yo mean "program dependent" or "programme dependent"?

Are you saying the ability to detect differences between lossy compression and uncompressed data depends of the computer program used or the programme of music auditioned?

 

Not here. I've proven I can hear compression already and I've no intention whatsoever of getting into a pissing contest with you just because my results don't fit with your low-res files and all-in-one speakers ideology. I'm done with it now to be honest.

I'd be interested in another poll if Rob was to say stick a different yet similar Jane Monheit and RATM track up and see if the results alter in any significant way this time, i.e. will the results differ now some clues as to what aspects lossy compression tends to impact, and therefore what to listen out for have been provided? I'm pretty sure this is all Rob was intending to do.

Tony.

 

I have only just come across this thread which is a shame because I would like to have had a go. I wont now, simply because I can't be sure I wouldn't be influenced by seeing the results to date.

But by way of information I thought I would give some info on a similar test I ran 5 years ago. I was a member of an AV forum (avtalk) and posted a message that I didn't believe the majority of people could tell the difference between MP3's and CD's. Boy did I get some stick, to a man everyone thought I was talking bollocks and person after person swore blind they could tell the difference. One guy was so aggressive in his assertion that he could tell (remember I said majority) that I offered to GIVE him my £600 AV amp if he could reliably tell 100%. Guess what after a lot of posturing he didn't take me up on it.

But i did set up a blind test at home which a few brave souls came along to. Difference on my test was that ALL the tracks were mixed up so it wasn't a case of one group verses another they had to pick with every single song. Bit rate varied between 64 and 320. There were 10 songs with 2 versions of each.

The result, pretty much dead heat 50% right 50% wrong. So as I posted in the thread " a room full of deaf people and the result would have been the same"

Funnily enough very few people responded and certainly the guy who said I can tell 100% kept very very quiet.

 

Use the forum and ask the question - see if you get some responses from people prepared to do your test. I've seen three people do it repeatedly but there must be more so put out a call. You are wanting this test to show something it wasn't designed to reveal, so you will have to do something different. This test is about general reaction, and challenging bias against MP3/lossy.

 

Absolutely.

As i said in post 1, this is simply designed to ask questions, set some challenges, push things forward slowly and learn in the process. It isn't about embarrassing people at all.

 

As one of the three that got it right both times, I'd be willing to do another test, but not if it's just trying to make me 'fail'. I don't claim to be able to get 100%, but I think I can get a positiver result.

 

Well exactly, I don't want a test that just tries to make people fail.

The idea is simply to establish where we are - a rough idea of how these types of files differ, the magnitude of the differences and the overall ability of the lossy compression to pass unnoticed.

This and other tests have i think now done that, so we go up a gear.
Things like, what exactly are the audible differences, are they important, at what bitrate do lossy files become near as makes no difference to lossless, etc.

FWIW, you, Tony and Paul R always get this right based on the the testing I've seen but there will be others out there who also believe they can do it with 100 consistency.

 

I find 256kbs VBR to be a nice balance between quality and the ability to pile a load of stuff on my 16Gb iPod. It's very obviously better than 128Kbs and good enough to really enjoy via my computer / HD-600s as well as on a bus or train with cheaper headphones (AKG K450). I still go fetch the record or CD when I listen to my main stereo. I've ripped stuff in compressed format as I just see it as my iPod library, I'm just not interested in the 'home music server' thing at all - I'll wait for high-res subscription based internet streaming services for that. It can't be more than a few years away now.

Tony.

 

I can see the logic behind the high-res streaming solution but I'm not sure it will arrive so quickly.
In theory, Spotify Premium at 320kbs should be very good indeed but I find it lacking in quality. I wonder if some of the 320 streams aren't taken from the lower quality files.
It always sounds a bit soggy and washed out to me - a bit like AAC 128.

I still think a home server solution is gong to be best for some time to come. Yesterday I put a 1TB drive in my 13" Macbook Pro and with about 1300 lossless albums on it and all of the system files I'm about one third full.
I got tired of external drives and wanted everything in the MBP because music is pretty much its main function. £114 for a WD drive at that capacity - just think about the cost and size of that a few years ago.

 

If I consistently preferred the compressed in a short AB test I'd still want to listen to the uncompressed. Preference is neither here nor there, accuracy is all.

I have no interest in iPod type stuff so I don't use lossy compression, it serves no purpose in my setup where at the moment everything ends up coming from a hard disk. Having a CD player seems a bit odd nowadays.

(FWIW I think I got this one wrong, but I'm reasonably sure I could have ABX'd the files apart. )

Paul

 

On the face of it the results show that the majority of respondents could not tell the difference.

However, the limitations of the test are such that none of the results can be said to have any real validity.

 

In testing the population of listeners no they are not conclusive, but as I can draw my own conclusions from it they are very valid & useful.

Good luck to anyone trying to determine more & better statistics. The participation on this was apathetic & as far as going through 20 odd files I don't think there is a dog's chance of getting a result which will prove anything statiscally.

 

As I have said, it would be a fascinating topic for a PhD project.

It would need an academically rigorous test to establish anything.

 

The D-Dur lossless radio stream is back online again

http://radio.cesnet.cz:8000/cro-d-dur.flac

It is possible to compare it to other streams of the same programme material here

http://www.rozhlas.cz/d-dur/english

I have compared with the 256 OGG stream


http://www.rozhlas.cz/audio/download/ddur_maxogg.m3u


No many media players will play the FLAC stream.

Foobar will in Windows and the VLC player on most platforms.

Can you reliably differentiate between the FLAC stream and the 256 OGG?

Note:

The technical stuff on how D-Dur pushes its stuff out in so many formats is here

http://www.cesnet.cz/doc/techzpravy/2008/using-flac-encoding/

 

It has validity, it just isn't 100% conclusive but then no test is.

Place a test such as this alongside the usual alternative - playing AAC and lossless to people (sighted) and asking for their comments and votes.
You'll get an overwhelming ability for the listeners to spot the AAC and describle it as inferior.

I know which I'd regard as more valid.

 

Well if it is valid, then most people (by a big margin) cannot tell lossless from kosher. Personally, I have no problem with that.

But it is not valid in any way as I am sure any psychologist, or scientist, or market researcher will tell you.

 

I suppose you are right that this test has no formal scientific validity (- which I am not sure whether anyone has claimed?) On the other hand, it is the only test of its kind (that I know of) with such a wide range of programme material and effective concealment of the filetypes until the final disclosure. As such, I personally find it very educating as well as indicative.

Thom

 

Yes

unfortunately I've chosen only one track (Nimrod) and for me who listened for the first time a AAC 256 it was something like answering the question:
"Do you prefer to have your right arm and leg amputated or your left arm and leg?"

Now it is certainly interesting for me hearing the differences (if any detectable) when using better recording quality lossless material.