Double blind tests

i was looking at the line array at the Ben Folds concert..

funny thing was, so was he, he stopped half way through the second song, apologised for the sound, made some signals to the sound guys, they tweaked it, and it sounded much better.

you don't see that very often...

 

I have never seen it . . . . . more's the pity

 

I thought the answer to that is pretty obvious! The results from such tests might let them know that the majority of listeners entering their shop do not prefer the more expensive system, so it would be waste of their money.

Actually as I pointed out it very well could. Such a person may not have even considered that a more affordable system could offer equal, if not preferable sound quality to one costing 10x as much.

A person can not listen to every piece of kit there is, so if they know that the majority of listeners proffered 'A' when personal bias (besides sound preference) has been removed, it is a damn good starting point.

Of course it is not the answer all. What it can provide is a strong pointer in the direction for what the majority of listeners prefer. That information is very useful in all manner of applications.

Why do you find debate of blind testing (and I assume the blind testing itself?) to be about 'invading your pleasures and enjoyment'? The information gained from it can help you find greater pleasure for less money a lot of the time!

 

I'll pass on the rest of your posting

I have done blind testing myself a fair number of times now, both individually and part of listening panels so I am no stranger to it. It was done for a definate purpose and certainly not for my own choice of hi-fi components. The reasons why are pretty obvious; Firstly, being on a panel makes you realise just how different people's perceptions really are, so what might please me easily revolts somebody else, so what real use is that information to a third party? I don't sit listening to my hi-fi with a big cloth draped over it, a blindfold over my eyes either, so what purpose does that serve me when I'm chilling out in the evening listening to my favourite tunes? I have no trouble at all switching the system off, introducing a new component and then making an assessment of what if any changes there are to the sound. It's the way I have done it for the last 35 years or more whereas the neurosis of double blind listening tests is a recent development spawned by keyboard cowboys on the internet, not because they have a valid point worth considering, but because nobody is sensible enough to argue the merits of it.

The remotest chance at all of making a blind listening test useful is if you have every product on the market in front of you and then pick the ones that please you the most after hearing them all. It will take a very long time I can assure you

 

Indeed.

I just don't see why anyone should take exception to such tests, unless they feel the need to defend their purchase decisions. But I haven't seen anyone being told to sell their kit and buy 'xyz' because the test said so. Let alone feel their pleasure and enjoyment is being invaded by the tests!

 

It's not the tests as such that bother me Tenson don't get me wrong, it's the people that get all evangelical about them as the absolute true path to enlightenment, supplanting the sound judgements that everyday folk are capable of making on their own. If they themselves rely on it and it gives the results that are right for them, then live and let live is my motto. I also say that if anyone else buys for example a new cable that floats their boat, then what gives anyone the right to barge into their pleasures and say it's the result of delusion or that they should have conducted a double blind test before purchasing. Do you see my point now?

 

I didn't notice anyone doing that. You can come to your own conclusions from the results yourself, or choose to ignore them if you think they are flawed.

What I did notice is that you were the first person to barge in and voice why such tests are so flawed and useless. Significant effort was put in to the tests and 68 people choose to take part in it as part of their hobby. To be honest, from someone that keeps saying how negative everyone else is, it seems rather hypocritical.

 

I didn't barge in anywhere, merely highlighted that ABX tests fail time, after time, after time, which they do. Either it's because the 'differences' are non-existant or there might be a possibility the ABX test has an inherent flaw that perpetually gives inconsistent results when overwhelming empirical evidence suggests otherwise. If youu are going to be objective about it then be objective or not at all.

Either you are going to score points off me or have an intelligent debate, the choice is yours. Yes I can be forthright and if you perceive that as arrogance then I do apologise bcause it's not intended to be.

 

In what way does the ABX test 'fail'? In this particular case, 63% of people could hear a difference between the systems and displayed a preference.

 

On Music, Recording and Replay

Folks,

I know this is a bit off topic, but I think we are debating the fleas on the dog and not the dog with that ABX Mudlark.

We MUST understand that the very act of recording a musical performance introduces massive "Distortion". What most people seek in higher fidelity audio is something that bridges as far as possible the gap between reality and recording.

And the fact is that at times systems that in classical electrical terms perform fairly poorly (extreme example - low powered SE Tube Amp's on Lowther Horns) seem to manage better to bridge that gap between reality and recording.

So, the very fact of the extreme imperfection in the recording/replay process gives rise to differences in "taste", meaning different people will accept different imperfections and react strongly to others.

So, to some people extended low frequency response is not essential, but midrange transparency and micro, as well macro dynamic performance is, yet others absolutely need to have subterranen bass to "feel" the music and if that is absent all the rest is no go.

So, I'll restate my earlier contention. The test showed that the two systems sounded different enough to show a relaible detetcion of it in around two thirds out of a reasonably large group, in statistical terms the result is quite significant and has substantial statistical power.

A second result from the test is that a significant number of people preferred the "cheap" system. Without more data we cannot tell WHY they preferred the cheap system, only that they did.

Does that mean the around 1/3rd of the people who preferred the expensive system should now receive a court order to only listen to the cheap system from now on? Should we draw the conclusion that all anyone should need and want is cheap Sony DVD Player and a Behringer ProSumer Amplifier?

I think not. What it does highlight is the need to listen carefully before any purchase and to make sure that one personally LIKES what one hears, regardless of cost.

Ciao T

 

Good post!

I'd totally agree with it but there are some people, several of whom post here, who wouldn't accept that as a 'fact' at all.

 

Well I'm one of them.

I don't think that SETs and Lowthers get particularly close to the original recording. They do introduce lots of 2nd harmonic distortion, lashings of IMD and have, to put it mildly, a rather wayward frequency response.

If you think that effectively bridges the gap between hi-fi and the original recording fine, but I'd suggest that it simply presents a coloured and tailored verson of the original that some prefer.

<yes i have run Lowthers>

 

Hi Rob, I'd also agree with you much of the time. there are very few, ime, good Lowther implementations and the only drivers they made which were really interesting were PM4A's imho. However hearing PM4A's with a simple, low powered amp can give a realism to certain instruments (piano for example) that few (if any) other amp/speaker combinations can match. Of course they have other shortcomings & won't play all types of music well, but I don't think that was the point that Thorsten was making. I certainly wouldn't recommend it as an ultimate way to go either. But for some people it is, I can understand why, & I respect their choice. You like the H2's. I regard those as equally flawed but for wholly different reasons.

(former Lowther Audiovector user) nearly recovered.

 

<wee bit off topic>

I agree that Lowthers can hit the spot but they only 'work' on a very limited range of music IMO. Being point source helps and is probably their biggest attribute. The pair I owned which had EX2 drivers could sound stunning on R4 plays and very simple material.
H2s are strange beasts. Many speakers have been through my place and they've stomped on the lot but I've recently spent a few weeks using a pair of tiny Rega R1s. Yesterday I popped the H2s back into the system and two things hit me - wow the scale is back and secondly that they are more coloured in the mid than I remember. We tune these things out over time, so long as the basic ability of the speaker can carry them. Even Linn Kans sound pretty flat if you play them long enough

 

I dredge out of my memory a Stereophile-organised blind test wherein "audiophiles" did what they thought was a same or different test on amplifiers. Only they were presented with music from the same amp every time. Nevertheless 68% claimed to have heard a difference.

This means that only the 32% who reported no difference could be validly employed in any subsequent DBT as to preferences once a different amp were substituted. And this may be an over-estimate - some of the 32% might have been insufficiently sensitive to hear any differences whatsoever. Like me when it comes to i/c cables.

So if you can't identify an amp with itself you don't belong in any DBT and nothing the self-appointed statisticians do in massaging the results can validate or invalidate your findings.

 

I think these sorts of test should be left in the medical labs ....it shows a distinct lack of imagination if we [the hifi community ] can't come up with our own testing regimes that will stand independant scrutiny.

Its clear these tests are " not fit for purpose" to use the current buz phrase. If you can't tell if it the same amp as the test is run then its clear there is a major failing in the test regime...ie the listens can't be relied upon to provide a consistent result.

It can't be beyond or wit to come up with an alternative ...can it ?

 

There will never be a resolution to the agreement / disagreement about ABX testing etc. Significance (statistically speaking) is very important - see 3D sonics eminently sensible post above.
Zanash, whilst not disagreeing with you I worry when you state "If you can't tell if it the same amp as the test is run then its clear there is a major failing in the test regime...ie the listens can't be relied upon to provide a consistent result..." - I would have thought that it would be equally logical to say 'If you can't tell if it the same amp as the test is run then its clear there is .......... little between them sonically, despite many other (design / cost / electronic etc) differences' ???

 

Finding an alternative is going to be about as easy as swimming the Atlantic with both arms tied behind your back.

Firstly, in truth there isn't a huge chasm of 'differences' between the budget end and the high end of components nowadays that there once was, so we are talking minute subtleties that are more down to the listener's own preferences and the ability to discriminate rather than having big identifiable obvious separation. Badge kudos also has a very large part to play in that scenario, let's be quite blunt about it.

Secondly, coupled with these factors above we have an almost unlimited variability of human hearing perceptions whereby some can almost hear a flea's flatulence at 10 paces whereas others are little fazed by a foghorn going off next to them, with every other combination and permutation in between. The bind ABX test calls for a black and white answer to the black and white question with no latitude whatsoever for those endless variables, hence the results are always compromised. Juggling the statistical numbers from those results looking for confidence levels is in my view an excuse to number juggle rather than making a realistic stab at the truth.

Finally, I say that the majority of folks that claim to have an ear for music don't actually have it. Yes they can toe-tap along to a good rhythmn like the best of them, but ask them to translate that into a set of words describing each part of the musical performance along with subtleties like timing, imaging, soundstage, micro and macro dynamics, plus a whole raft of other complex sounds and the words are not forthcoming because in essence this is a foreign language. Each of these elements needs to be assimilated by the mind to a high degree before moving on to accepting there are tangible differences between components. The real indicator I think is the folks that need instant switching between component changes to identify that any change at all has taken place. I don't class myself as having "golden ears" by any means, but I can highlight differences not in terms of seconds and minutes, but weeks and months apart, so when someone tells me as fact my audible memory span is around 3 seconds then quite naturally I am inclined to disagree. You have to understand music before you can fully understand hi-fi.

 

so if we can't think of a test .....how about reverting to the age old method of comparison.

A standard .....lets say for example an ic cable is made ...ie a twisted pair 0.75m long made from 1mm copper [standard wire wrap] using ptfe as a dielectric using sy series neutrics rca plugs .....

you could then test any ic against it and say
does it sound different yes/no
if yes is it better or worse
if better you can score it 1-10 on a subjective scale
if worse score the same way.

this is just an idea ..so don't jump on me for postulating .

the same could be done with amps ..pick a company like quad that rarely changes the design and have that as the "standard"..

 

Why would you want to do any of those things? I'm jumping on you Pete for having illogical thought processes for seeking a solution to a problem that doesn't actually exist.

Someone will try/buy an interconnect then it will either float their boat or not as the case may be. That's the beginning, middle and end of the argument. Pass that same cable on though to someone else and what are the chances of that same result occuring again? Some may not hear any difference at all with whatever cable no matter what it's made from. On that basis then, what point is there for 10, 100, or even a thousand golden eared listeners blind testing away until the sun goes cold?

Who always insists on double blind testing cables anyway? The very same people who won't try or buy a better sounding cable Good on them if that's their decision. Those that can clearly hear the differences and are very happy with their purchase never squawk about it because fooling around with cables is part of their pleasure in the hobby. Not just cables, but box swapping too