DBT Report

[PeteH]

Originally posted by merlin

I find that a poor analogy I'm afraid. When tossing the coin, the result is purely down to chance unless using a rigged coin. If you are talking about guessing heads or tails, again you are talking about pure guesswork. When the coin is in the air, you are not asking the participent to declare heads or tails by using any of the five senses, purely by guessing or using some sixth sense.

By using this analogy, you are inferring that the listeners in question, when asked to identify something using one of their senses, rely instead on pure guesswork. This would statistically not result in an 83% success rate. What you are implying is that the listeners are not using their ears.

He's not quite implying that the listeners aren't using their ears, just that they may as well not be as it won't affect their ability to distinguish the two cables.

Originally posted by merlin

What you are saying is that if a difference is detected initially, keep testing long enough and it is likely that the results will even themselves out.

No. If a difference is detected, keep testing long enough and the identification success rate will hold steady (obviously not allowing for fatigue etc. which needs to be accounted for). If the correct identification is made by chance, then keep testing long enough and the results will even themselves out.

 

[merlin]

Originally posted by michaelab
If I toss a coin 7 times and get 6 heads the result doesn't mean anything as it's quite likely to happen by chance alone. There's a relatively high probability (much higher than 5%) that in any given 7 coin throws you will get 6 heads (or 6 tails).


Michael.

I don't think there is actually Michael. Anyone care to do the maths?

There is I believe 7 chances in 154 that you would get six correct identifications, which makes the probability of pure luck just over 4% if my maths is OK

Still, not good enough eh, keep testing till we get the result we want

 

[wolfgang]

Merlin, Forget about the maths for a while. If you throw a coin and you to get 6 heads in a row, is that amazing? Chance of that happening is 2x2x2x2x2x2=64. That is 1 in 64 chance. Some may think it is impressive numerically but what does your common sense tell you?

Why not ask 50 guys from the Zerogain forum. The inclusion to the test is only those who think they could hear differences between cables are allowed It is pointless testing anyone who never could hear any differences btw cables. Use these other group for something more easy like comparing amplifiers.

They all listen as long as they like to A or B. For as many times as they like. Then X is presented. Each person have only one chance to guess what is it. Why do they need more then one guess since cables are so obviously different to these people. If about half people get is right I will not be impressed. Since I will expect approx half will get it right if they all simply guess.

If on the other hand 40 or more people get it correct I would be impress. Between 26 to 39 not conclusive. However, you could tell me why you think otherwise.

If the objectivists hypothesis is true my believe is even with Single Blind Test less then 10% or 5 people will get it right.

 

[merlin]

Originally posted by wolfgang
Merlin, Forget about the maths for a while. If you throw a coin and you to get 6 heads in a row, is that amazing? Chance of that happening is 2x2x2x2x2x2=64. That is 1 in 64 chance. Some may think it is impressive numerically but what does your common sense tell you?

Wolfgang,

But why forget the maths? Your example in relation to the quoted stats is in fact incorrect, but still the probability is less than the 5% Michael stated was acceptable for you guys. So why are the results unimportant?

Now you are saying, don't use maths and results, but common sense. Well again common sense would suggest if 6 out of 7 identified a difference, then the difference existed. Anyone out there with a different view and happy to put a 4% chance down to luck?

For me, this debate has ended up quite positive. Surely you can apreciate that the whole ABX/DBT process, whilst I have no doubt valuable to a degree, is as flawed in practice as everything else?

I could identify a cable swap under DBT but the objectivists would claim it was luck. I could do it again 6 times, and again it would be down to chance ( less than a 1% chance admittedly). After that, I can rightfully claim other factors such as listener fatigue distort the end results. And of course, just as importantly, the interpretation of those results is a wholly subjective procedure, as has been clearly demonstrated over the past few pages.

Maybe, just maybe, this all helps to explain why common ground is so very hard to find.

 

[wolfgang]

Yes. You are correct to point out it will be difficult. Everyone will have a different level before they accept a test has reach a significant level. For you if even one person seems to have manage to identify between 2 cables under some form of DBT 6 out of 7 times as significant prove. My calculation is assuming he is comparing 2 cables. Every time he is ask to select between 2 cables he has a 1 in 2 chance of selecting the correct cable. The second time he does it will be still 1 in 2 chance. If he manages to get it right 6 times in a row then the probability is 1 in 64. In percentage it is 1.6 % which I agree it sounds like a small number. Michaelab is trying to express the idea in another way. I think he is using significant test. Do correct me if I am wrong.

Unfortunately, looking at it from my side of the fence I assume he could not really hear actual differences and every selection he makes each time is purely a guess. In other words, it is no more impressive then like tossing a 2 sided coin. We all could at one time or another manages to throw 6 heads in a row. Therefore that person result for 6 correct answers is not a big deal looking at it this way.

We do have to think carefully when trying to understand statistics. The numbers can mislead. If a doctor tells you a person DNA was found at a crime scene and the probability for some one else with this type of DNA 'fingerprinting' is 1 in a million (a guess) we will quickly assume that person must be the criminal. However, if told there could be at least 6 other person in the whole of UK who has the same DNA then we may not be so sure.

 

[PeteH]

Originally posted by merlin
I don't think there is actually Michael. Anyone care to do the maths?

There is I believe 7 chances in 154 that you would get six correct identifications, which makes the probability of pure luck just over 4% if my maths is OK

Still, not good enough eh, keep testing till we get the result we want

The chance of getting 6/7 with a 50% chance of success each time is 5.47% (7*0.5^7). However, if we take this statistic as a given, we have to compare that probability against the probability that there actually were real differences between the cables. What's that? 0.01%?

I'm still interested in knowing where this "6/7" figure magically came from - I certainly wouldn't trust Harley to interpret any experimental results...

 

[wolfgang]

Thanks PeteH.

 

[merlin]

Originally posted by PeteH
However, if we take this statistic as a given, we have to compare that probability against the probability that there actually were real differences between the cables. What's that? 0.01%?

Complete waste of time having dialogue with you then isn't it

Your mind is made up. If you only had an idea as to how ignorant that comment will appear to many of us, you might have thought twice about posting.

 

[PeteH]

Heh, quite the wind-up merchant, aren't you You're just poking us to see what we'll do now

 

[garyi]

Lets be honest if you are all bleating on about this after so many pages, bringing it down to statistical probability then the chances are the whole cable debate is a load of bollocks.

The problem for me and many others is the claimed 'massive differences', the 'night and day' changes a cable apparently makes. It dosn't, infact its so subtle that there is a clear divide as to if there is a change at all.

Push this to a car for instance. the price differences between cables could easily be applied to a ford KA and a Porshe. However no one could argue that the porshe was better in every way, more over that conclusion could be tested and proved.

Its simply can't with cables because at most the changes are very very subtle.

So those that claim to hear differences stop spoating bullshit that the difference is night and day.

Those that don't hear the differences, let those that do get on with it. Its there moeny after all.

Every one is happy and we can all get away from this terribly boring subject, its a f**king piece of wire for f**ks sake.

 

[merlin]

Originally posted by PeteH
Heh, quite the wind-up merchant, aren't you

Errr No.

I quoted some statistics from a test carried out by your very own messiah. These showed a 95% success rate. Somehow, with the help of your personal bias, you managed to distort that figure to just 0.01% to suit your preconceptions and arguement.

So it would appear that it is you that is distorting the figures in an attempt to "wind up" others.

 

[Ultrasonic]

Havent realy got anything constructive to say but all i can say is the original post of this thread was quite interesting and i cant see why its turned into such a mass-debate

I tried stay away from the heated debate on cables but i must say even with el cheapo cable that i have changed around over the years i can hear differences. something as simple as a change from QED original and silver anniversary has some readily apparant improvements

Michael seems to be especially sensitive to this topic but to be fair, at £40/m? the Dynaudio OCOS cable you use is hardly cheap! Thats probably why you cant hear much, if any improvements when switching to other cables.I believe up to a certain price point improvements with cables barely justify their elevated price tags, all IMHO of course and i must confess i have very little experience with cables above £150.


Maybe its fair to say that when cable prices delves into megabucks it may be worth spending the cash on better electronics as im sure £30K of speakers, amp or cd player added to a system using adeqaute cables will reap more sonic benefits than adding those £30K siltech cables to cheaper electronics

Hmm enough of my rambling


Ultra

 

[PeteH]

Originally posted by merlin
Errr No.

I quoted some statistics from a test carried out by your very own messiah. These showed a 95% success rate. Somehow, with the help of your personal bias, you managed to distort that figure to just 0.01% to suit your preconceptions and arguement.

So it would appear that it is you that is distorting the figures in an attempt to "wind up" others.

Note the smiley after the 0.01% comment. I guess I should maybe have written =====> *JOKE* <===== to avoid injuring those of touchy sensibilities

The point I was trying to make is that statistics are a minefield in any case, but particularly when you're trying to apply them retrospectively, ie if something has already happened then simple consideration of the chance of it happening in the first place isn't particularly helpful. Wolfgang's point about the misuse of DNA evidence in hundreds of criminal cases through abuse of the statistics is relevant here.

I'd also suggest, with respect, that your adversarial and aggressive posting style isn't really conducive to furthering the discussion. As I've said though I suspect you're just having a laugh trying to poke us and make us go

 

[wadia-miester]

Originally posted by PeteH

I'd also suggest, with respect, that your adversarial and aggressive posting style isn't really conducive to furthering the discussion. As I've said though I suspect you're just having a laugh trying to poke us and make us go

Pete,
to be fair sir, you guys are doing a damn fine job on your own

 

[PeteH]

Originally posted by wadia-miester
Pete,
to be fair sir, you guys are doing a damn fine job on your own

Hey, that's what we're for

 

[merlin]

Originally posted by PeteH
The point I was trying to make is that statistics are a minefield in any case, but particularly when you're trying to apply them retrospectively, ie if something has already happened then simple consideration of the chance of it happening in the first place isn't particularly helpful.

So please let me get this straight. Statistics are in the case of DBT generally the results of a test, that test designed to create statistics that can be used to prove or disprove the efficacy of a product.

How can you create those statistics in anyway other than retrospective to the DBT. The test procedure produced say six positives and one negative, the odds of this happening are c.5%. The maths existed prior to the test, but are implemented retrospective to the tests to produce what you would claim are meaningful statistics.

I would agree that statistics can be interpreted in many ways and are indeed a minefield. But you and others are advocating DBT/ABX procedures that create statistics to support an arguement. Retrospectively of course.

Now we have already argued that the procedure is flawed, given the potential for outside influences such as fatigue to cloud the result. We now find you saying that statistics are misleading and should not be used retrospectively.

Please continue to enlighten me as to why I should consider your methodology to be superior to the traditional listening test. I don't consider myself to be agressive BTW. Direct and to the point yes, but I imagine you would find this to resemble agression if you feel threatened by any of the points raised. If that is the case, may I apologise for having the temerity to question your thinking on the matter.

 

[PeteH]

Originally posted by merlin
If you only had an idea as to how ignorant that comment will appear to many of us, you might have thought twice about posting.

Maybe you could credit Zerogain members with the ability to get by without you having to tell them what to think.

Look, the 0.01% thing was a throwaway comment, a "joke" if you will, in case I haven't made that clear enough yet. To say again what I said in my last post, given a set of statistical data all you can do is interpret them in the light of what you already know. Looking at the data, you have to decide what it tells you by thinking what the most likely way is that it could have come about. This is the relevance of the DNA testing at crime scenes - when an individual's DNA matched that found at the crime scene the jury was frequently dazzled with the millions-to-one odds of that happening by chance and hence a conviction was secured. As Wolfgang points out though, the same evidence could be used to convict several thousand people worldwide.

So "given" that we have this 6/7 figure - and I'm still not convinced as to where that's come from, but let's continue for the sake of argument - we have to ask ourselves, what does it mean? Given the weight of evidence I'd say it came about by chance, as to my knowledge any and all blind tests ever conducted properly prior to this have come back with null results, so in itself this is nowhere near enough to prove any kind of correlation, as the weight of evidence from prior results suggests that the probability of a difference existing between cables is smaller than the probability that this result came back by chance. However, if a series of tests of this nature came back with consistent results, well, the odds of that happening purely by chance fall off rapidly, and we'd have to think again. In that case we'd effectively have something more like 60/70 or 600/700 as Wolfgang suggests, and that'd perhaps be strong enough evidence to throw previous results into question.

 

[bottleneck]

Interestingly (I think) this brings us full circle.

If we take it for granted that -

DBT can provide unbiased information when:

1) A sample large enough to provide statistical significance is used (500 + ? whats the benchmark?)

2) Large bodies of people are used to ensure even handedness

3) Hearing tests need to be used to ensure the audience can detect differences

4) Timing and duration has to be brief and over many weeks to ensure factors like tiredness dont come into play

5) Trained staff need to be used to ensure that visual, verbal and other cues dont give any hints.

6) To be truly D.B., the tester must not know the differences - therefore the products must look visually identical, other testing facilities must also be put into place


Have I missed anything out????

My conclusion is, that for DBT to be of any relevance to any single one of us, would mean a test of proportion that none of us would likely be prepared to conduct.

Therefore, the only logical conclusion is that DBT is of no use whatsoever in an individual testing equipment for their own use.

Wolfgangs comment that DBT should be used (at least by the larger) manufacturers is valid though, as they have the facilities and budgets to run that kind of test.

 

[merlin]

Originally posted by PeteH
Maybe you could credit Zerogain members with the ability to get by without you having to tell them what to think.

Pete. I do. They PM me

Chris,

A perfect summation if I may say so, my conclusions were very similar. It is in fact completely impractical to use DBT to prove or disprove our findings as humble audiophiles.

If anyone is really interested in this subject, see if you can get to the end of this .

Nothing really to add to what Chris put so succinctly, but do remember, if you choose to DBT your next component/cable choice rather than using the traditional method, be prepared to perform at least 100 comparisons with the help of a few mates before you can have any confidence in your findings.

 

[PeteH]

Originally posted by bottleneck

Therefore, the only logical conclusion is that DBT is of no use whatsoever in an individual testing equipment for their own use.

Wolfgangs comment that DBT should be used (at least by the larger) manufacturers is valid though, as they have the facilities and budgets to run that kind of test.

For the purposes of settling the "big questions" it's likely to be difficult for an individual to conduct a test sufficiently tightly controlled and of large enough scale - and somehow, despite the obvious marketing coup, no cable manufacturer has ever quite got round to putting one on. For the purposes of deciding which gear to buy for yourself, well, I think Lawrie's method is quite sensible if you want to make purchases purely on sonic grounds.

OTOH though if Z/G did get together and produce a set of results under well controlled conditions that did show differences between audiophile cables I'd certainly be up for writing it up for publication in the JAES