Power Cable Test – Volunteers Needed

My point here Mosfet was simple. One or two people seemed to want to attach some importance to the equality of preferences shown in the test. Applying similar logic, it is easy for me to counter the arguement, to put forward an opoosing conclusion.

What we now have to two opposing camps trying to spin these meaningless results to suit their agenda. Both points of view are perfectly valid and neither has been disproved to my knowledge. But spin from either party is purely assumption based on meaningless test results.

So what was the point of this test bearing in mind that you knew it's results would be meaningless even prior to performing it?

 

A modest proposal for the preventing of Blind listening tests being meaningless

A modest proposal

for the preventing of Blind listening tests being meaningless

It is no doubt a melancholy object to the educated and genteele reader to notice the deplorable state of debate and listening tests on subjects of "fringe audio", which of course is mirorred in many other fields and will likely agree that this state is best not perpetuated.

We have to say that it pains us to simply dismiss with a wave of the hand the results of hard work (as we have on occasion conducted blind tests ourselves we are well aware of the work involved), yet it seems no other option remains as long as experimenters insist on applying methodology that is substantiatively flawed and inapropriate.

Thusly, this little tract is meant for the edification and eludication of those insisting on certain types of tests, so that these tests can be implemented in a manner and fashion that allows the results to be of use.

1) Let the seeing lead the blind

If you wish for a meaningfull blind test you must assure that the test is blind. This can be done in many ways, but no matter how it is done, the propensity of the human component of the test to percieve what he or she is convinced of being the case (placebo effect) must be accounted.

Just as the "cable sceptic" will not hear any differences, even if significant while he or she remains convinced that he or she is listening "to cables" and will hear the same difference just fine if they are presented as being in an area where he or she accepts audible differences (we have carried out such tests ourselfs) so the "cable believer" will hear a difference, even if non whatsoever exists as long as while he or she remains convinced that he or she is listening "to cables" and will hear the absence of a difference just fine if they are presented as being in an area where he or she is convinced audible differences are absent.

Thus as long as the subjects may be reasonably expected to have a strong emotional reaction to the subject of the test they must be blinded not just to identity of the various devices under test but equally to the actual nature of the test, in order to TRUELY ELIMINATE prejudice as source of results, instead of making it certain results are dictated entierly by prejudice.

2) Test the test and tester

If I where to present evidnece on the presence or absence of a given effect and where then found to have made my tests using an apparatus that had never been shown to reliably detect anything known to be detectable and had not been calibrated against known quantities I should not be surprised to have the scientific community to question severely my results, if not throwing them out completely unconsidered.

Hence, if an experimenter wishes to cary out tests whose results can be used for any purpose, he or she MUST first test the actual testsetup using known phenomenae. If one expects the phenomenae investigated to be "marginal" in nature, then in fact one may be constrained to use stimuli in the test which are known to be effective in a significant but marginal manner (eg. large testsets reveal a modest audibility, near the limits) and only retain test setups that show AT LEAST the same if not better ability to detect marginal phenomenae as where shown previously.

Failure to show that the actua testsetup used is capable of reliably detecting know effective stimuli reliably and conclusively precludes any use of the test to suggest that the stimuli being tested is ineffective, it is capable only to illustrate the presence of a stimulas that in the context of the testsetup is reliably detectable.

But as nothing has been done to establish the ability of the test setup to discriminate known stimuli no reference for comparison exists to judge if the absence of reliable detetcion is due to the testsetup itself or because the stimulus is below the detetction threshold, as said threshold is UNKNOWN.

3) Test the statistics

There are lies, damn lies, statements by the British or US Government on WOMD in Iraq, Huricanes and other items and then there are statistics.

In fact, we do not trust any ststistics we have not faked ourself.

What are we trying to say? Statistics is an interresting field and statistical analysis is such too. In the end we can only determine one thing using statistics, namely how likely a given result is to have resulted from random effects or not.

In blind listening test just as in other tests experimenters seem to focus invariably on the type a statistical error, that is to say the degree of certainty that "null hyphothesis" (that is no significant stimulus is present) has not been recjected in error. In plain english, they want to be sure that if they say "a difference was observed" that it was not merely a fluke.

What is being missed in this case is that there is an opposite side to this, namely a risk that in fact a weak stimulus existed but that the "null hyphothesis" was not rejected.

Both risks decrease as the datasets become larger and the stimulus stronger, EQUALLY, they increase as datasets become smaller and stmuli weaker. If we hold one of the risks to a fixed factor, then the weaker the stimulus and the smaller the dataset the greater the other risk.

If we evaluate the risk of both errors in small scale listening for weak stimuli (I would class the differences between cables as "weak) we find that the brainless application of a .05 significance (in other words that is a 5 in 100 chance that the "null hyphothesis" is rejected in error) leads to a rather high risk of NOT rejecting the "null hyphothesis" in error.

If for example we have a stimulus that is reliably detectable in a listening test only 60% of the time (I would class cables in this region, especially if the tests are done in unfamiliar acoustics with unfamilar material, never mind material that is not selected for allowing easy discrimination, plus acounting for lengthy tests and attention span limits, aural fatigue etc.) and we have in total 16 Datapoints and we apply a .05 Significance to the data in order to reject the "null hyphothesis", what is the risk that if we do not reject the "null hyphothesis" we do so in error? (see the answer below).

The experimenter(s) should evaluate this type b statistical error risk, consider if they need to alter their test methode to reduce type b errors AND when stating their rejection or not of the "null hyphothesis" should state not only the type a error risk (significance) but also the type b error risk.

All the above will make sure that tests conducted can be meaningfull and at least can be placed in context. Failure to allow the test data to be placed in a context simply means that meaingless data has been generated.

Ciao T

PS: Here the solution to our little statistical riddle, if we apply to a 16 trial test a significance of .05 (that is out of 100 tests only 5 would reject the null hypothesis in absence of a stimulus) we require 12 correct identifications to reject the null hypothesis.

But for a stimulus only reliably detectable 60% of the time this means that our risk of not rejecting the "null hyphothesis" despite an audible stimulus being present is .8334 or in other words that only 17 tests out of a 100 would reject the "null hypothesis" correctly.

Still in other words, if your risk of incorrectly concluding an audible difference between the tested items is 5% then your "risk" of actually finding an audible difference that exists is only 17%, given the small dataset.

A chance of detetecting a real difference of only 17% may well be conscrued as a near certainty that small audible differences would not be detected even if present.

 

Thank you for your conclusion Thorsten.

Much preferable to the smugness conceit of “Amusing.â€Â

From the conclusions I posted with the test results:

To properly disprove the hypothesis was always well outside of what could feasibly be achieved - however this was the point to aim for.

What has been achieved is a set of test results that can be considered as you wish.

 

Although in fairness, following the debate both here and on Wigwam, is there anyone who doesn't find it just a teensy bit amusing?

It puts me in mind of Dr. Zeuss with his Sneetches on the Beaches (half with stars on the bellies and the other half without) ...

Now the Cable Detecters could hear difference in sound
While with Non-Detecters not a difference was found.
And the Cable Detecters all thought they were smarties
and Non-Detecters wouldn't go to their hi-fi bake parties ...

Well, I'm sure you see what I mean.
Anyway, sorry for the interruption. Let the battle resume.

Footnote: It should be pointed out that with two young children and reading stories to them every night, practically everything puts me in mind of Dr Zeuss. Is that sad or what? No, what the hey - Zeuss Rules!

 

Hi,

Not so much a conclusion, but a generic position paper.

It was not meant smug or full of conceit.

By extraction I am saxon and we have retained the rather frank directness of other germanic tribes well, I think the Anglo Saxons got to spend too much time with the french so they allways try to secondguess simple statements that mean nothing more than said/written.

In which case I find them amusing and meaningless still, without conceit or smugness, rather with certain sadness. As said, I dislike telling people that lot of hard work amounted to exactly a hill of beans after the weevils got through with them.

Ciao T

 

Which is?

Surely if the aftermarket power cables offer to the average hi-fi enthusiast an audible improvement, as the various superlatives suggest they do, this would have been borne out in every instance of the question “which cable sounded better in your judgement?†being asked?

Or at the least the majority of instances in which this question was asked? The standard mains IEC cable had “no right†to perform anywhere near as well as it did if those superlatives are indeed justified

Irrespective of statistical criticisms of the test results, the results remain as presented.

The arguments now seem to be resolving down to improvements that are only audible under certain conditions and with specific caveats applied. I'm accepting of this as a possibility (but that's all).

To do something the mainstream hi-fi magazines are failing to do SM; a genuine attempt to look for objective information.

The results are meaningless in your estimation - more or less meaningless than a sighted review involving one reviewer out of interest?

I was of open mind prior to the test and try to remain of such afterwards.

 

I've tried to be clear Thorsten. But once more.

To properly disprove the hypothesis was always well outside of what could feasibly be achieved - however this was the point to aim for.

Does this mean the results are both “meaningless†and “amusing†given the other objective of the test?

and to do something the hi-fi magazines are failing to do.

Same question. More or less “meaningless†than a sighted review one might read from one reviewer?

 

For an additional control say you are swapping sometimes and then keep the cable the same - that will characterise the effect of predisposition to imagining hearing a difference just because you think a change has occured.

 

That was also included in the test Nick

 

Whoops did I miss those results? Did people also hear a difference in that case?

 

Would anyone care to comment on what I wrote? Is there ANYONE out there who is prepared to do a large number of blind listenning tests with a cable and see if they get it right over 70% of the time? Taking something completely mentally priced or top of the range so it's the "best available" e.g. Nordosts top Valhalla mains cable
(not for contentions sake but there are some blindingly good reviews)

Until someone does this I'm not interested in custom mains cables... volunteers?

Thanks, Tim

 

Well I tried half a dozen cables in a sighted test on myself and two others. Comparing nordost, kimber etc with standard kettle leads. It all sounded the same.

 

Doest it matter? Seriously, if it works for you then whats the problem? I honestly dont think theres a conclusion to this debate, no matter what tests you do. Its all about getting the most from the music and however thats achieved is down to the individual.

Regarding the actual event....those pics on wigwam are top notch; grown men sat round a bart simpson towel with clipboards is a sight to behold!!

 

Acutally I think it does matter and it matters a great deal. It'll save me a vast amount of cash for a start. In fact I may make some money selling my current cables. I have a limited expenditure and would rather not get conned.

 

Yes I think it does do too - especially as I beleive it is hokum. Plus progress in hifi both personal and technologicial is based on scientific (or at least logical) method - anything else is voodoo and just leads to stagnation. As has arguably happened anyway. Which is why people end up paying a $1000 for a record clamp and people just accept it.

 

Tim,

You can always try a kimber/RA cable from Russ Andrews as they come with a 60 day money back guarantee. That should be plenty of time to try for yourself, in your own system and decide if it is worth the expenditure.

I have several RA powercords and I think these are ok but I wouldn't go for very expensive ones.

I have a Nordost Shiva and don't think it makes any difference at all and not worth the £100 or so extra over the RA cables.

 

But why do you need blind tests to tell YOU if cables work or not As pointed out most genuine companies offer money back shemes on their products plus theres plenty of listening offers on here. If you cant here a difference fair enough but if you can and you think its worth the asking price then whats the problem??

Put it this way, if 99/100 people said they did make a difference but your the one who didnt think they make a difference would you still buy them??

I just think its a futile debate TBH.

 

At the end of the day if you buy something and you are happy with it then it was worth the money.

We tend to over complicate this hobby sometimes.

Put the cables in give it a couple of weeks, if good keep them in and enjoy them, if not send them back and get the money back, spend it on records/cds and just enjoy your system.

Easy peasy.

 

Hi,

My point exactly. If to provide any form of proof that the null hypothesis was false (or a [statisticaly] SIGNIFICANT indication of lack of reason to reject the null hypothesis) was not possible due to the arrangement of the test, why bother doing it?

If I carry out a laborius task only to find myself at the end of it exactly where I started it is normally considered amusing. It certainly features in many cartoons (Roadrunner & Coyote anyone?), slapstic movies of the silent film era and in modern situation comedies as reason to break out into uncontrolled laughter. So yes, in that definition it fits "amusing".



The results are meaningless and completely so because no attempt was made to establish if the testsetup was able to resolve known audible differences prior to attempting to judge highly disputed ones, because the testsubjects where highly likely to hold strong convitions and views on the subject (which disregardless of the actual nature of their views adds a highly randomising factor) and because statistically speaker there was never enough data in case of a lack of audibility to support any conclusion.

To me that is meaningless.

:band:

I am certain there many other things HiFi Magazines do not do, largely because of realising the futility of doing so in suchlike manner.

Just because HiFi Magazines do not conduct small scale, semi-casual double blind tests does not mean that those who do (including the US ABX Mafia und Don Krueger) are to be commended for doing so and for producing reams of meaningless data in the process (they are DEFINITLY TO BE CENSURED for presenting said data as proof for the absence of audible differences).

In my experience the "debunkers" are as guilty in causing the current situation as the rest of the high end industry.

As a result most people still do not understand the actual underlying mechanisms that CAN lead to what is often called "cable sound" (largely inaccuratly so as well) and that most cables tend to be constructed on ludicrous principles usually contrary to what is actually required. In fact, we are still arguing if cables (or amplifiers, capacitors et al) can or can not lead to audible differences, instead on focusing on understanding what makes "good sound".

Okay. I'm ranting. I'm good now.

I am rather in agreement that "cable reviews" have little use. This is down mainly to the lack of understanding what is going on. They are useful insofar that they illtustrate what specifically works in a specific system, according to the reviewers taste (and imagination, hearing acuity etc.).

I wrote one myself on interconnects, where I included blind listening tests (fast switching), however while working a lot on the DIY Side of that cabeling lark I had to get to understand what is actually happening inside systems and why cables make a difference which ended up with me realising the futility of such reviews.

(BTW, my first experience of hearing a cable make an audible difference happened with a microphone cable in a pro audio application and was most annoying, because it threw off my mix I was used to and had me looking for weeks as to what had changed the sound [which was BAD - because before it was exactly what I had wanted] before hitting on around 10m worth of balanced mike cable as culprit).

So, with that intro, I feel that the test was exactly as useless and as meaningless as one in a mag, but with a greater degree of amusement potential, which may be why I felt amused.



Ciao T

 

Audiophiles can bleat on all they like about how jaw-dropping the differences between power cables are, when the reality is that given just how "highly disputed" the subject is, the differences in reality must, at best, be tiny. If they weren't tiny, they'd be obvious to everyone, cable scepticism wouldn't exist, and, presumably, the differences would be measurable too. As it is, they're unknown to all but a small minority of audiophiles.

Anybody who claims the difference are obvious, jaw-dropping, night and day, etc, should find it easy to prove they can repeatedly hear such differences, even under the stresses and pressures of a blind test (poor darlings). Shame such people consistently refuse to take part in such tests, or suddenly decide the system involved isn't "revealing" enough to show up what they'd previously claimed was a jaw-dropping difference. Kudos to all those who put this test together, though, nice to see some positive efforts being made.

-- Ian