titian said:
They were talking about scientist and moral today and about an international institution that had to be created in order to keep an eye on the results of physicists and mathematicians because in these days they are more and more cases of frauds.
Interesting, though, are also the more and more cases of scientists who tweek / massage their results to become famous (greedingness of fame). I really got a shock watching this TV programme but it opened my eyes believing everything a scientist says.
Shall we now start talking about accurancy about the scientists?
Sure we can do that.
It comes as no surprise that there is fraud in science, there has been from the beginning. Alchemy, for instance, was a bust. However, as time has progressed we've got better at weeding out the frauds from the facts. The peer-review system is acknowledged as not being perfect, but it is the best that we have.
Good journals have standards for how to control bias in experiments and the reviewers know how to determine if a result is statistically significant for the construction of evidence.
When science goes wrong there are checks and balances: you must document your experiment sufficiently well that others can try it out, and thus they can verify what you are saying. Occasionally results are reviewed as correct, even though the reviewer hasn't performed the experiment to verify it. There is a heirarchy in scientific publications which lends credence to results based on the title of the journal and the review process tends to be better.
Scientific fraud is possible, but futile; you can't generally build products based on science that doesn't work. If your result is purely academic, then someone is likely to try and build on it, and in the process determine whether your results are valid. In the latter case you'll be quickly exposed as a fraud. Occasionally, fruads, or rather errors are rapidly exposed, like the debacle of Fleischmann and Ponds not correctly validating their cold fusion result before announcing it to the world. In summary the upside for fraud is limited, and the downside is very high (exposure as a fraud isn't conducive to further employment).
One of the easiest paths to commiting a scientific fraud is to decide what the result should be before gathering and analysing the evidence. Once you've decided the result, then you merely cherry-pick the data so that when it is analysed it supports your hypothesis. This can be a little tricky in cases where the reviewer know to look for bias. It's made harder by statistical significance which requires more than 2 or 3 supporting pieces of evidence. Massaging the evidence isn't easy, the checks and balances of the review process should catch this kind of sleight of hand. Perpetrating this kind of "fraud" isn't necessarily a "concious" choice on the part of the scientist.
So, to do science, you need to outline a (repeatable) experiment in sufficient detail that others can perform it, gather data to test you hypothesis and then analyse that data for statistical significance. You also need to implement bias controls, so that the experimenter can't rig the result, and that the experimentee can't guess the correct answer. Finally, you need to change just one variable at a time.
Now, what does science have to do with the process through which people buy hifi? In short
nothing.
For instance, some people claim that cables make a detectable difference to the sound of their system (perhaps going as far as proclaiming "night and day" differences or claiming changes in part of the frequency response and so on..)
However, the trap these folks have fallen into is "experimenter expectation": they've decided the result before the test has even been started. An environmental bias (magazine review, the price of the item, how shiney it is, some "voodoo science" about magic properties etc) has colored their judgement.
They then compound that error by just running ONE trial: try "old" item; try "new" item; decide new item better, end of test. Rarely, if ever do people swap back and forth, and if they do, the number of trials is not significant because there are insuffient swaps.. (It can be tedious to swap back and forth, but it's worth reflecting on just how tedious it was to earn the money you are spending before deciding that swapping back and forth is a waste of time.)
When folks do swap back and forth, they know what they are listening to: new toy, or old toy. A clear example of bias.
Next up, changing more than one variable: listening to a new component in a strange system - what might be called the "dealer demo". And let's not forget the demo room has a substantial effect even of all the components are the same as at home.
The next big bias control is "level matching" - minor level differences in volume alone explain the perceived differences in CD players.
The next source of bias is the difference in frequency response - it is quite easy to make components sound different by fiddling with their frequency response (Wadia, for instance, do this). The difference is audible, but is something that could be readily acheived with a simple tone control.
Just about every review of each item on the market, every dealer demo, and most manufacturer literature is a generated by a process which is worthless from a scientific standpoint. Naim (for instance) will never get a paper published on the "audibility of microphony" anywhere other than their own marketing literature.
Moreover, the reason Science isn't validating the claims made about differences in amplifiers, cables, cd players is because the experiments have been run and no one, in proper tests, has been able to provide any evidence that such differences are audible..
Perhaps you should have explored the word "mostly" in
"After 5K, it's mostly willy waving" before lauching into this rather poor Impulse Response (a little joke there for the EE types):
titian said:
So now I feel directly accused here.
I will reply saying that your affirmation is based on ignorance, envy and basic psychological animal selfdefense instinc.
These are the very traits you are exhibiting in the following paragraph:
titian said:
If you ever gave such a system the same chance that you give your own system and if you would objectively compare them, then you would hear a huge difference. With same chance I mean the time and passion to optimize it, to do tweeking, and to to all the care you do for your own system. And especially the time to listen to it without any "inverted" placebo effect, with this I mean: thinking that more expensive = snob (because of your envy that you don't have that money). I though agree that for that amount of money there are also very "special" products which don't have so much to do with high reproduction quality but putting everything in the same hat is very ignorant from your side. Actually thinking about it, there are also such products in the lower money segment.
If there is any vanity here it is yours and 100% (not 90%). Your vanity that you are the guy who's doing the right things and those others, who spend more than you can afford, aren't. You have my compassion.
If there is a "huge difference" in sound (not withstanding bass extension, and the effect of the different rooms) between your system and one that measures well, but costs substantally less (like $5k), then there is something wrong with your system.
Your speakers and your room are 99.9% of the sound you are getting. If your tube amps are particularly bad, then 5% of your sound might be your amps (distortion), and if you have a really badly designed CD player that might have a non flat response and also be "contributing".
There are perfectly rational scientific explanations to why your system sounds the way it does and, if you think you can hear something that science can't explain, then you need to familiarize yourself with a different branch of science: psychology..