A/B Testing

I am not really convinced by A/B testing. I prefer to live with a piece of music/equipment for some time as the real character of how the HiFi interacts with the music gradually reveals itself to you. After a week or two I seem more able to decide on whether I can live with it or not. I live a:MILD: long way from the nearest HiFi emporium or shop with listening room (an 14 hour ferry trip away), so I rely on forums and the like. With the advent of ebay the hobby has become less risky as if I dont like it, I can always resell. Quick changes from one piece of equipment to an other can lead to opinions becoming informed by the initial few minutes of listening. During this phase the ear or (psych ear) is easily tricked by novelty.

What do you think??

 

I agree. Our primeval instinct is to listen for differences in our envinronment to alert us to danger. Unless the various scientific studies over the years suggest differently, we are likely to notice the difference and may wrongly gravitate towards it. There's an argument that as we listen more and gain more experience over the years we develop an instinct for listening and short comparisons are often enough. The problem with this is that is can be flawed for various reasons.
1. Kit might not be burned/run-in (if you accept that principle, some don't)
2. Kit might not be warmed up properly (ditto)
3. Adverse reaction may only be highlighting something else wrong
4. Adverse reaction might be due to a shift in overall balance suggesting previously masked tonal characteristics elsewhere no longer cut it
5. Good systems have good and better days depending on mains, humidity, room temp et al and one short dem does not necessarily show the best
6. Longer sessions all ow you to listen, come back to things, vary them, adjust the system etc.

I'm sure there are more but it's enough. It also means you don't have to sit there whilst some goon taps his feet or nods away as if you're supposed to get it, whatever it is, under subtle duress. Unless you get perverse fun out of showing them up once your experience outweighs theirs. That would be sad.

 

I think A/B versus 'long listen' tell you different things ie they are both valid.

A/B tells you big differences like frequency response. This is basically speakers, since everything else is virtually a flat response.

Long listen tells you about detail and various things that may bug you but not someone else. Since they only crop up after listening to lots of music - this is also valid.

 

Also, listening over a long period can tell you if an item is non-fatiguing and fun to listen to. A quick burst of another piece of kit can sound exciting but might be unrewarding after a couple of hours.

 

It is a bit like judging a few paintings having all of them in the same room under the same condition all at once is more informative usually. If you are into photography you might like to judge what photographs taken by different cameras appear better. It is good if you could study one for as long as one would like to but having them side by side on the same table is very informative as well as it really highlight any obvious differences and why some are so much better then the others.

Big problem with long term comparison of any individual component without a reference is after a period of time our ears usually try to adjust and accomodate to any mark abnormalies like distortions in the frequency response.

 

I've always found with A/B testing that you almost invariably prefer the B. I prefer to try a long test in my system and then replace the original item. If it's better it's obvious, if it's worse it's even more obvious.

 

I'd agree with Graham C

They tell you different things both have a place.

Fior me A-B test will highlight the differences between the two, but living with it will show if you like the item.

 

that was the only conclusion to be deduced from the cable test carried out on hifiwigwam/zerogain several months ago - a substantive and statistically significant preference for B.

 

In that case, always listen to the cheaper item second

 

good plan - except its an artifact, and then over time quality counts.

... that's one of the two reasons why when blind A/B testing both A,B and B,A differences should be measured/recorded

 

The problem with measurement is we have to know how to measure the differences we perceive and the act of measuring must not affect the subject being measured. HUman beings can sense so much more than that which is measureable as we use analogue senses and not integer sampling. - I think!

 

in most A/B (blind) testing, the humans are the measuring instruments - and like most/all measuring instruments they are fallible, being suscepible to lack of reliability (when given the same task) and bias.. not sure what yu mean by 'analogue senses and not integer sampling'