Mainly the fact that our systems must necessarily apply the same distortion systematically to anything regardless of context or circumstances, because they can't make a judgement call on whether or not it's necessary or desirable in a particular case.

Don't get me wrong though, I don't mind what anyone personally prefers. I think I prefer a totally clean, accurate sound - ie. an exact reproduction of the live musical event, bearing in mind that 99% of the time I'm listening to fully acoustically produced music - but then obviously I've never heard a system that was even nearly 'ideal'.