I'm out of my depth with this.

With my poor grade GCSE maths

For an assignment we have to develop our own compression and encoding system using the Entropy theory

http://en.wikipedia.org/wiki/Information_theory

Those formula's may as well be written in Urdu.

 

Any compression system involves the concept of entropy, since entropy is a measure of information, and maximising information density or minimising redundancy is what compression is all about.

Can you enlarge on your actual assignment?

Paul

 

I haven't got the assignment yet, that is next week, this week are supposed to be doing background reading on Entropy, I think I need to read a bit of Huffman.

I've not been taught any of this stuff, we ust get a little bit of background reading to each week and this week it is entropy.

One thing is describe the properties of the entropy function, I am guessing logerithims have somthign to with it.

 

Say we had 32 equally likely symbols. The Shannon Entropy function suggests that this requires 'minus one times the sum, where n=0-31, of the probability of the nth symbol times the log2 of the probability of the nth symbol' bits per symbol. Since each symbol is equally likely this resolves to -32*(1/32)*log2(1/32). I think this calculates out at '5' which is convenient.

Paul

 

Aaaaaahhhghhghgghghhhhhh - the flashbacks are starting - the HELL of cybernetics has finally come back to haunt me! Information Theory - I *knew* I'd heard that "entropy" word somewhere before...

Good luck AT - it's not as scary as all that (but I've forgotten all of what I was taught on it!)

 

Huffman? Sounds familiar. Isn't that the one used for fax compression. I seem to remember writing a codec for that once. My boss at the time implemented an LZW codec in assembler in about 2 weeks so it can't be all that hard. Don't panic AT!

 

From Flanders and Swann's song about thermodynamics: