Shannon48

• The basis of syntactic information theory

• The basis for most modern communications systems theory and much of its practice

• H(M) is the ‘information content’ of M

  • For a finite symbol system with known probability distribution of symbol occurrence in its language
  • H(M)=sum of -p(si)*log(p(si)) (for all symbols si)
    • Where p(si) is the probability of symbol si
  • If M is chosen at random, H(M)=|M|
    • If not, H(M) < |M|

Previous slide

Next slide

Back to first slide

View graphic version