In the early 1940's, C. E. Shannon developed a mathematical theory,called information theory, for dealing with the most fundamental aspects of communication systems. Characteristics of information theory:
1.probability theory highly involved
In the past 40 years, information theory has been made more
|
The Mathematical Model of Communication Systems
We use a maticalmatical model to stand for the real communication systems in information theory. The system is modeled by:
2 stages. One encodes the source, and the other further encodes to fit the channel. So the decoding block is devided too.
The purpose of the source encoder is
The purpose of the channel encoder and
|
![]() |
Assume there is an information source of symbols
![]()
We introduce the probability
The self-information of each symbol is defined as: ![]() the amount of information.
The average value of self-information of the source alphabet is known ![]() |
Let {a1,...,ak} be the X sample space and {b1,...,bj} be the Y sample space in an XY joint ensemble with the prob. assignment Px|y(ak|bj).
The occurrence of y=bj changes the probability of x = ak from the priori
The information provided about the event x=ak by the occurence of the event y=bj is: ![]() |
Topic Content |