Information Theory and Coding Technique

Announcement

2008/05/19 - The MPEG programming homework is announced.

2008/05/05 - The JPEG programming homework is announced.

2008/04/07 - The 1st programming homework is announced.

2008/03/10 - The 1st homework is announced.

2008/02/18 - The ITCT course starts.

Course Information

Lecturer: Prof. Ja-Ling Wu (wjl [at] csie.ntu.edu.tw).

TA: Wen-Huang Cheng (d93944001 [at] csie.ntu.edu.tw).
          Chia-Hu Chang (chchang [at] cmlab.csie.ntu.edu.tw).

Outline

Basics of Information Theory
     - Entropy Function, Source Coding Theory.

Lossless Compression Techniques
     - Huffman Coding, Arithmetic Coding, LZ-Coding, Predictive Coding.

Lossy Compression Techniques
     - Quantization, Transform-coding, Vector-Quantization, Subband coding, Multi-resolution, Perceptual coding.

Multimedia Data Compression Techniques
     - Image Compression, Audio Compression, Video Compression.

Overview of Coding Standards
     - JPEG, MPEG-1, MPEG-2, MPEG-4, H.26X.

Lecture Notes

Information Theory

- What is Information Theory?. (Updated: 2008/02/15)

- Introduction. (Updated: 2007/02/26)

- Entropy, Relative Entropy and Mutual Information. (Updated: 2008/03/05)

- Channel and Channel Capacity - Part 1. (Updated: 2008/03/17)

- Channel and Channel Capacity - Part 2. (Updated: 2008/03/17)

- The Asymptotic Equipartition Property. (Updated: 2008/03/10)

Data Compression

- Introduction. (Updated: 2008/03/28)

- Source Coding. (Updated: 2008/03/24)

- Huffman Codes. (Updated: 2008/04/09)

- Arithmetic Coding. (Updated: 2008/04/09)

- Dictionary Codes and Lempel-Ziv Coding. (Updated: 2006/04/24)

- Lempel-Ziv Coding. (Updated: 2006/04/17)

- Lempel-Ziv Coding: Adaptive Dictionary Compression Algorithm. (Updated: 2008/04/11)

- Transform Coding. (Updated: 2008/04/22)

- Discrete Cosine Transform. (Updated: 2007/05/07)

- Image Data Compression. (Updated: 2006/05/22)

- Fast Algorithms for the DCT. (Updated: 2002/04/13)

- Motion Estimation for Video Coding Standards. (Updated: 2006/05/29)

- An Overview of Video Coding Algorithms. (Updated: 2007/05/14)

Reference Data

- An Introduction to MPEG Video Compression. (Updated: 2002/04/09)

- DCT reference - 1. (Updated: 2002/05/16)

- DCT reference - 2. (Updated: 2002/05/16)

- DCT reference - 3. (Updated: 2002/05/16)

- Search Algorithms for Block-Matching in Motion Estimation. (Updated: 2002/05/16)

- Reference at the course of 2006/03/06. (Updated: 2006/03/06)

- An Overview of Video Compression Algorithms. (Updated: 2006/06/05)

- Quality Measures for Gray Scale Image Compresion. (Updated: 2006/06/05)

- Rate Distortion Function and Optimal Bit-allocation. (Updated: 2007/06/04)

- Integer DCTs and Fast Algorithms. (Updated: 2006/06/05)

- Video Coding (MPEG-1). (Updated: 2007/05/21)

- Motion Estimation and Compensation. (Updated: 2008/05/19)

- Video Coding. (Updated: 2008/06/02)

Homework

2008/03/10 - Homework 1. (Due: 2008/03/24)

2008/04/07 - Programming Homework 1 and ppt. (Due: 2008/04/28)
Test-Image-I
and Test-Image-II

2008/05/05 - The JPEG decoder programming homework. (Due: 2008/05/19)
Overview of JPEG International Standard - Baseline Part
JPEG International Standard (T.81E)
monalisa.jpg | teatime.jpg | gig-sn01.jpg | gig-sn08.jpg

2008/05/19 - The MPEG-1 decoder programming homework. (Due: 2008/06/09)
Overview of MPEG-1 International Standard
MPEG International Standard Part 2 - Visual (ISO 11172-2)
I_ONLY.M1V | IP_ONLY.M1V | IPB_ALL.M1V

Grades

Submission Status (Updated: 2008/06/19)

Resource & Reference

Thomas M. Cover and Joy A. Thomas, Elements of Information Theory, Wiley & Sons, 1991.

Khalid Sayood, Introduction to data compression 2ed, Morgan Kaufmann, 2000.

JPEG Standard.

MPEG-1 Standard.

Darrel Hankerson, Greg A. Harris, and Peter D. Johnson Jr., Introduction to information theory and data compression, CRC, 1998.