Announcement

Course Information

  • Lecturer: Prof. Ja-Ling Wu (wjl@csie.ntu.edu.tw).
  • TA: 敺隤 (r04922107@ntu.edu.tw) R505
    單曹遙 (r04922113@ntu.edu.tw) R505
  • itct@cmlab.csie.ntu.edu.tw
    Attention: If your mail is over 5MB please kindly put your file link in your mail, since there is a 5MB limit for each mail
    This mail address is only for homework, TA's email is available for questions
  • Course Requirements

    Outline

  • Basics of Information Theory
    - Entropy, Relative Entropy, Mutual Information.
  • Data Compression / Compaction
    - Kraft Inequality, the prefix condition and Instantaneous decodable codes.
  • Variable Length Codes
    - Huffman code, Arithmetic code and L-Z code.
  • Coding Techniques
    - DPCM (predictive coding).
    - Transform coding (Discrete Cosine Transform).
    - JPEG (JPEG2000).
    - Motion Estimation and Compression.
    - MPEG-1, 2, 4.
    - H.26P, H.264, HEVC.
    - Distributed Video Coding
  • Steganography and Information Hiding
    - Digital Watermarking.
  • Lecture Notes

    Week 1:Introduction

    Lecture 1.1.pdf. (Last Modified: 2016/02/22 03:43 PM)
    Lecture 1.1.mp4. (Last Modified: 2016/02/22 03:43 PM)
    Lecture 1.2.pdf. (Last Modified: 2016/02/22 03:43 PM)
    Lecture 1.2.mp4. (Last Modified: 2016/02/22 03:43 PM)

    Week 2:Deferred holiday

    228

    Week 3:Entropy, Relative Entropy and Mutual Information

    Lecture 2.pdf. (Last Modified: 2014/03/01 12:47 AM)
    Lecture 2.mp4. (Last Modified: 2014/03/01 12:48 AM)

    Week 4:The Asymptotic Equipartition Property and Entropy Rates

    Lecture 2-Extra.pdf. (Last Modified: 2011/07/30 02:57 PM)
    Lecture 3.pdf. (Last Modified: 2014/03/05 04:04 PM)
    Lecture 3.mp4. (Last Modified: 2014/03/05 04:04 PM)

    Week 5:Channel and Channel Capacity

    Lecture 4.1.pdf. (Last Modified: 2014/03/19 08:41 PM)
    Lecture 4.1.mp4. (Last Modified: 2014/03/19 08:41 PM)
    Lecture 4.2.pdf. (Last Modified: 2014/03/19 08:41 PM)
    Lecture 4.2.mp4. (Last Modified: 2014/03/19 09:02 PM)
    Lecture 4.3.pdf. (Last Modified: 2014/03/19 09:02 PM)
    Lecture 4.3.mp4. (Last Modified: 2014/03/19 09:02 PM)
    Differential Entropy1.pdf. (Last Modified: 2014/03/19 08:41 PM)

    Week 6:Data Compression

    Lecture 5.pdf. (Last Modified: 2014/03/19 09:02 PM)
    Lecture 5.mp4. (Last Modified: 2014/03/19 09:03 PM)
    Lecture 6.1.pdf. (Last Modified: 2014/03/19 09:03 PM)
    Lecture 6.1.mp4. (Last Modified: 2014/03/19 09:03 PM)
    Lecture 6.2.pdf. (Last Modified: 2014/04/09 02:34 PM)
    Lecture 6.2.mp4. (Last Modified: 2014/04/09 02:34 PM)

    Week 7:holiday


    Week 8:Image Data Compression

    Lecture 9.1.pdf. (Last Modified: 2014/04/30 01:52 PM)
    Lecture 9.1.mp4. (Last Modified: 2014/04/30 01:53 PM)
    Lecture 9.2.pdf. (Last Modified: 2014/04/30 01:53 PM)
    Lecture 9.2.mp4. (Last Modified: 2014/04/30 01:53 PM)
    Lecture 9.3.pdf. (Last Modified: 2014/04/30 01:53 PM)
    Lecture 9.3.mp4. (Last Modified: 2014/04/30 01:53 PM)

    Week 9:Transform Coding Technique

    Lecture 10.1.pdf. (Last Modified: 2014/05/07 08:20 AM)
    Lecture 10.1.mp4. (Last Modified: 2014/05/07 08:20 AM)
    Lecture 10.2.pdf. (Last Modified: 2014/05/07 08:20 AM)
    Lecture 10.2.mp4. (Last Modified: 2014/05/07 08:20 AM)
    Lecture 10.3.pdf. (Last Modified: 2014/05/07 08:20 AM)
    Lecture 10.3.mp4. (Last Modified: 2014/05/07 08:20 AM)

    Week 10:Oral Presentation


    Week 11:An Overview of Video Coding Algorithms

    Lecture 11.1.pdf. (Last Modified: 2014/05/07 08:21 AM)
    Lecture 11.1.mp4. (Last Modified: 2014/05/07 08:21 AM)
    Lecture 11.2.pdf. (Last Modified: 2014/05/07 08:21 AM)
    Lecture 11.2.mp4. (Last Modified: 2014/05/07 08:21 AM)
    Lecture 11.3.pdf. (Last Modified: 2014/05/07 08:21 AM)
    Lecture 11.3.mp4. (Last Modified: 2014/05/07 08:21 AM)
    Lecture 11.4.pdf. (Last Modified: 2014/05/14 07:51 PM)

    Week 12:Rate Distortion Function and Optimal Bit-AllocationCoding Algorithms

    Lecture 12.pdf. (Last Modified: 2014/05/20 10:44 PM)
    Lecture 12.mp4. (Last Modified: 2014/05/20 10:44 PM)

    Week 13:Information Theory in Data Visualization

    Information Theory in Data Visualization-1.pdf. (Last Modified: 2016/05/16 01:16 PM)
    Information Theory in Data Visualization-1.mp4. (Last Modified: 2016/05/16 01:14 PM)
    Information Theory in Data Visualization-2.pdf. (Last Modified: 2016/05/16 01:16 PM)
    Information Theory in Data Visualization-2.mp4. (Last Modified: 2016/05/16 01:14 PM)

    Week 14:Arithmetic Code

    Lecture 7.1.pdf. (Last Modified: 2014/04/16 02:03 PM)
    Lecture 7.1.mp4. (Last Modified: 2014/04/16 02:04 PM)
    Lecture 7.2.pdf. (Last Modified: 2014/04/16 02:03 PM)
    Lecture 7.2.mp4. (Last Modified: 2014/04/16 02:03 PM)
    Lecture 7.3.pdf. (Last Modified: 2014/04/16 05:37 PM)
    Lecture 7.3.mp4. (Last Modified: 2014/04/16 05:37 PM)

    Week 15:LZ Code

    Lecture 8.1.pdf. (Last Modified: 2014/04/23 02:07 PM)
    Lecture 8.1.mp4. (Last Modified: 2014/04/23 02:08 PM)
    Lecture 8.2.pdf. (Last Modified: 2014/04/23 02:07 PM)
    Lecture 8.2.mp4. (Last Modified: 2014/04/23 02:07 PM)
    Lecture 8.3.pdf. (Last Modified: 2014/04/23 02:07 PM)
    Lecture 8.3.mp4. (Last Modified: 2014/04/23 02:07 PM)

    Week 16: Encryption then Compression Scheme

    Lecture 13.1.pdf. (Last Modified: 2014/04/23 02:10 PM)
    Lecture 13.1.mp4. (Last Modified: 2014/05/20 10:43 PM)
    Lecture 13.2.pdf. (Last Modified: 2014/04/23 02:10 PM)
    Lecture 13.2.mp4. (Last Modified: 2014/05/20 10:44 PM)
    Lecture 13.3.pdf. (Last Modified: 2014/04/23 02:10 PM)
    Lecture 13.3.mp4. (Last Modified: 2014/05/20 10:44 PM)

    Others:

    Lecture 15.1.pdf. (Last Modified: 2014/05/27 11:11 PM)
    Lecture 15.1.mp4. (Last Modified: 2014/05/27 11:11 PM)
    Lecture 15.2.pdf. (Last Modified: 2014/05/27 11:11 PM)
    Lecture 15.2.mp4. (Last Modified: 2014/05/27 11:11 PM)
    Lecture 15.3.pdf. (Last Modified: 2014/05/27 11:11 PM)
    Lecture 15.3.mp4. (Last Modified: 2014/05/27 11:11 PM)
    Advanced Video Compression.pdf. (Last Modified: 2014/06/11 02:19 AM)

    Homework

  • 2017/03/06 - Homework 1
  • Due date:2017/3/20, handwritten and submit in the class.
  • You must finish the "Must done" section. The "Recommended" part is NOT included in grading.
  • 2014/03/13 - Homework 2
  • Due date:2017/3/27, handwritten and submit in the class.
  • ***NOTICE***Dont have to do question 11.
  • Resource & Reference

  • Thomas M. Cover and Joy A. Thomas, Elements of Information Theory, Second Edition, Wiley & Sons, 2006.
  • Khalid Sayood, Introduction to data compression 2ed, Morgan Kaufmann, 2000.
  • JPEG Standard.
  • MPEG-1 Standard.
  • Darrel Hankerson, Greg A. Harris, and Peter D. Johnson Jr., Introduction to information theory and data compression, CRC, 1998.
  • The Minimum Description Length Principle, Peter D. Grunward, the MIT Press, 2007.
  • IEEE Trans. on IT, CSVT, SP, IP, MM, IFS, Comm, Comput
  • Information_Theory_Reference.pdf.
  • Reference from the Journal Entropy.pdf.
  • Last Modified

    Last Edited Time: 2017/03/13 05:22 PM

  • MPEG Grades.
  • Home