Announcement

  • 2018/02/01 - Site open.
  • 2018/02/26 - Update Week 1 Lecture.
  • 2018/03/05 - Update Week 2 Lecture, Homework 1 announced.
  • 2018/03/12 - Update Week 3 Lecture, Final Suggest Reading
          Final : 可以選擇寫程式或報告,報告的題目可以參考上面連結裡老師建議的或是自己找再跟老師討論。
          報告時間會根據人數調整。
  • 2018/03/19 - Update Week 4 Lecture, Homework 2 announced.
  • 2018/03/26 - Update Week 5 Lecture.
  • 2018/04/09 - Update Week 7 Lecture, HW1 Grading and FAQ, Reading assignment.
  • 2018/04/16 - Update Week 8 Lecture, Programming Homework(Midterm) announced, Midterm Presentation Registry, Please fill in before 04/24 PM11:59.
  • 2018/04/23 - Update Week 9 Lecture, Presentation order is in the Google Doc, about 30 mins for each person.
  • 2018/04/24 - Update Reference for Final from Professor.
  • 2018/05/07 - Update Week 11 Lecture.
  • 2018/05/13 - Update Mid(HW3) Grading, Update Week 12 Lecture.
  • 2018/05/16 - Update Final Project announced, Programming Homework(Final) announced, Final Presentation Registry, Please fill in before 06/03(Sunday) PM11:59.
  • 2018/05/21 - Update Week 13 Lecture.
  • 2018/06/11 - Please mail the report to itct@cmlab.csie.ntu.edu.tw. You can only mail the ppt and anything you want, like your midterm ppt.
  • 2018/06/24 - HW2 Grading.

  • Last Edited Time : 18/06/26 09:26 AM : Week 13, Final Announced.
  • Course Information

  • Lecturer: Prof. 吳家麟 (wjl@csie.ntu.edu.tw) R214
  • TA: 廖建棋 (r05922002@ntu.edu.tw) R505
          吳雨澄 (r05922049@ntu.edu.tw) R505
  • TA Hour : By appointment via email.
  • itct@cmlab.csie.ntu.edu.tw
  • Course Requirements

  • Homework : 30%
  • Midterm : 30%
  • Term Project : 40%
  • Outline

  • Basics of Information Theory
    - Entropy, Relative Entropy, Mutual Information.
  • Data Compression / Compaction
    - Kraft Inequality, the prefix condition and Instantaneous decodable codes.
  • Variable Length Codes
    - Huffman code, Arithmetic code and L-Z code.
  • Coding Techniques
    - DPCM (predictive coding).
    - Transform coding (Discrete Cosine Transform).
    - JPEG (JPEG2000).
    - Motion Estimation and Compression.
    - MPEG-1, 2, 4.
    - H.26P, H.264, HEVC.
    - Distributed Video Coding
  • Steganography and Information Hiding
    - Digital Watermarking.
  • Homework

  • For late submission, contact TA to hand-in.
    Score *= (0.8)^(late days)
  • 2018/03/05 - Homework 1
    Due date : 2018/03/26, handwritten or printed and submit in the class.
    You must finish the "Must done" section. The "Recommended" part is NOT included in grading.
    Hint : log(2) = 1.
  • 2018/03/19 - Homework 2
    Due date : 2018/04/09, handwritten or printed and submit in the class.
    ***NOTICE***Don't have to do question 11.
  • 2018/04/16 - Programming Homework (Midterm)
    reference:
    JPEG Intruction,
    SPEC,
    Testing files,
    JPEG_SPEC,
    Parser (for debug),
    A good reference web.
    Deadline: 2018/05/07 11:59 PM submit to itct@cmlab.csie.ntu.edu.tw
  • 2018/5/16 - Final Coding Project,
    MPEG_SPEC,
    Deadline: 2018/6/22 11:59 PM
    Demo Time: 2018/6/25 after/before class or break time
  • Grades

  • HW1 Grading and FAQ.
  • Mid(HW3) Grading.
  • HW2 Grading .
  • FYI : 國立臺灣大學學生個人獎懲辦法
        第十條 學生個人有下列各款情形之一者,予以勒令退學:
            二、考試作弊,情節嚴重。
        第十一條 學生個人有下列各款情形之一者,予以開除學籍:
            二、連續考試作弊,情節嚴重。
  • Lecture Notes

    Week 1 : Introduction

    Lecture 1.1.pdf   (Last Modified: 2018/02/26 03:59 PM)
    Lecture 1.1.mp4 (Last Modified: 2018/02/26 03:59 PM)
    Lecture 1.2.pdf   (Last Modified: 2018/02/26 03:59 PM)
    Lecture 1.2.mp4 (Last Modified: 2018/02/26 03:59 PM)

    Week 2 : Entropy, Relative Entropy and Mutual Information

    Lecture 2.pdf   (Last Modified: 2018/03/05 02:20 PM)
    Lecture 2.mp4 (Last Modified: 2018/03/05 02:20 PM)

    Week 3 : The Asymptotic Equipartition Property and Entropy Rates

    Lecture 2_Extra1.pdf (Last Modified: 2018/03/12 03:24 PM)
    Lecture 2_Extra2.pdf (Last Modified: 2018/03/12 03:24 PM)
    Lecture 3.pdf   (Last Modified: 2018/03/12 02:19 PM)
    Lecture 3.mp4 (Last Modified: 2018/03/12 02:19 PM)

    Week 4 : The Asymptotic Equipartition Property and Entropy Rates - 2

    Same as last week.

    Week 5 : Channel and Channel Capacity

    Lecture 4.1.pdf   (Last Modified: 2018/03/26 02:24 PM)
    Lecture 4.1.mp4 (Last Modified: 2018/03/26 02:24 PM)
    Lecture 4.2.pdf   (Last Modified: 2018/03/26 02:24 PM)
    Lecture 4.2.mp4 (Last Modified: 2018/03/26 02:24 PM)
    Lecture 4.3.pdf   (Last Modified: 2018/03/26 02:24 PM)
    Lecture 4.3.mp4 (Last Modified: 2018/03/26 02:24 PM)

    Week 6 : Spring Vacation


    Week 7 : Data Compression & Huffman Codes

    Lecture 5.pdf   (Last Modified: 2018/04/09 02:45 PM)
    Lecture 5.mp4 (Last Modified: 2018/04/09 02:45 PM)
    Lecture 6.1.pdf   (Last Modified: 2018/04/09 02:45 PM)
    Lecture 6.1.mp4 (Last Modified: 2018/04/09 02:45 PM)

    Week 8 : Image Data Compression

    Lecture 6.2.pdf   (Last Modified: 2018/04/09 02:45 PM)
    Lecture 6.2.mp4 (Last Modified: 2018/04/09 02:45 PM)
    Lecture 9.1.pdf   (Last Modified: 2018/04/16 01:04 AM)
    Lecture 9.1.mp4 (Last Modified: 2018/04/16 01:04 AM)

    Week 9 : Transform Coding Technique(DCT)

    Lecture 9.2.pdf   (Last Modified: 2018/04/16 01:04 AM)
    Lecture 9.2.mp4 (Last Modified: 2018/04/16 01:04 AM)
    Lecture 9.3.pdf   (Last Modified: 2018/04/16 01:04 AM)
    Lecture 9.3.mp4 (Last Modified: 2018/04/16 01:04 AM)
    Lecture 10.1.pdf   (Reading) (Last Modified: 2018/04/23 10:21 PM)
    Lecture 10.1.mp4 (Reading) (Last Modified: 2018/04/23 10:21 PM)
    Lecture 10.2.pdf   (Last Modified: 2018/04/23 10:21 PM)
    Lecture 10.2.mp4 (Last Modified: 2018/04/23 10:21 PM)
    Lecture 10.3.pdf   (Last Modified: 2018/04/23 10:21 PM)
    Lecture 10.3.mp4 (Last Modified: 2018/04/23 10:21 PM)

    Week 10 : Oral Presentation


    Week 11 : An Overview of Video Coding Algorithms

    Lecture 11.3.pdf   (Last Modified: 2014/05/07 08:21 AM)
    Lecture 11.3.mp4 (Last Modified: 2014/05/07 08:21 AM)

    Week 12 : An Overview of Video Coding Algorithms - 2

    Lecture 11.1.pdf   (Last Modified: 2014/05/07 08:21 AM)
    Lecture 11.1.mp4 (Last Modified: 2014/05/07 08:21 AM)
    Lecture 11.2.pdf   (Last Modified: 2014/05/07 08:21 AM)
    Lecture 11.2.mp4 (Last Modified: 2014/05/07 08:21 AM)
    Lecture 11.4.pdf   (Last Modified: 2014/05/14 07:51 PM)

    Week 13:Rate Distortion Function and Optimal Bit-AllocationCoding Algorithms & Arithmetic Code

    Lecture 12.pdf   (Last Modified: 2014/05/20 10:44 PM)
    Lecture 12.mp4 (Last Modified: 2014/05/20 10:44 PM)
    Lecture 7.1.pdf   (Last Modified: 2018/04/16 02:42 PM)
    Lecture 7.1.mp4 (Last Modified: 2018/04/16 02:42 PM)
    Lecture 7.2.pdf   (Last Modified: 2018/04/16 02:42 PM)
    Lecture 7.2.mp4 (Last Modified: 2018/04/16 02:42 PM)
    Lecture 7.3.pdf   (Last Modified: 2018/04/16 02:42 PM)
    Lecture 7.3.mp4 (Last Modified: 2018/04/16 02:42 PM)



    Week 14:Information Theory in Data Visualization

    Information Theory in Data Visualization-1.pdf. (Last Modified: 2016/05/16 01:16 PM)
    Information Theory in Data Visualization-1.mp4. (Last Modified: 2016/05/16 01:14 PM)
    Information Theory in Data Visualization-2.pdf. (Last Modified: 2016/05/16 01:16 PM)
    Information Theory in Data Visualization-2.mp4. (Last Modified: 2016/05/16 01:14 PM)

    Week 15:Arithmetic Code

    Week 16:LZ Code

    Lecture 8.1.pdf. (Last Modified: 2014/04/23 02:07 PM)
    Lecture 8.1.mp4. (Last Modified: 2014/04/23 02:08 PM)
    Lecture 8.2.pdf. (Last Modified: 2014/04/23 02:07 PM)
    Lecture 8.2.mp4. (Last Modified: 2014/04/23 02:07 PM)
    Lecture 8.3.pdf. (Last Modified: 2014/04/23 02:07 PM)
    Lecture 8.3.mp4. (Last Modified: 2014/04/23 02:07 PM)

    Week 17: Encryption then Compression Scheme

    Lecture 13.1.pdf. (Last Modified: 2014/04/23 02:10 PM)
    Lecture 13.1.mp4. (Last Modified: 2014/05/20 10:43 PM)
    Lecture 13.2.pdf. (Last Modified: 2014/04/23 02:10 PM)
    Lecture 13.2.mp4. (Last Modified: 2014/05/20 10:44 PM)
    Lecture 13.3.pdf. (Last Modified: 2014/04/23 02:10 PM)
    Lecture 13.3.mp4. (Last Modified: 2014/05/20 10:44 PM)

    Others:

    Lecture 15.1.pdf. (Last Modified: 2014/05/27 11:11 PM)
    Lecture 15.1.mp4. (Last Modified: 2014/05/27 11:11 PM)
    Lecture 15.2.pdf. (Last Modified: 2014/05/27 11:11 PM)
    Lecture 15.2.mp4. (Last Modified: 2014/05/27 11:11 PM)
    Lecture 15.3.pdf. (Last Modified: 2014/05/27 11:11 PM)
    Lecture 15.3.mp4. (Last Modified: 2014/05/27 11:11 PM)
    Advanced Video Compression.pdf (Last Modified: 2018/04/09 02:49 PM)
    Differential Entropy.pdf (Last Modified: 2018/04/09 02:46 PM)

    Resource & Reference

  • Thomas M. Cover and Joy A. Thomas, Elements of Information Theory, Second Edition, Wiley & Sons, 2006.
  • Khalid Sayood, Introduction to data compression 2ed, Morgan Kaufmann, 2000.
  • JPEG Standard.
  • MPEG-1 Standard.
  • Darrel Hankerson, Greg A. Harris, and Peter D. Johnson Jr., Introduction to information theory and data compression, CRC, 1998.
  • The Minimum Description Length Principle, Peter D. Grunward, the MIT Press, 2007.
  • IEEE Trans. on IT, CSVT, SP, IP, MM, IFS, Comm, Comput
  • Information_Theory_Reference.pdf.
  • Reference from the Journal Entropy.pdf.