UAE University
Department of Electrical Engineering


Course Name & Number:
Information Theory & Coding (50741541, ELEC541)
Instructor:
Dr. Eesa Mohammed Abdulrazzaq Bastaki
Office:
EE Dept/ Jimi Campus, Tel- 7051757,
Email- eesa@uaeu.ac.ae,
Homepage:http://faculty.uaeu.ac.ae/~eesa/
or http://www.bastaki.ae/ or http://www.bastaki.net/
Textbook:
Richard B. Wells, Applied Coding & Information Theory for Engineers, Prentice Hall, NJ 1999

Grading:
1. Exams20%
2. Homeworks, Short Papers and Term Project20%
3. Midterm Exam20%
4. Final Exam40%
_______________________________________
Total100%

Objective:
To introduce to the students the concept of amount of information, entropy, channel capacity, error-detection and error-correction codes, block coding, convolutional coding, and Viterbi decoding algorithm.

Assignments:
Download homework assignments.
Course Files:
Download Information Theory & Coding File

Prerequisites: Communications Theory (50740441, ELEC441)

Course Outline:
  1. Information Theory
    1. Introduction
    2. Uncertainty, Information, and Entropy
    3. Source-Coding Theorem
    4. Data Compaction
    5. Huffman Coding
    6. Lempel-Ziv Coding
    7. Discrete Memoryless Channels (DMC)
    8. Mutual Information
    9. Channel Capacity
    10. Channel Coding Theorem
    11. Information Capacity Theorem
  2. Error-Control Coding
    1. Introduction
    2. Linear Block Codes
    3. Generator Matrix
    4. Parity-Check Matrix
    5. Syndrome
    6. Group Theory
    7. Examples in Using Group Theory to Correct Erroneous Codes
  3. Convolutional Codes
    1. Introduction
    2. Convolutional Encoder
    3. An Example of a Convolutional Encoder
    4. Usual Mode of Operation
    5. General Rate 1/n Constraint Length K Code
    6. Tree Representation of Convolutional Codes
    7. Finite-State Machine Code Representation
    8. Trellis Representation of Convolutional Codes
    9. ML Decoding of a Convolutional Code
    10. Viterbi Algorithm
    11. Free Distance of a Convolutional Code


References:
  1. Haykin, Simon, Communication Systems, 3rd Edition, John Wiley & Sons, N.Y. 1994
  2. Hamming, Richard W., Coding and Information Theory, 2nd Ed., Prentice-Hall Inc.,1986
  3. Ziemer, R.E. and Tranter,W.H., Principles of Communication Systems, 3rd Ed., John Wiley & Sons, 1994
  4. Stark, H./Tuteur, F./Anderson, J., Modern Electrical Communications, 3rd Ed.,
    Houghton Mifflin Co., 1990

Information Theory & Coding

Course Schedule

Week
No
Dates Coverage of
Course Contents
Problem Sessions,
Labs, Homeworks,
and Exams
1 Feb. 15 - 19 Introduction
Information
Entropy
Examples in Information
2 Feb. 22 - 26 Some Properties of Entropy
Extension of DMS
Problem Session
Homework #1
3 Mar. 1 - 5 Source Coding Theorem
Data Compression
Prefix Coding
Problem Session
Homework #2
4 Mar. 8 - 12 Kraft Inequality
Huffman Coding
Problem Session
Homework #3
5 Mar. 15 - 19 Huffman Coding
Lempel-Ziv Coding
Problem Session
Homework #4
6 Mar. 22 - 26 Discrete Memoryless Channels
Mutual Information
Exam #1
7 Mar. 29 - Apr. 2 Channel Capacity
Channel Coding Theorem
Problem Session
Homework #5
8 Apr. 5 - 9 Application of Channel
Coding Theorem in BSC
Discussion
9 Apr. 12 - 16 Information Capacity Theorem Midterm Exam
10 Apr. 19 - 23 Linear Block Codes LabVIEW Assignment
Homework #6
11 Apr. 26 - 30 Generator Matrix
Parity-Check Matrix
Group Theory
Discussion
12 May 3 - 7 Convolutional Encoder
An Example
Exam #2
13 May 10 - 14 General rate 1/n
and Constraint K
Problem Session
Homework #7
14 May 17 - 21 Tree Representation
Trellis Representation
Problem Session
Homework #8
15 May 24 - 28 Viterbi Algorithm Project Presentations
16 May 31 - June 4 Free Distance of
Convolutional Codes
Review
17,18 June 7 - 19 Final Examinations Final
Males: 14/6/2003, 8:00-10:00
Females: 17/6/2003, 8:00-10:00

Return to Eesa Bastaki's Home Page
Return to Departmental Courses' Home Page