Information Theory by Yao Xie
This note will explore the basic concepts of information theory. It is highly recommended for students planning to delve into the fields of communications, data compression, and statistical signal processing. Topics covered includes: Entropy and mutual information, Chain rules and inequalities, Data processing, Fano's inequality, Asymptotic equipartition property, Entropy rate, Source coding and Kraft inequality, Optimal code length and roof code, Huffman codes, Shannon-Fano-Elias and arithmetic codes, Maximum entropy, Channel capacity, Channel coding theorem, Differential entropy, Gaussian channel, Parallel Gaussian channel and water-filling, Quantization and rate-distortion.
Author(s): Yao Xie
NA Pages 
Lecture Notes on Information Theory by Prof. Dr. rer. nat. Rudolf Mathar
This lecture note covers introduction, Fundamentals of Information Theory, Source Coding and Information Channels.
Author(s): Prof. Dr. rer. nat. Rudolf Mathar, Institute for Theoretical Information Technology Kopernikusstr, Germany
59 Pages
Applied Digital Information theory
This note serves as a comprehensive guide to fundamental concepts in information theory and coding. This pdf provides discrete probability theory, information theory, and coding principles. Beginning with Shannon's measure of information, then delves into the efficient coding of information, the methodology of typical sequences is introduced, emphasizing the distinction between lossy and lossless source encoding. The text also discusses coding for noisy digital channels, block coding principles and tree and trellis coding principles.
Author(s): James L.Massey
153 Pages
Lecture Notes on statistics and information Theory
This lecture note navigates through information theory, statistics and measure theory. It covers fundamental concepts such as definitions, chain rules, data processing inequalities, and divergences and extends to optimal procedures, LeCam’s and Fano’s inequalities, and operational results like entropy and source coding. It also focus on exponential families and statistical modeling, fitting procedures, and lower bounds on testing parameters, sub-Gaussian and sub-exponential random variables, martingale methods, uniformity covering topics such as Kullback-Leibler divergence, PAC-Bayes bounds, interactive data analysis, and error bounds.
Author(s): John Duchi
464 Pages
This book explains basics of thermodynamics, including thermodynamic potentials, microcanonical and canonical distributions, and evolution in the phase space, The inevitability of irreversibility, basics of information theory, applications of information theory, new second law of thermodynamics and quantum information.
Author(s): Gregory Falkovich
165 Pages
This note will explore the basic concepts of information theory. It is highly recommended for students planning to delve into the fields of communications, data compression, and statistical signal processing. Topics covered includes: Entropy and mutual information, Chain rules and inequalities, Data processing, Fano's inequality, Asymptotic equipartition property, Entropy rate, Source coding and Kraft inequality, Optimal code length and roof code, Huffman codes, Shannon-Fano-Elias and arithmetic codes, Maximum entropy, Channel capacity, Channel coding theorem, Differential entropy, Gaussian channel, Parallel Gaussian channel and water-filling, Quantization and rate-distortion.
Author(s): Yao Xie
NA Pages
Information Theory, Inference, and Learning Algorithms (David J.C. MacKay)
Currently this section contains no detailed description for the page, will update this page soon.
Author(s):
Pages
