An Introduction to Information Theory and Applications

Lecture Notes on Information Theory by Prof. Dr. rer. nat. Rudolf Mathar

This lecture note covers introduction, Fundamentals of Information Theory, Source Coding and Information Channels.

Author(s): Prof. Dr. rer. nat. Rudolf Mathar, Institute for Theoretical Information Technology Kopernikusstr, Germany

59 Pages

Applied Digital Information theory

This note serves as a comprehensive guide to fundamental concepts in information theory and coding. This pdf provides discrete probability theory, information theory, and coding principles. Beginning with Shannon's measure of information, then delves into the efficient coding of information, the methodology of typical sequences is introduced, emphasizing the distinction between lossy and lossless source encoding. The text also discusses coding for noisy digital channels, block coding principles and tree and trellis coding principles.

Author(s): James L.Massey

153 Pages

Basics of information theory

This book explains basics of thermodynamics, including thermodynamic potentials, microcanonical and canonical distributions, and evolution in the phase space, The inevitability of irreversibility, basics of information theory, applications of information theory, new second law of thermodynamics and quantum information.

Author(s): Gregory Falkovich

165 Pages

Information Theory for Data Communications and Processing

The PDF covers the following topics related to Information Theory : Information Theory for Data Communications and Processing, On the Information Bottleneck Problems: Models, Connections,Applications and Information Theoretic Views, Variational Information Bottleneck for Unsupervised Clustering: Deep Gaussian Mixture Embedding, Asymptotic Rate-Distortion Analysis of Symmetric Remote Gaussian Source Coding: Centralized Encoding vs. Distributed Encoding, Non-Orthogonal eMBB-URLLC Radio Access for Cloud Radio Access Networks with Analog Fronthauling, Robust Baseband Compression Against Congestion in Packet-Based Fronthaul Networks Using Multiple Description Coding, Amplitude Constrained MIMO Channels: Properties of Optimal Input Distributions and Bounds on the Capacity, Quasi-Concavity for Gaussian Multicast Relay Channels, Gaussian Multiple Access Channels with One-Bit Quantizer at the Receiver, Efficient Algorithms for Coded Multicasting in Heterogeneous Caching Networks, Cross-Entropy Method for Content Placement and User Association in Cache-Enabled Coordinated Ultra-Dense Networks, Symmetry, Outer Bounds, and Code Constructions: A Computer-Aided Investigation on the Fundamental Limits of Caching.

Author(s): Shlomo Shamai, Abdellatif Zaidi

296 Pages

Information Theory and its applications in theory of computation

This note covers the following topics: Entropy, Kraft's inequality, Source coding theorem, conditional entropy, mutual information, KL-divergence and connections, KL-divergence and Chernoff bounds, Data processing and Fano's inequalities, Asymptotic Equipartition Property, Universal source coding: Lempel-Ziv algorithm and proof of its optimality, Source coding via typical sets and universality, joint typicality and joint AEP, discrete channels and channel capacity, Proof of Noisy channel coding theorem, Constructing capacity-achieving codes via concatenation, Polarization, Arikan's recursive construction of a polarizing invertible transformation, Polar codes construction, Bregman's theorem, Shearer's Lemma and applications, Source coding and Graph entropy, Monotone formula lower bounds via graph entropy, Optimal set Disjointness lower bound and applications, Compression of arbitrary communication protocols, Parallel repetition of 2-prover 1-round games.

Author(s): Venkatesan Guruswami and Mahdi Cheraghchi

NA Pages

Information Theory by Yao Xie

This note will explore the basic concepts of information theory. It is highly recommended for students planning to delve into the fields of communications, data compression, and statistical signal processing. Topics covered includes: Entropy and mutual information, Chain rules and inequalities, Data processing, Fano's inequality, Asymptotic equipartition property, Entropy rate, Source coding and Kraft inequality, Optimal code length and roof code, Huffman codes, Shannon-Fano-Elias and arithmetic codes, Maximum entropy, Channel capacity, Channel coding theorem, Differential entropy, Gaussian channel, Parallel Gaussian channel and water-filling, Quantization and rate-distortion.

Author(s): Yao Xie

NA Pages