Information Theory, Inference, and Learning Algorithms

Más información sobre el libro

Information theory and inference, typically taught separately, are combined in this engaging textbook, central to various fields such as communication, signal processing, data mining, machine learning, and bioinformatics. The text introduces theory alongside practical applications, covering communication systems like arithmetic coding for data compression and sparse-graph codes for error correction. A comprehensive toolbox of inference techniques, including message-passing algorithms, Monte Carlo methods, and variational approximations, is developed alongside applications in clustering, convolutional codes, independent component analysis, and neural networks. The book also explores advanced error-correcting codes, such as low-density parity-check codes, turbo codes, and digital fountain codes, which are essential for modern satellite communications, disk drives, and data broadcasting. Richly illustrated with worked examples and over 400 exercises, some with detailed solutions, this groundbreaking work is suitable for self-study as well as undergraduate and graduate courses. Interludes on crosswords, evolution, and sex add an entertaining touch. Overall, this textbook serves as an invaluable resource for students and professionals in diverse fields, including computational biology, financial engineering, and machine learning.