Deep Learning Architectures

This book describes how neural networks operate from the mathematical point of view. As a result, neural networks can be interpreted both as function universal approximators and information processors. The book bridges the gap between ideas and concepts of neural networks, which are used nowadays at an intuitive level, and the precise modern mathematical language, presenting the best practices of the former and enjoying the robustness and elegance of the latter.This book can be used in a graduate course in deep learning, with the first few parts being accessible to senior undergraduates. In addition, the book will be of wide interest to machine learning researchers who are interested in a theoretical understanding of the subject.

Introductory Problems.- Activation Functions.- Cost Functions.- Finding Minima Algorithms.- Abstract Neurons.- Neural Networks.- Approximation Theorems.- Learning with One-dimensional Inputs.- Universal Approximators.- Exact Learning.- Information Representation.- Information Capacity Assessment.- Output Manifolds.- Neuromanifolds.- Pooling.- Convolutional Networks.- Recurrent Neural Networks.- Classification.- Generative Models.- Stochastic Networks.- Hints and Solutions.