This text emphasizes nonlinear models for a course in time series analysis. After introducing stochastic processes, Markov chains, Poisson processes, and ARMA models, the authors cover functional autoregressive, ARCH, threshold AR, and discrete time series models as well as several complementary approaches. They discuss the main limit theorems for Markov chains, useful inequalities, statistical techniques to infer model parameters, and GLMs. Moving on to HMM models, the book examines filte...

This book covers the classical theory of Markov chains on general state-spaces as well as many recent developments. The theoretical results are illustrated by simple examples, many of which are taken from Markov Chain Monte Carlo methods. The book is self-contained, while all the results are carefully and concisely proven. Bibliographical notes are added at the end of each chapter to provide an overview of the literature.Part I lays the foundations of the theory of Markov chain on ...

This book contains contributions from the participants of the international conference “Foundations of Modern Statistics” which took place at Weierstrass Institute for Applied Analysis and Stochastics (WIAS), Berlin, during November 6–8, 2019, and at Higher School of Economics (HSE University), Moscow, during November 30, 2019. The events were organized in honor of Professor Vladimir Spokoiny on the occasion of his 60th birthday. Vladimir Spokoiny has pioneered the field of adaptive statis...

This open-access textbook's significant contribution is the unified derivation of data-assimilation techniques from a common fundamental and optimal starting point, namely Bayes' theorem. Unique for this book is the "top-down" derivation of the assimilation methods. It starts from Bayes theorem and gradually introduces the assumptions and approximations needed to arrive at today's popular data-assimilation methods. This strategy is the opposite of most textbooks and reviews on data assimil...

This open access book provides a comprehensive treatment of recent developments in kernel-based identification that are of interest to anyone engaged in learning dynamic systems from data. The reader is led step by step into understanding of a novel paradigm that leverages the power of machine learning without losing sight of the system-theoretical principles of black-box identification. The authors’ reformulation of the identification problem in the light of regularization theory not only...

Together with the fundamentals of probability, random processes and statistical analysis, this insightful book also presents a broad range of advanced topics and applications. There is extensive coverage of Bayesian vs. frequentist statistics, time series and spectral representation, inequalities, bound and approximation, maximum-likelihood estimation and the expectation-maximization (EM) algorithm, geometric Brownian motion and Itô process. Applications such as hidden Markov models (HMM),...

The Likelihood plays a key role in both introducing general notions of statistical theory, and in developing specific methods. This book introduces likelihood-based statistical theory and related methods from a classical viewpoint, and demonstrates how the main body of currently used statistical techniques can be generated from a few key concepts, in particular the likelihood. Focusing on those methods, which have both a solid theoretical background and practical relevance, the author give...

In an attempt to introduce application scientists and graduate students to the exciting topic of positive definite kernels and radial basis functions, this book presents modern theoretical results on kernel-based approximation methods and demonstrates their implementation in various settings. The authors explore the historical context of this fascinating topic and explain recent advances as strategies to address long-standing problems. Examples are drawn from fields as diverse as function ...

Since the time of Isaac Newton, physicists have used mathematics to describe the behavior of matter of all sizes, from subatomic particles to galaxies. In the past three decades, as advances in molecular biology have produced an avalanche of data, computational and mathematical techniques have also become necessary tools in the arsenal of biologists. But while quantitative approaches are now providing fundamental insights into biological systems, the college curriculum for biologists has n...

Linear Models and the Relevant Distributions and Matrix Algebra provides in-depth and detailed coverage of the use of linear statistical models as a basis for parametric and predictive inference. It can be a valuable reference, a primary or secondary text in a graduate-level course on linear models, or a resource used (in a course on mathematical statistics) to illustrate various theoretical concepts in the context of a relatively complex setting of great practical importa...

This book provides a straightforward look at the concepts, algorithms and advantages of Bayesian Deep Learning and Deep Generative Models. Starting from the model-based approach to Machine Learning, the authors motivate Probabilistic Graphical Models and show how Bayesian inference naturally lends itself to this framework. The authors present detailed explanations of the main modern algorithms on variational approximations for Bayesian inference in neural networks. Each algorithm of this s...

This state-of-the-art survey is dedicated to the memory of Emmanuil Markovich Braverman (1931-1977), a pioneer in developing machine learning theory.The 12 revised full papers and 4 short papers included in this volume were presented at the conference "Braverman Readings in Machine Learning: Key Ideas from Inception to Current State" held in Boston, MA, USA, in April 2017, commemorating the 40th anniversary of Emmanuil Braverman's decease. The papers present an overview of some of ...