Maths for Machine Learning

Reasons why Maths is important for machine learning:

Nowadays many people are planning the transition to AI/ML/Data Science world which is very encouraging and matching the pace of changing the world. The goal of machine learning is to design general-purpose methodologies to extract valuable models from data, ideally without a lot of domain-specific expertise. To achieve this goal, we design models that are generally related to the process that generates data, similar to the model of the dataset provided to us.

A model learns from data if its performance on a given task improves after taking the data into account. The goal is to find good models that generalize well to still invisible data, which we might care about in the future. Learning can be understood as a means of learning to automatically find patterns and structure in data by optimizing model parameters.

To stay in the field of machine learning for a longer time, I believe that the Maths for machine learning is very important for understanding the fundamentals on which more complex machine learning systems are built.

There are many more reasons why Maths for machine learning is important, some of the reasons are:

• Selecting the right algorithm which includes considerations on the accuracy, learning time, model complexity, number of parameters and number of features
• Choice of parameter settings and validation strategies.
• Estimation of the correct confidence interval and uncertainty.

The Maths you need for Machine Learning:

The main question when trying to understand an interdisciplinary field such as Machine Learning is the amount of mathematics necessary and the level of mathematics needed to understand these techniques. The answer to this question is multidimensional and depends on the level and interest of the individual. Research in mathematical formulations and theoretical advancement of Machine Learning is ongoing and some researchers are working on more advanced techniques. The following are believed to be the minimum level of mathematics needed to be a Machine Learning Scientist/Engineer and the importance of each mathematical concept.

• Linear Algebra: In ML, Linear Algebra comes up everywhere. Topics such as Principal Component Analysis (PCA), Singular Value Decomposition (SVD), Eigen decomposition of a matrix, LU Decomposition, QR Decomposition/Factorization, Symmetric Matrices, Orthogonalization & Orthonormalization, Matrix Operations, Projections, Eigenvalues & Eigenvectors, Vector Spaces and Norms, are needed for understanding the optimization methods used for machine learning.
• Probability Theory and Statistics: Machine Learning and Statistics aren’t very different fields. Actually, someone recently defined Machine Learning as ‘doing statistics on a Mac’. Some of the fundamental Statistical and Probability Theory needed for ML are Combinatorics, Probability Rules & Axioms, Bayes’ Theorem, Random Variables, Variance and Expectation, Conditional and Joint Distributions, Standard Distributions (Bernoulli, Binomial, Multinomial, Uniform and Gaussian), Moment Generating Functions, Maximum Likelihood Estimation (MLE), Prior and Posterior, Maximum a Posteriori Estimation (MAP) and Sampling Methods.
• Multivariate Calculus: Some of the necessary topics include Differential and Integral Calculus, Partial Derivatives, Vector-Values Functions, Directional Gradient, Hessian, Jacobian, Laplacian and Lagrangian Distribution.
• Algorithms and Complex Optimizations: This is important for understanding the computational efficiency and scalability of our Machine Learning Algorithm, and for exploiting sparsity in our datasets. Knowledge of data structures (Binary Trees, Hashing, Heap, Stack, etc), Dynamic Programming, Randomized & Sublinear Algorithm, Graphs, Gradient/Stochastic Descents and Primal-Dual methods are needed.
• Others: This comprises of other Mathematics topics not covered in the four major areas described above. They include Real and Complex Analysis (Sets and Sequences, Topology, Metric Spaces, Single-Valued and Continuous Functions, Limits), Information Theory (Entropy, Information Gain), Function Spaces and Manifolds.