Mathematics for AI & Data Science

Given the dependence of Machine Learning and Deep Learning courses on concepts of statistics and probability, linear algebra, multivariate differential calculus, and optimization, this course aims to familiarize students with these concepts from both theoretical and practical perspectives.

1. Linear Algebra (4 sessions)
   • Vector space and linear independence
   • Basis and rank
   • Linear mapping and matrices
   • Solving systems of linear equations, linear regression
   • Inner product and norm
   • Orthogonal projection
   • Singular Value Decomposition
   • Eigenvector and eigenvalue decomposition of a matrix
2. Differential Calculus (4 sessions)
   • Multivariable functions
   • Partial derivatives and gradient
   • Chain rule of derivatives
   • Automatic differentiation
   • Derivatives of vector functions
   • Taylor expansion of multivariable functions
3. Statistics and Probability (6 sessions)
   • Probability concepts
   • Random variables
   • Sum rule and Law of Total Probability
   • Bayes' rule
   • Probability distribution concept and types (PMF and PDF)
   • Independence of random variables
   • Sample distributions (categorical and Gaussian)
   • Joint distribution function of multiple random variables
   • Moments of single and multiple random variables (mean, variance, covariance, correlation coefficient)
   • Multivariate Gaussian distribution
   • Central Limit Theorem and Law of Large Numbers
   • Estimation theory (Maximum Likelihood and Bayesian)
4. Optimization (4 sessions)
   • Numerical optimization using gradient (Gradient Descent)
   • Constrained optimization, Lagrange multipliers and duality
   • Convex optimization

1. Assignments: 20%
2. Quizzes: 20%
3. Final exam: 60%