VECTOR CALCULUS

Functions of several variables, Differentiation and partial differentials, gradients of vector-valued functions, gradients of matrices, useful identities for computing gradients, linearization and multivariate Taylor series.

(8 hours)

(RBT Levels: L1, L2 and L3)

APPLICATIONS OF VECTOR CALCULUS

Backpropagation and automatic differentiation, gradients in a deep network, The Gradient of Quadratic Cost, Descending the Gradient of Cost, The Gradient of Mean Squared Error.

(8 hours)

(RBT Levels: L1, L2 and L3)

Convex Optimization-1

Local and global optima, convex sets and functions separating hyperplanes, application of Hessian matrix in optimization, Optimization using gradient descent, Sequential search 3-point search and Fibonacci search.

(8 hours)

(RBT Levels: L1, L2 and L3)

Convex Optimization-2

Unconstrained optimization -Method of steepest ascent/descent, NR method, Gradient descent, Mini batch gradient descent, Stochastic gradient descent.

(8 hours)

(RBT Levels: L1, L2 and L3)

Advanced Optimization

Momentum-based gradient descent methods: Adagrad, RMSprop and Adam.

Non-Convex Optimization: Convergence to Critical Points, Saddle-Point methods.

(8 hours)

(RBT Levels: L1, L2 and L3)