# Optimization

#### Notes

Name | Tags |
---|---|

Optimization basics | Basics |

Quadratic Forms & PSD | Basics |

Hessian Matrix | Basics |

Jacobian Matrix | Basics |

Matrix Calculus: fundamentals | Backprop |

Matrix Derivative Practicals | Backprop |

Matrix Calculus: Common Forms | Backprop |

Gradient Techniques (algorithms) | Backprop |

Natural Gradient | Second Order |

Newtonâ€™s Method | Second Order |

Conjugate Gradient Descent | Second Order |

Hessian-Free Gradients | Second Order |

Dual Gradient Descent | Constrained |

Constrained Optimization: an intro | Constrained |

Constrained Optimization | Constrained |

CEM and CMA | Gradient Free |

Overdetermined Linear Equations and Least squares (left inverses) | Closed Form |

Gauss-Newton Method + Regularized Least Squares | Closed Form |

Underdetermined Linear Equations (right inverses) | Closed Form |