Mathematical Marvels: Demystifying the Back-propagation Algorithm in Machine Learning

Unlocking the Secrets of Back-propagation: Delving into the Mathematical Core of Machine Learning

In this article, we will be discussing the step-by-step approach for forward pass and backward propagation. I will be using Overleaf for mathematics, as it is discussed in depth.

A 3 layer neural network will help us understand the propagation in a simplified manner.

3-layer Neural Network

Let’s start by calculating the values of the hidden layer and the output layer.

Hidden layer: h1 and h2

Output layer: y1 and y2

Error in the node y1 and y2, add up to get the total error.

Backward propagation