Before going into this let's understand it by a simple example- 

Now we can predict our Y_hat for 3 (estimated y). Don't be stress if you don't understand these mathematics explanation read    this. 

Let's try it. 

Here numpy essential fom mathematical operations and matplotlib is to plot the graph of our linear model.

here 'A' is a matrix with normal distribution mu = 2 & alpha = 0.01

Here we are defining x_bar and Y_bar (Xmean/Ymean). see the algorithm below. 

Here one can see that we need to follow above step to complete simple linear regression with matrix. Therefore, we are taking mean of X & Y. 

Now we are going to calculate our betas (parameter).

This is how we calculate our betas (parameter). One can simply relate to the code and formula. 

this is our current beta. It can be different for you, since we are randomly initializing the data e.i. matrix. 

Here we found out our prediction for ground truth e.t. Y_hat. Let's Plot the training points from matrix A and predicted values in the form of line graph.

At the end let's use numpy.linalg lstsq to replace step 2 for learning values of .Beta_0 and .Beta_1

here you can see that we have optimize the same beta as we did before manually. Hope It will help you all to understand simple regression better. 

1) Logistic Regression

2) Gradient Descent 

3) Stochastic Gradient Descent