Linear Regression from Scratch<\/h2>\n\n\n\n
In this article, we will implement the Linear Regression from scratch using only Numpy<\/a>. <\/p>\n\n\n\n While there are many loss functions to implement, We will use the Mean Squared Error<\/a> function as our loss function. <\/p>\n\n\n\n A mean squared error function as the name suggests is the mean of squared sum of difference between true and predicted value. <\/p>\n\n\n\n As the predicted value of y depends on the slope and constant, hence our goal is to find the values for slope and constant that minimize the loss function or in other words, minimize the difference between y predicted and true values. <\/p>\n\n\n\n Optimization algorithms are used to find the optimal set of parameters given a training dataset that minimizes the loss function, in our case we need to find the optimal value of slope (m) and constant (b). <\/p>\n\n\n\n One such Algorithm is Gradient Descent. <\/strong><\/p>\n\n\n\n Gradient descent is by far the most popular optimization algorithm used in machine learning.<\/p>\n\n\n\n Using gradient descent we iteratively calculate the gradients of the loss function with respect to the parameters and keep on updating the parameters till we reach the local minima. <\/p>\n\n\n\n Let’s understand how the gradient descent algorithm works behind the scenes. <\/p>\n\n\n\n Step-1 Initializing the parameters<\/strong><\/p>\n\n\n\n Here, we need to initialize the values for our parameters. Let’s keep We will also need a learning rate to determine the step size at each iteration while moving toward a minimum value of our loss function.<\/p>\n\n\n\n Step -2 Calculate the Partial Derivatives with respect to parameters<\/strong><\/p>\n\n\n\n Here we partially differentiate our loss function with respect to the parameters we have. <\/p>\n\n\n\n Step – 3 Updating the parameters<\/strong><\/p>\n\n\n\n Now, we update the values of our parameters using the equations given below:<\/p>\n\n\n\n The updated values for our parameters will be the values with which, each step minimizes our loss function and reduces the difference between the true and predicted values.<\/p>\n\n\n\n Repeat the process to reach a point of local minima. <\/p>\n\n\n\n Now that we have an idea about how Linear regression can be implemented using Gradient descent, let’s code it in Python.<\/p>\n\n\n\n We will define We create an instance of our The With the Now as we created our class let’s test in on the data. Learn more about how to split training and testing data sets<\/a>. You can find the datasets and other resources used within this tutorial here<\/a>.<\/p>\n\n\n Works fine !<\/p>\n\n\n\n This article was all about how we can make a Linear Regression model from scratch using only Numpy. The goal of this tutorial was to give you a deeper sense of what Linear Regression actually is and how it works.<\/p>\n\n\n\n Till we meet next time.<\/p>\n\n\n\n Happy Learning!<\/p>\n","protected":false},"excerpt":{"rendered":" In this article, we’ll learn to implement Linear regression from scratch using Python. Linear regression is a basic and most commonly used type of predictive analysis. It is used to predict the value of a variable based on the value of another variable. The variable we want to predict is called the dependent variable. The […]<\/p>\n","protected":false},"author":16,"featured_media":11845,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[9],"tags":[],"class_list":["post-11152","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-examples"],"blocksy_meta":[],"_links":{"self":[{"href":"https:\/\/www.askpython.com\/wp-json\/wp\/v2\/posts\/11152","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.askpython.com\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.askpython.com\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.askpython.com\/wp-json\/wp\/v2\/users\/16"}],"replies":[{"embeddable":true,"href":"https:\/\/www.askpython.com\/wp-json\/wp\/v2\/comments?post=11152"}],"version-history":[{"count":0,"href":"https:\/\/www.askpython.com\/wp-json\/wp\/v2\/posts\/11152\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.askpython.com\/wp-json\/wp\/v2\/media\/11845"}],"wp:attachment":[{"href":"https:\/\/www.askpython.com\/wp-json\/wp\/v2\/media?parent=11152"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.askpython.com\/wp-json\/wp\/v2\/categories?post=11152"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.askpython.com\/wp-json\/wp\/v2\/tags?post=11152"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}1. Understanding Loss Function<\/h3>\n\n\n\n
![\"MSE\"](\"https:\/\/www.askpython.com\/wp-content\/uploads\/2020\/12\/MSE-300x83.gif\")
2. Optimization Algorithm<\/h3>\n\n\n\n
3. Steps to Implement Gradient Descent<\/h3>\n\n\n\n
slope = 0<\/code> and constant = 0<\/code>. <\/p>\n\n\n\n![\"Gradient](\"https:\/\/www.askpython.com\/wp-content\/uploads\/2020\/12\/Gradient-of-slope-and-bias.png\")
![\"Updating](\"https:\/\/www.askpython.com\/wp-content\/uploads\/2020\/12\/Updating-Parameters.jpg\")
4. Implementing Linear Regression from Scratch in Python<\/h3>\n\n\n\n
LinearRegression<\/code> class with two methods .fit( )<\/code> and .predict( )<\/code><\/p>\n\n\n\n#Import required modules\nimport numpy as np\n\n#Defining the class\nclass LinearRegression:\n def __init__(self, x , y):\n self.data = x\n self.label = y\n self.m = 0\n self.b = 0\n self.n = len(x)\n \n def fit(self , epochs , lr):\n \n #Implementing Gradient Descent\n for i in range(epochs):\n y_pred = self.m * self.data + self.b\n \n #Calculating derivatives w.r.t Parameters\n D_m = (-2\/self.n)*sum(self.data * (self.label - y_pred))\n D_b = (-1\/self.n)*sum(self.label-y_pred)\n \n #Updating Parameters\n self.m = self.m - lr * D_m\n self.c = self.b - lr * D_c\n \n def predict(self , inp):\n y_pred = self.m * inp + self.b \n return y_pred\n<\/pre><\/div>\n\n\n
LinearRegression<\/code> class with training data as the input to the class and initialize the bias and constant values as 0.<\/p>\n\n\n\n.fit( )<\/code> method in our class implements Gradient Descent where with each iteration we calculate the partial derivatives of the function with respect to parameters and then update the parameters using the learning rate and the gradient value. <\/p>\n\n\n\n.predict( )<\/code> method we are simply evaluating the function y = m * x + b<\/code> , using the optimal values of our parameters, in other words, this method estimates the line of best fit.<\/p>\n\n\n\n4. Testing the Linear Regression Model<\/h3>\n\n\n\n
\n#importing Matplotlib for plotting\nimport matplotlib.pyplot as plt\n\n#Loding the data\ndf = pd.read_csv('data_LinearRegression.csv')\n\n#Preparing the data\nx = np.array(df.iloc[:,0])\ny = np.array(df.iloc[:,1])\n\n#Creating the class object\nregressor = LinearRegression(x,y)\n\n#Training the model with .fit method\nregressor.fit(1000 , 0.0001) # epochs-1000 , learning_rate - 0.0001\n\n#Prediciting the values\ny_pred = regressor.predict(x)\n\n#Plotting the results\nplt.figure(figsize = (10,6))\nplt.scatter(x,y , color = 'green')\nplt.plot(x , y_pred , color = 'k' , lw = 3)\nplt.xlabel('x' , size = 20)\nplt.ylabel('y', size = 20)\nplt.show()\n<\/pre><\/div>\n\n\n![\"Prediction](\"https:\/\/www.askpython.com\/wp-content\/uploads\/2020\/12\/Prediction-Linear-Regression-scaled.jpeg\")
Conclusion<\/h2>\n\n\n\n