Non-Linear Models for Regression

So far we discussed about the linear models of Regression like Linear Regression [OLS] and SVM [with kernel as linear].

Here we will talk about the non-linear regression models like

1. Polynomial Regression
2. SVM
   • kernel = [poly , rbf, sigmoid]
3. K-NN (K Nearest Neighbors)
4. Decision Trees
5. Bagging
   • Random Forest
6. Boosting
   • AdaBoost
   • GBM
   • XGBoost
   • CatBoost
   • LightGBM

Polynomial Regression is like linear regression but the equation for the best fit line is not a straight/linear line [y = m*x + c], rather its a polynomial equation of degree 2 or more resulting in a curve [y = m*x^2 + c], etc.
Refer below different polynomial equation forms:

SVM when used with kernel function as poly or RBF then its used to separate the data non-linearly either using a polynomial equation [ curved line] or Radial Basis function to separate the classes by adding more/higher dimensions. 
Refer below for the non-linearly separable data and use of SVM with RBF to create a hyperplane for separation of 2 classes:

KNN stands for K Nearest Neighbors where K has to a odd number like 1, 3, 5 and so on. It's like you want to take decision based on the choices made by 3 people similar to you. If majority of them have a particular choice you also opt for it. 

KNN Model explained in detail here

We will discuss more about the Tree Based models in detail under the upcoming blog posts