Statistical tests can be performed to identify which attributes have the strongest link to the output variable. The SelectKBest class in the scikit-learn library can be used with a variety of statistical tests to choose a certain number of features.<\/p>\n\n\n\n
The chi-squared (chi2) statistical test for non-negative features is used in the example below to select 10 of the top features from the Mobile Price Range Prediction Dataset.<\/p>\n\n\n
\nimport pandas as pd\nimport numpy as np\nfrom sklearn.feature_selection import SelectKBest\nfrom sklearn.feature_selection import chi2\ndata = pd.read_csv("C:\/\/Users\/\/Intel\/\/Documents\/\/mobile_price_train.csv")\nX = data.iloc[:,0:20] #independent variable columns\ny = data.iloc[:,-1] #target variable column (price range)\n\n#extracting top 10 best features by applying SelectKBest class\nbestfeatures = SelectKBest(score_func=chi2, k=10)\nfit = bestfeatures.fit(X,y)\ndfscores = pd.DataFrame(fit.scores_)\ndfcolumns = pd.DataFrame(X.columns)\n\n#concat two dataframes\nfeatureScores = pd.concat([dfcolumns,dfscores],axis=1)\nfeatureScores.columns = ['Specs','Score'] #naming the dataframe columns\nprint(featureScores.nlargest(10,'Score')) #printing 10 best features\n<\/pre><\/div>\n\n\nOutput:<\/p>\n\n\n\n
Specs Score\n13 ram 931267.519053\n11 px_height 17363.569536\n0 battery_power 14129.866576\n12 px_width 9810.586750\n8 mobile_wt 95.972863\n6 int_memory 89.839124\n15 sc_w 16.480319\n16 talk_time 13.236400\n4 fc 10.135166\n14 sc_h 9.614878<\/pre>\n\n\n\n2. Feature Importance<\/h3>\n\n\n\n The feature importance attribute of the model can be used to obtain the feature importance of each feature in your dataset.<\/p>\n\n\n\n
Feature importance assigns a score to each of your data’s features; the higher the score, the more important or relevant the feature is to your output variable. We will use Extra Tree Classifier in the below example to extract the top 10 features for the dataset because Feature Importance is an inbuilt class that comes with Tree-Based Classifiers.<\/p>\n\n\n
\nimport pandas as pd\nimport numpy as np\ndata = pd.read_csv("C:\/\/Users\/\/Intel\/\/Documents\/\/mobile_price_train.csv")\nX = data.iloc[:,0:20] #independent variable columns\ny = data.iloc[:,-1] #target variable column (price range)\nfrom sklearn.ensemble import ExtraTreesClassifier\nimport matplotlib.pyplot as plt\nmodel = ExtraTreesClassifier()\nmodel.fit(X,y)\nprint(model.feature_importances_) \n\n#plot the graph of feature importances \nfeat_importances = pd.Series(model.feature_importances_, index=X.columns)\nfeat_importances.nlargest(10).plot(kind='barh')\nplt.show()\n<\/pre><\/div>\n\n\nOutput:<\/p>\n\n\n\n
[0.05945479 0.02001093 0.03442302 0.0202319 0.03345326 0.01807593\n 0.03747275 0.03450839 0.03801611 0.0335925 0.03590059 0.04702123\n 0.04795976 0.38014236 0.03565894 0.03548119 0.03506038 0.01391338\n 0.01895962 0.02066298]<\/pre>\n\n\n\n3. Correlation Statistics with Heatmap<\/h3>\n\n\n\n Correlation describes the relationship between the features and the target variable.
Positive<\/strong>: An increase in one feature’s value improves the value of the target variable or <\/li>Negative:<\/strong> An increase in one feature’s value decreases the value of the target variable.<\/li><\/ul>\n\n\n\nWe will plot a heatmap of correlated features using the Seaborn library to find which features are most connected to the target variable.<\/p>\n\n\n
\nimport pandas as pd\nimport numpy as np\nimport seaborn as sns\ndata = pd.read_csv("C:\/\/Users\/\/Intel\/\/Documents\/\/mobile_price_train.csv")\nX = data.iloc[:,0:20] #independent variable columns\ny = data.iloc[:,-1] #targetvariable column (price range)\n\n#obtain the correlations of each features in dataset\ncorrmat = data.corr()\ntop_corr_features = corrmat.index\nplt.figure(figsize=(20,20))\n#plot heat map\ng=sns.heatmap(data[top_corr_features].corr(),annot=True,cmap="RdYlGn")\n<\/pre><\/div>\n\n\nOutput:<\/p>\n\n\n\n
Go to the last row and look at the price range. You will see all the features correlated to the price range. ‘ram’ is the feature that is highly correlated to the price range, followed by features such as battery power, pixel height, and width.m_dep, clock_speed, and n_cores are the features least correlated with the price range. <\/p>\n\n\n\n
Conclusion<\/strong><\/h2>\n\n\n\nWe learned how to choose relevant features from data using the Univariate Selection approach, feature importance, and the correlation matrix in this article. Choose the method that suits your case the best and use it to improve your model\u2019s accuracy.<\/p>\n","protected":false},"excerpt":{"rendered":"
This article is a little on the advanced side. We’ll discuss feature selection in Python for training machine learning models. It’s important to identify the important features from a dataset and eliminate the less important features that don’t improve model accuracy. Model performance can be harmed by features that are irrelevant or only partially relevant. […]<\/p>\n","protected":false},"author":39,"featured_media":26948,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[9],"tags":[],"class_list":["post-26618","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-examples"],"blocksy_meta":[],"_links":{"self":[{"href":"https:\/\/www.askpython.com\/wp-json\/wp\/v2\/posts\/26618","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.askpython.com\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.askpython.com\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.askpython.com\/wp-json\/wp\/v2\/users\/39"}],"replies":[{"embeddable":true,"href":"https:\/\/www.askpython.com\/wp-json\/wp\/v2\/comments?post=26618"}],"version-history":[{"count":0,"href":"https:\/\/www.askpython.com\/wp-json\/wp\/v2\/posts\/26618\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.askpython.com\/wp-json\/wp\/v2\/media\/26948"}],"wp:attachment":[{"href":"https:\/\/www.askpython.com\/wp-json\/wp\/v2\/media?parent=26618"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.askpython.com\/wp-json\/wp\/v2\/categories?post=26618"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.askpython.com\/wp-json\/wp\/v2\/tags?post=26618"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}
![\"Feature](\"https:\/\/www.askpython.com\/wp-content\/uploads\/2022\/01\/Feature-Importance-Plot.png\")
<\/figure><\/div>\n\n\n\n![\"HEAT](\"https:\/\/www.askpython.com\/wp-content\/uploads\/2022\/01\/HEAT-MAP-968x1024.png\")
<\/figure><\/div>\n\n\n\n