Chapter # 09
Dimension Reduction & PCA

Introduction

This document is prepared using Quarto in RStudio. The Quarto file (.qmd) can be converted to notebook (.ipynb) by running the code - quarto convert filename.qmd - in the terminal of RStudio. Similarly a Python notebook file (.ipynb) can be converted into Quarto (.qmd) by running the code - quarto convert filename.ipynb in the terminal of RStudio. More about the conversion can be found here.

Useful Youtube video link for PCA

What is Principal Component Analysis (PCA)?

Working Directory

import os
os.getcwd()
for files in os.listdir():
  print(files)

# Ignore Warnings 
import warnings 
warnings.filterwarnings('ignore')

Loading Necessary Python Packages

# For Data Manipulation & Analysis
import pandas as pd
import numpy as np
# For visualization 
import matplotlib.pyplot as plt 
import seaborn as sns
# For Machine Learning 
import sklearn 

Loading Dataset

from sklearn import datasets
cancer = datasets.load_breast_cancer()
dir(cancer)
cancer.keys()

dict_keys(['data', 'target', 'frame', 'target_names', 'DESCR', 'feature_names', 'filename', 'data_module'])

df = pd.DataFrame(cancer.data, columns = cancer.feature_names)
df.head()

	mean radius	mean texture	mean perimeter	mean area	mean smoothness	mean compactness	mean concavity	mean concave points	mean symmetry	mean fractal dimension	...	worst radius	worst texture	worst perimeter	worst area	worst smoothness	worst compactness	worst concavity	worst concave points	worst symmetry	worst fractal dimension
0	17.99	10.38	122.80	1001.0	0.11840	0.27760	0.3001	0.14710	0.2419	0.07871	...	25.38	17.33	184.60	2019.0	0.1622	0.6656	0.7119	0.2654	0.4601	0.11890
1	20.57	17.77	132.90	1326.0	0.08474	0.07864	0.0869	0.07017	0.1812	0.05667	...	24.99	23.41	158.80	1956.0	0.1238	0.1866	0.2416	0.1860	0.2750	0.08902
2	19.69	21.25	130.00	1203.0	0.10960	0.15990	0.1974	0.12790	0.2069	0.05999	...	23.57	25.53	152.50	1709.0	0.1444	0.4245	0.4504	0.2430	0.3613	0.08758
3	11.42	20.38	77.58	386.1	0.14250	0.28390	0.2414	0.10520	0.2597	0.09744	...	14.91	26.50	98.87	567.7	0.2098	0.8663	0.6869	0.2575	0.6638	0.17300
4	20.29	14.34	135.10	1297.0	0.10030	0.13280	0.1980	0.10430	0.1809	0.05883	...	22.54	16.67	152.20	1575.0	0.1374	0.2050	0.4000	0.1625	0.2364	0.07678

5 rows × 30 columns

df.shape
print ("The number of rows and columns in the dataset is {} and {} respectively".format(df.shape[0], df.shape[1]))

The number of rows and columns in the dataset is 569 and 30 respectively

Metadata of the Dataset

df.info()

<class 'pandas.core.frame.DataFrame'>
RangeIndex: 569 entries, 0 to 568
Data columns (total 30 columns):
 #   Column                   Non-Null Count  Dtype  
---  ------                   --------------  -----  
 0   mean radius              569 non-null    float64
 1   mean texture             569 non-null    float64
 2   mean perimeter           569 non-null    float64
 3   mean area                569 non-null    float64
 4   mean smoothness          569 non-null    float64
 5   mean compactness         569 non-null    float64
 6   mean concavity           569 non-null    float64
 7   mean concave points      569 non-null    float64
 8   mean symmetry            569 non-null    float64
 9   mean fractal dimension   569 non-null    float64
 10  radius error             569 non-null    float64
 11  texture error            569 non-null    float64
 12  perimeter error          569 non-null    float64
 13  area error               569 non-null    float64
 14  smoothness error         569 non-null    float64
 15  compactness error        569 non-null    float64
 16  concavity error          569 non-null    float64
 17  concave points error     569 non-null    float64
 18  symmetry error           569 non-null    float64
 19  fractal dimension error  569 non-null    float64
 20  worst radius             569 non-null    float64
 21  worst texture            569 non-null    float64
 22  worst perimeter          569 non-null    float64
 23  worst area               569 non-null    float64
 24  worst smoothness         569 non-null    float64
 25  worst compactness        569 non-null    float64
 26  worst concavity          569 non-null    float64
 27  worst concave points     569 non-null    float64
 28  worst symmetry           569 non-null    float64
 29  worst fractal dimension  569 non-null    float64
dtypes: float64(30)
memory usage: 133.5 KB

df.isnull().sum()

mean radius                0
mean texture               0
mean perimeter             0
mean area                  0
mean smoothness            0
mean compactness           0
mean concavity             0
mean concave points        0
mean symmetry              0
mean fractal dimension     0
radius error               0
texture error              0
perimeter error            0
area error                 0
smoothness error           0
compactness error          0
concavity error            0
concave points error       0
symmetry error             0
fractal dimension error    0
worst radius               0
worst texture              0
worst perimeter            0
worst area                 0
worst smoothness           0
worst compactness          0
worst concavity            0
worst concave points       0
worst symmetry             0
worst fractal dimension    0
dtype: int64

Correlation of the variables

correlation = df.corr()
plt.figure(figsize=(20,17))
sns.heatmap(correlation, vmax=1, square=True,annot=True,cmap='cubehelix')
plt.title('Correlation between different features')
plt.show()

Preprocessing of the Dataset

from sklearn.preprocessing import StandardScaler
scaler = StandardScaler()
scaled_data = scaler.fit_transform(df)
scaled_df = pd.DataFrame(scaled_data, columns = cancer.feature_names)
scaled_df.sample(5)

	mean radius	mean texture	mean perimeter	mean area	mean smoothness	mean compactness	mean concavity	mean concave points	mean symmetry	mean fractal dimension	...	worst radius	worst texture	worst perimeter	worst area	worst smoothness	worst compactness	worst concavity	worst concave points	worst symmetry	worst fractal dimension
200	-0.538858	0.062913	-0.553145	-0.551441	-0.035603	-0.444810	-0.589196	-0.202461	0.611100	-0.378161	...	-0.378793	0.436874	-0.450100	-0.425737	0.461654	-0.318484	-0.645212	-0.100590	-0.376548	-0.122237
20	-0.297446	-0.833008	-0.261106	-0.383638	0.792763	0.429422	-0.541362	-0.459627	0.567289	0.753087	...	-0.366368	-0.844707	-0.332744	-0.439624	-0.051226	0.148443	-0.399099	-0.636110	0.458227	-0.117250
105	-0.288925	-0.867914	-0.196026	-0.354629	3.091407	1.367520	1.485262	1.214661	0.413949	2.001996	...	0.008451	-0.533675	-0.025652	-0.093843	2.359748	0.990056	1.753070	1.278939	0.398369	3.133996
290	0.080293	0.102473	0.167272	-0.011061	-0.625564	1.198852	0.595114	0.441099	-0.356400	1.293194	...	-0.103373	-0.577643	-0.165645	-0.199142	-1.426358	-0.044944	-0.240780	-0.190427	-1.017189	0.224112
508	0.617080	-0.835335	0.524391	0.469024	-0.148756	-0.705393	-0.421085	-0.084840	-0.367353	-0.882828	...	0.217604	-1.289271	0.075620	0.083706	0.132884	-0.751695	-0.371753	0.321186	-0.971891	-0.645362

5 rows × 30 columns

print('The mean and standard deviation of  \
the variable "mean radius" are {} and {} \
respectively.'.format(round(scaled_df['mean radius'].mean(),4), round(scaled_df['mean radius'].std(),4)))

The mean and standard deviation of  the variable "mean radius" are -0.0 and 1.0009 respectively.

Covariance Matrix

mean_vec = np.mean(scaled_data, axis=0)
cov_mat = (scaled_data - mean_vec).T.dot((scaled_data - mean_vec)) / (scaled_data.shape[0]-1)
print('Covariance matrix \n%s' %cov_mat)

Covariance matrix 
[[ 1.00176056e+00  3.24351929e-01  9.99612069e-01  9.89095475e-01
   1.70881506e-01  5.07014640e-01  6.77955036e-01  8.23976636e-01
   1.48001350e-01 -3.12179472e-01  6.80285970e-01 -9.74887767e-02
   6.75358538e-01  7.37159198e-01 -2.22992026e-01  2.06362656e-01
   1.94545531e-01  3.76831225e-01 -1.04504545e-01 -4.27163418e-02
   9.71245907e-01  2.97530545e-01  9.66835698e-01  9.42739295e-01
   1.19826732e-01  4.14190751e-01  5.27839123e-01  7.45524434e-01
   1.64241985e-01  7.07832563e-03]
 [ 3.24351929e-01  1.00176056e+00  3.30113223e-01  3.21650988e-01
  -2.34296930e-02  2.37118951e-01  3.02950254e-01  2.93980713e-01
   7.15266864e-02 -7.65717560e-02  2.76354360e-01  3.87037830e-01
   2.82169018e-01  2.60302460e-01  6.62542133e-03  1.92312595e-01
   1.43545353e-01  1.64139495e-01  9.14323671e-03  5.45533955e-02
   3.53193674e-01  9.13650301e-01  3.58669926e-01  3.44150782e-01
   7.76398084e-02  2.78318729e-01  3.01555198e-01  2.95835766e-01
   1.05192783e-01  1.19415220e-01]
 [ 9.99612069e-01  3.30113223e-01  1.00176056e+00  9.88243612e-01
   2.07643090e-01  5.57916732e-01  7.17396452e-01  8.52475240e-01
   1.83349443e-01 -2.61937255e-01  6.92982910e-01 -8.69138267e-02
   6.94355197e-01  7.46294283e-01 -2.03050882e-01  2.51185131e-01
   2.28483899e-01  4.07933847e-01 -8.17730406e-02 -5.53311534e-03
   9.71183188e-01  3.03571890e-01  9.72095315e-01  9.43207466e-01
   1.50814456e-01  4.56576647e-01  5.64872009e-01  7.72598608e-01
   1.89447989e-01  5.11083511e-02]
 [ 9.89095475e-01  3.21650988e-01  9.88243612e-01  1.00176056e+00
   1.77340047e-01  4.99379326e-01  6.87190545e-01  8.24718286e-01
   1.51559440e-01 -2.83608244e-01  7.33851949e-01 -6.63969041e-02
   7.27907603e-01  8.01494523e-01 -1.67070287e-01  2.12956816e-01
   2.08025659e-01  3.72975776e-01 -7.26242231e-02 -1.99219755e-02
   9.64441062e-01  2.87994769e-01  9.60808165e-01  9.60902082e-01
   1.23740409e-01  3.91097651e-01  5.13508396e-01  7.23287782e-01
   1.43822678e-01  3.74417763e-03]
 [ 1.70881506e-01 -2.34296930e-02  2.07643090e-01  1.77340047e-01
   1.00176056e+00  6.60283643e-01  5.22902753e-01  5.54669988e-01
   5.58756786e-01  5.85821565e-01  3.01997850e-01  6.85268821e-02
   2.96613222e-01  2.46986503e-01  3.32960611e-01  3.19504817e-01
   2.48832996e-01  3.81345895e-01  2.01127852e-01  2.84106006e-01
   2.13495353e-01  3.61353055e-02  2.39273141e-01  2.07082304e-01
   8.06742020e-01  4.73300254e-01  4.35691429e-01  5.03939011e-01
   3.95003689e-01  5.00195447e-01]
 [ 5.07014640e-01  2.37118951e-01  5.57916732e-01  4.99379326e-01
   6.60283643e-01  1.00176056e+00  8.84675460e-01  8.32598309e-01
   6.03702036e-01  5.66364031e-01  4.98349280e-01  4.62861772e-02
   5.49871647e-01  4.56455058e-01  1.35537471e-01  7.40022356e-01
   5.71521303e-01  6.43392594e-01  2.30381479e-01  5.08211293e-01
   5.36257855e-01  2.48569687e-01  5.91249531e-01  5.10500995e-01
   5.66536837e-01  8.67333351e-01  8.17712354e-01  8.17009092e-01
   5.11121711e-01  6.88592503e-01]
 [ 6.77955036e-01  3.02950254e-01  7.17396452e-01  6.87190545e-01
   5.22902753e-01  8.84675460e-01  1.00176056e+00  9.23013194e-01
   5.01548072e-01  3.37376288e-01  6.33037366e-01  7.63525354e-02
   6.61553447e-01  6.18513825e-01  9.87372735e-02  6.71458893e-01
   6.92487233e-01  6.84462839e-01  1.78322604e-01  4.50091771e-01
   6.89448091e-01  3.00406844e-01  7.30849362e-01  6.77177350e-01
   4.49612218e-01  7.56297185e-01  8.85659158e-01  8.62839447e-01
   4.10185014e-01  5.15836457e-01]
 [ 8.23976636e-01  2.93980713e-01  8.52475240e-01  8.24718286e-01
   5.54669988e-01  8.32598309e-01  9.23013194e-01  1.00176056e+00
   4.63311644e-01  1.67211252e-01  6.99278795e-01  2.15173981e-02
   7.11901016e-01  6.91513854e-01  2.77019938e-02  4.91287673e-01
   4.39940250e-01  6.16717994e-01  9.55186580e-02  2.58037239e-01
   8.31779458e-01  2.93267121e-01  8.57430035e-01  8.11055024e-01
   4.53550155e-01  6.68628771e-01  7.53724145e-01  9.11757700e-01
   3.76405667e-01  3.69310185e-01]
 [ 1.48001350e-01  7.15266864e-02  1.83349443e-01  1.51559440e-01
   5.58756786e-01  6.03702036e-01  5.01548072e-01  4.63311644e-01
   1.00176056e+00  4.80766262e-01  3.03913382e-01  1.28278372e-01
   3.14445389e-01  2.24364533e-01  1.87650956e-01  4.22401505e-01
   3.43230240e-01  3.93990298e-01  4.49927276e-01  3.32370277e-01
   1.86054739e-01  9.08102844e-02  2.19554419e-01  1.77505338e-01
   4.27426215e-01  4.74033112e-01  4.34484601e-01  4.31054176e-01
   7.01057885e-01  4.39185353e-01]
 [-3.12179472e-01 -7.65717560e-02 -2.61937255e-01 -2.83608244e-01
   5.85821565e-01  5.66364031e-01  3.37376288e-01  1.67211252e-01
   4.80766262e-01  1.00176056e+00  1.11190486e-04  1.64463005e-01
   3.99000547e-02 -9.03289980e-02  4.02672109e-01  5.60822319e-01
   4.47416643e-01  3.41798745e-01  3.45614805e-01  6.89343077e-01
  -2.54138135e-01 -5.13594647e-02 -2.05512393e-01 -2.32262646e-01
   5.05831058e-01  4.59605900e-01  3.46843443e-01  1.75634121e-01
   3.34606745e-01  7.68647654e-01]
 [ 6.80285970e-01  2.76354360e-01  6.92982910e-01  7.33851949e-01
   3.01997850e-01  4.98349280e-01  6.33037366e-01  6.99278795e-01
   3.03913382e-01  1.11190486e-04  1.00176056e+00  2.13622773e-01
   9.74506342e-01  9.53505869e-01  1.64803858e-01  3.56691450e-01
   3.32942674e-01  5.14250220e-01  2.40990897e-01  2.28154507e-01
   7.16324113e-01  1.95141512e-01  7.20950853e-01  7.52871625e-01
   1.42168410e-01  2.87608629e-01  3.81254678e-01  5.31997297e-01
   9.47092790e-02  4.96466850e-02]
 [-9.74887767e-02  3.87037830e-01 -8.69138267e-02 -6.63969041e-02
   6.85268821e-02  4.62861772e-02  7.63525354e-02  2.15173981e-02
   1.28278372e-01  1.64463005e-01  2.13622773e-01  1.00176056e+00
   2.23563635e-01  1.11763668e-01  3.97942224e-01  2.32107621e-01
   1.95341772e-01  2.30688828e-01  4.12345364e-01  2.80215217e-01
  -1.11886951e-01  4.09722842e-01 -1.02421925e-01 -8.33414586e-02
  -7.37873381e-02 -9.26020990e-02 -6.90776223e-02 -1.19848153e-01
  -1.28440488e-01 -4.57349464e-02]
 [ 6.75358538e-01  2.82169018e-01  6.94355197e-01  7.27907603e-01
   2.96613222e-01  5.49871647e-01  6.61553447e-01  7.11901016e-01
   3.14445389e-01  3.99000547e-02  9.74506342e-01  2.23563635e-01
   1.00176056e+00  9.39306209e-01  1.51341309e-01  4.17055330e-01
   3.63119754e-01  5.57243422e-01  2.66956259e-01  2.44572602e-01
   6.98428059e-01  2.00723620e-01  7.22300731e-01  7.31999440e-01
   1.30283361e-01  3.42521416e-01  4.19636314e-01  5.55874162e-01
   1.10123974e-01  8.55829815e-02]
 [ 7.37159198e-01  2.60302460e-01  7.46294283e-01  8.01494523e-01
   2.46986503e-01  4.56455058e-01  6.18513825e-01  6.91513854e-01
   2.24364533e-01 -9.03289980e-02  9.53505869e-01  1.11763668e-01
   9.39306209e-01  1.00176056e+00  7.52826451e-02  2.85341536e-01
   2.71371654e-01  4.16461487e-01  1.34345087e-01  1.27294619e-01
   7.58706592e-01  1.96842594e-01  7.62552799e-01  8.12836496e-01
   1.25610187e-01  2.83755229e-01  3.85778129e-01  5.39113790e-01
   7.42567956e-02  1.75701742e-02]
 [-2.22992026e-01  6.62542133e-03 -2.03050882e-01 -1.67070287e-01
   3.32960611e-01  1.35537471e-01  9.87372735e-02  2.77019938e-02
   1.87650956e-01  4.02672109e-01  1.64803858e-01  3.97942224e-01
   1.51341309e-01  7.52826451e-02  1.00176056e+00  3.37288855e-01
   2.69157796e-01  3.29007720e-01  4.14234129e-01  4.28126626e-01
  -2.31096855e-01 -7.48745546e-02 -2.17686332e-01 -1.82516245e-01
   3.15011078e-01 -5.56559523e-02 -5.84010247e-02 -1.02186386e-01
  -1.07531080e-01  1.01658978e-01]
 [ 2.06362656e-01  1.92312595e-01  2.51185131e-01  2.12956816e-01
   3.19504817e-01  7.40022356e-01  6.71458893e-01  4.91287673e-01
   4.22401505e-01  5.60822319e-01  3.56691450e-01  2.32107621e-01
   4.17055330e-01  2.85341536e-01  3.37288855e-01  1.00176056e+00
   8.02679026e-01  7.45392672e-01  3.95407752e-01  8.04683023e-01
   2.04967390e-01  1.43254348e-01  2.60974494e-01  1.99722335e-01
   2.27794574e-01  6.79975390e-01  6.40271956e-01  4.84059046e-01
   2.78367653e-01  5.92013208e-01]
 [ 1.94545531e-01  1.43545353e-01  2.28483899e-01  2.08025659e-01
   2.48832996e-01  5.71521303e-01  6.92487233e-01  4.39940250e-01
   3.43230240e-01  4.47416643e-01  3.32942674e-01  1.95341772e-01
   3.63119754e-01  2.71371654e-01  2.69157796e-01  8.02679026e-01
   1.00176056e+00  7.73162805e-01  3.09973347e-01  7.28652769e-01
   1.87232571e-01  1.00417464e-01  2.27079511e-01  1.88684259e-01
   1.68777943e-01  4.85711424e-01  6.63730620e-01  4.41247742e-01
   1.98136040e-01  4.40102736e-01]
 [ 3.76831225e-01  1.64139495e-01  4.07933847e-01  3.72975776e-01
   3.81345895e-01  6.43392594e-01  6.84462839e-01  6.16717994e-01
   3.93990298e-01  3.41798745e-01  5.14250220e-01  2.30688828e-01
   5.57243422e-01  4.16461487e-01  3.29007720e-01  7.45392672e-01
   7.73162805e-01  1.00176056e+00  3.13330893e-01  6.12119921e-01
   3.58757174e-01  8.68939233e-02  3.95694673e-01  3.42873752e-01
   2.15729735e-01  4.53685716e-01  5.50559967e-01  6.03510257e-01
   1.43367633e-01  3.11201479e-01]
 [-1.04504545e-01  9.14323671e-03 -8.17730406e-02 -7.26242231e-02
   2.01127852e-01  2.30381479e-01  1.78322604e-01  9.55186580e-02
   4.49927276e-01  3.45614805e-01  2.40990897e-01  4.12345364e-01
   2.66956259e-01  1.34345087e-01  4.14234129e-01  3.95407752e-01
   3.09973347e-01  3.13330893e-01  1.00176056e+00  3.69727869e-01
  -1.28346334e-01 -7.76098171e-02 -1.03935708e-01 -1.10537008e-01
  -1.26840915e-02  6.03609620e-02  3.71843990e-02 -3.04669411e-02
   3.90088053e-01  7.82169401e-02]
 [-4.27163418e-02  5.45533955e-02 -5.53311534e-03 -1.99219755e-02
   2.84106006e-01  5.08211293e-01  4.50091771e-01  2.58037239e-01
   3.32370277e-01  6.89343077e-01  2.28154507e-01  2.80215217e-01
   2.44572602e-01  1.27294619e-01  4.28126626e-01  8.04683023e-01
   7.28652769e-01  6.12119921e-01  3.69727869e-01  1.00176056e+00
  -3.75536172e-02 -3.20065392e-03 -1.00215889e-03 -2.27761757e-02
   1.70868612e-01  3.90845741e-01  3.80643631e-01  2.15582894e-01
   1.11289544e-01  5.92369136e-01]
 [ 9.71245907e-01  3.53193674e-01  9.71183188e-01  9.64441062e-01
   2.13495353e-01  5.36257855e-01  6.89448091e-01  8.31779458e-01
   1.86054739e-01 -2.54138135e-01  7.16324113e-01 -1.11886951e-01
   6.98428059e-01  7.58706592e-01 -2.31096855e-01  2.04967390e-01
   1.87232571e-01  3.58757174e-01 -1.28346334e-01 -3.75536172e-02
   1.00176056e+00  3.60554418e-01  9.95457402e-01  9.85746984e-01
   2.16955724e-01  4.76657749e-01  5.74985227e-01  7.88810161e-01
   2.43957953e-01  9.36565772e-02]
 [ 2.97530545e-01  9.13650301e-01  3.03571890e-01  2.87994769e-01
   3.61353055e-02  2.48569687e-01  3.00406844e-01  2.93267121e-01
   9.08102844e-02 -5.13594647e-02  1.95141512e-01  4.09722842e-01
   2.00723620e-01  1.96842594e-01 -7.48745546e-02  1.43254348e-01
   1.00417464e-01  8.68939233e-02 -7.76098171e-02 -3.20065392e-03
   3.60554418e-01  1.00176056e+00  3.65741024e-01  3.46451160e-01
   2.25826298e-01  3.61467607e-01  3.69014138e-01  3.60387980e-01
   2.33437721e-01  2.19508204e-01]
 [ 9.66835698e-01  3.58669926e-01  9.72095315e-01  9.60808165e-01
   2.39273141e-01  5.91249531e-01  7.30849362e-01  8.57430035e-01
   2.19554419e-01 -2.05512393e-01  7.20950853e-01 -1.02421925e-01
   7.22300731e-01  7.62552799e-01 -2.17686332e-01  2.60974494e-01
   2.27079511e-01  3.95694673e-01 -1.03935708e-01 -1.00215889e-03
   9.95457402e-01  3.65741024e-01  1.00176056e+00  9.79299180e-01
   2.37191461e-01  5.30339746e-01  6.19432713e-01  8.17759288e-01
   2.69967228e-01  1.39201504e-01]
 [ 9.42739295e-01  3.44150782e-01  9.43207466e-01  9.60902082e-01
   2.07082304e-01  5.10500995e-01  6.77177350e-01  8.11055024e-01
   1.77505338e-01 -2.32262646e-01  7.52871625e-01 -8.33414586e-02
   7.31999440e-01  8.12836496e-01 -1.82516245e-01  1.99722335e-01
   1.88684259e-01  3.42873752e-01 -1.10537008e-01 -2.27761757e-02
   9.85746984e-01  3.46451160e-01  9.79299180e-01  1.00176056e+00
   2.09513547e-01  4.39067932e-01  5.44287093e-01  7.48734680e-01
   2.09513722e-01  7.97872577e-02]
 [ 1.19826732e-01  7.76398084e-02  1.50814456e-01  1.23740409e-01
   8.06742020e-01  5.66536837e-01  4.49612218e-01  4.53550155e-01
   4.27426215e-01  5.05831058e-01  1.42168410e-01 -7.37873381e-02
   1.30283361e-01  1.25610187e-01  3.15011078e-01  2.27794574e-01
   1.68777943e-01  2.15729735e-01 -1.26840915e-02  1.70868612e-01
   2.16955724e-01  2.25826298e-01  2.37191461e-01  2.09513547e-01
   1.00176056e+00  5.69186845e-01  5.19436186e-01  5.48655147e-01
   4.94707764e-01  6.18711558e-01]
 [ 4.14190751e-01  2.78318729e-01  4.56576647e-01  3.91097651e-01
   4.73300254e-01  8.67333351e-01  7.56297185e-01  6.68628771e-01
   4.74033112e-01  4.59605900e-01  2.87608629e-01 -9.26020990e-02
   3.42521416e-01  2.83755229e-01 -5.56559523e-02  6.79975390e-01
   4.85711424e-01  4.53685716e-01  6.03609620e-02  3.90845741e-01
   4.76657749e-01  3.61467607e-01  5.30339746e-01  4.39067932e-01
   5.69186845e-01  1.00176056e+00  8.93831781e-01  8.02490717e-01
   6.15522263e-01  8.11881713e-01]
 [ 5.27839123e-01  3.01555198e-01  5.64872009e-01  5.13508396e-01
   4.35691429e-01  8.17712354e-01  8.85659158e-01  7.53724145e-01
   4.34484601e-01  3.46843443e-01  3.81254678e-01 -6.90776223e-02
   4.19636314e-01  3.85778129e-01 -5.84010247e-02  6.40271956e-01
   6.63730620e-01  5.50559967e-01  3.71843990e-02  3.80643631e-01
   5.74985227e-01  3.69014138e-01  6.19432713e-01  5.44287093e-01
   5.19436186e-01  8.93831781e-01  1.00176056e+00  8.56939906e-01
   5.33457264e-01  6.87719567e-01]
 [ 7.45524434e-01  2.95835766e-01  7.72598608e-01  7.23287782e-01
   5.03939011e-01  8.17009092e-01  8.62839447e-01  9.11757700e-01
   4.31054176e-01  1.75634121e-01  5.31997297e-01 -1.19848153e-01
   5.55874162e-01  5.39113790e-01 -1.02186386e-01  4.84059046e-01
   4.41247742e-01  6.03510257e-01 -3.04669411e-02  2.15582894e-01
   7.88810161e-01  3.60387980e-01  8.17759288e-01  7.48734680e-01
   5.48655147e-01  8.02490717e-01  8.56939906e-01  1.00176056e+00
   5.03413227e-01  5.12013995e-01]
 [ 1.64241985e-01  1.05192783e-01  1.89447989e-01  1.43822678e-01
   3.95003689e-01  5.11121711e-01  4.10185014e-01  3.76405667e-01
   7.01057885e-01  3.34606745e-01  9.47092790e-02 -1.28440488e-01
   1.10123974e-01  7.42567956e-02 -1.07531080e-01  2.78367653e-01
   1.98136040e-01  1.43367633e-01  3.90088053e-01  1.11289544e-01
   2.43957953e-01  2.33437721e-01  2.69967228e-01  2.09513722e-01
   4.94707764e-01  6.15522263e-01  5.33457264e-01  5.03413227e-01
   1.00176056e+00  5.38795122e-01]
 [ 7.07832563e-03  1.19415220e-01  5.11083511e-02  3.74417763e-03
   5.00195447e-01  6.88592503e-01  5.15836457e-01  3.69310185e-01
   4.39185353e-01  7.68647654e-01  4.96466850e-02 -4.57349464e-02
   8.55829815e-02  1.75701742e-02  1.01658978e-01  5.92013208e-01
   4.40102736e-01  3.11201479e-01  7.82169401e-02  5.92369136e-01
   9.36565772e-02  2.19508204e-01  1.39201504e-01  7.97872577e-02
   6.18711558e-01  8.11881713e-01  6.87719567e-01  5.12013995e-01
   5.38795122e-01  1.00176056e+00]]

Eigenvalues and Eigenvectors

eig_vals, eig_vecs = np.linalg.eig(cov_mat)

print('\nEigenvalues \n%s' %eig_vals)

Eigenvalues 
[1.33049908e+01 5.70137460e+00 2.82291016e+00 1.98412752e+00
 1.65163324e+00 1.20948224e+00 6.76408882e-01 4.77456255e-01
 4.17628782e-01 3.51310875e-01 2.94433153e-01 2.61621161e-01
 2.41782421e-01 1.57286149e-01 9.43006956e-02 8.00034045e-02
 5.95036135e-02 5.27114222e-02 4.95647002e-02 1.33279057e-04
 7.50121413e-04 1.59213600e-03 6.91261258e-03 8.19203712e-03
 1.55085271e-02 1.80867940e-02 2.43836914e-02 2.74877113e-02
 3.12142606e-02 3.00256631e-02]

eig_val_df = pd.DataFrame(eig_vals, columns=['Eigenvalue'])
eig_val_df

	Eigenvalue
0	13.304991
1	5.701375
2	2.822910
3	1.984128
4	1.651633
5	1.209482
6	0.676409
7	0.477456
8	0.417629
9	0.351311
10	0.294433
11	0.261621
12	0.241782
13	0.157286
14	0.094301
15	0.080003
16	0.059504
17	0.052711
18	0.049565
19	0.000133
20	0.000750
21	0.001592
22	0.006913
23	0.008192
24	0.015509
25	0.018087
26	0.024384
27	0.027488
28	0.031214
29	0.030026

print('Eigenvectors \n%s' %eig_vecs)

Eigenvectors 
[[ 2.18902444e-01 -2.33857132e-01 -8.53124284e-03  4.14089623e-02
  -3.77863538e-02  1.87407904e-02  1.24088340e-01  7.45229622e-03
  -2.23109764e-01  9.54864432e-02  4.14714866e-02  5.10674568e-02
   1.19672116e-02 -5.95061348e-02  5.11187749e-02 -1.50583883e-01
   2.02924255e-01  1.46712338e-01 -2.25384659e-01 -7.02414091e-01
   2.11460455e-01 -2.11194013e-01 -1.31526670e-01  1.29476396e-01
   1.92264989e-02 -1.82579441e-01  9.85526942e-02 -7.29289034e-02
  -4.96986642e-02  6.85700057e-02]
 [ 1.03724578e-01 -5.97060883e-02  6.45499033e-02 -6.03050001e-01
   4.94688505e-02 -3.21788366e-02 -1.13995382e-02 -1.30674825e-01
   1.12699390e-01  2.40934066e-01 -3.02243402e-01  2.54896423e-01
   2.03461333e-01  2.15600995e-02  1.07922421e-01 -1.57841960e-01
  -3.87061187e-02 -4.11029851e-02 -2.97886446e-02 -2.73661018e-04
  -1.05339342e-02  6.58114593e-05 -1.73573093e-02  2.45566636e-02
  -8.47459309e-02  9.87867898e-02  5.54997454e-04 -9.48006326e-02
  -2.44134993e-01 -4.48369467e-01]
 [ 2.27537293e-01 -2.15181361e-01 -9.31421972e-03  4.19830991e-02
  -3.73746632e-02  1.73084449e-02  1.14477057e-01  1.86872582e-02
  -2.23739213e-01  8.63856150e-02  1.67826374e-02  3.89261058e-02
   4.41095034e-02 -4.85138123e-02  3.99029358e-02 -1.14453955e-01
   1.94821310e-01  1.58317455e-01 -2.39595276e-01  6.89896968e-01
   3.83826098e-01 -8.43382663e-02 -1.15415423e-01  1.25255946e-01
  -2.70154137e-02 -1.16648876e-01  4.02447050e-02 -7.51604777e-02
  -1.76650122e-02  6.97690429e-02]
 [ 2.20994985e-01 -2.31076711e-01  2.86995259e-02  5.34337955e-02
  -1.03312514e-02 -1.88774796e-03  5.16534275e-02 -3.46736038e-02
  -1.95586014e-01  7.49564886e-02  1.10169643e-01  6.54375082e-02
   6.73757374e-02 -1.08308292e-02 -1.39669069e-02 -1.32448032e-01
   2.55705763e-01  2.66168105e-01  2.73221894e-02  3.29473482e-02
  -4.22794920e-01  2.72508323e-01  4.66612477e-01 -3.62727403e-01
   2.10040780e-01  6.98483369e-02 -7.77727342e-03 -9.75657781e-02
  -9.01437617e-02  1.84432785e-02]
 [ 1.42589694e-01  1.86113023e-01 -1.04291904e-01  1.59382765e-01
   3.65088528e-01 -2.86374497e-01  1.40668993e-01  2.88974575e-01
   6.42472194e-03 -6.92926813e-02 -1.37021842e-01  3.16727211e-01
   4.55736020e-02 -4.45064860e-01  1.18143364e-01 -2.04613247e-01
   1.67929914e-01 -3.52226802e-01  1.64565843e-01  4.84745766e-03
  -3.43466700e-03 -1.47926883e-03  6.96899233e-02  3.70036864e-02
  -2.89548850e-02  6.86974224e-02  2.06657211e-02 -6.38229479e-02
   1.71009601e-02  1.19491747e-01]
 [ 2.39285354e-01  1.51891610e-01 -7.40915709e-02  3.17945811e-02
  -1.17039713e-02 -1.41309489e-02 -3.09184960e-02  1.51396350e-01
  -1.67841425e-01  1.29362000e-02 -3.08009633e-01 -1.04017044e-01
   2.29281304e-01 -8.10105720e-03 -2.30899962e-01  1.70178367e-01
  -2.03077075e-02  7.79413843e-03 -2.84222358e-01 -4.46741863e-02
  -4.10167739e-02  5.46276696e-03  9.77487054e-02 -2.62808474e-01
  -3.96623231e-01 -1.04135518e-01 -5.23603957e-02  9.80775567e-02
   4.88686329e-01 -1.92621396e-01]
 [ 2.58400481e-01  6.01653628e-02  2.73383798e-03  1.91227535e-02
  -8.63754118e-02 -9.34418089e-03  1.07520443e-01  7.28272853e-02
   4.05910064e-02 -1.35602298e-01  1.24190245e-01  6.56534798e-02
   3.87090806e-01  1.89358699e-01  1.28283732e-01  2.69470206e-01
  -1.59835337e-03 -2.69681105e-02 -2.26636013e-03 -2.51386661e-02
  -1.00147876e-02 -4.55386379e-02  3.64808397e-01  5.48876170e-01
   9.69773167e-02  4.47410568e-02 -3.24870378e-01  1.85212003e-01
  -3.33870858e-02 -5.57175335e-03]
 [ 2.60853758e-01 -3.47675005e-02 -2.55635406e-02  6.53359443e-02
   4.38610252e-02 -5.20499505e-02  1.50482214e-01  1.52322414e-01
  -1.11971106e-01  8.05452775e-03 -7.24460264e-02  4.25892667e-02
   1.32138097e-01  2.44794768e-01  2.17099194e-01  3.80464095e-01
   3.45095087e-02 -8.28277367e-02  1.54972363e-01  1.07726530e-03
  -4.20694931e-03  8.88309714e-03 -4.54699351e-01 -3.87643377e-01
   1.86451602e-01  8.40276972e-02  5.14087968e-02  3.11852431e-01
  -2.35407606e-01  9.42381870e-03]
 [ 1.38166959e-01  1.90348770e-01 -4.02399363e-02  6.71249840e-02
   3.05941428e-01  3.56458461e-01  9.38911345e-02  2.31530989e-01
   2.56040084e-01  5.72069479e-01  1.63054081e-01 -2.88865504e-01
   1.89933673e-01 -3.07388563e-02  7.39617071e-02 -1.64661588e-01
  -1.91737848e-01  1.73397790e-01  5.88111647e-02  1.28037941e-03
  -7.56986244e-03 -1.43302642e-03 -1.51648349e-02  1.60440385e-02
   2.45836949e-02  1.93394733e-02  5.12005770e-02  1.84067326e-02
   2.60691555e-02  8.69384844e-02]
 [ 6.43633464e-02  3.66575471e-01 -2.25740897e-02  4.85867649e-02
   4.44243602e-02 -1.19430668e-01 -2.95760024e-01  1.77121441e-01
  -1.23740789e-01  8.11032072e-02 -3.80482687e-02  2.36358988e-01
   1.06239082e-01  3.77078865e-01 -5.17975705e-01 -4.07927860e-02
   5.02252456e-02  8.78673570e-02  5.81570509e-02  4.75568480e-03
   7.30143287e-03  6.31168651e-03 -1.01244946e-01  9.74048386e-02
   2.07221864e-01 -1.33260547e-01  8.46898562e-02 -2.87868885e-01
  -1.75637222e-01  7.62718362e-02]
 [ 2.05978776e-01 -1.05552152e-01  2.68481387e-01  9.79412418e-02
   1.54456496e-01 -2.56032561e-02 -3.12490037e-01 -2.25399674e-02
   2.49985002e-01 -4.95475941e-02 -2.53570194e-02 -1.66879153e-02
  -6.81952298e-02 -1.03474126e-02  1.10050711e-01  5.89057190e-02
  -1.39396866e-01 -2.36216532e-01 -1.75883308e-01  8.71109373e-03
   1.18442112e-01  1.92223890e-01  2.12982901e-01 -4.99770798e-02
   1.74930429e-01 -5.58701567e-01  2.64125317e-01  1.50274681e-01
  -9.08005031e-02 -8.63867747e-02]
 [ 1.74280281e-02  8.99796818e-02  3.74633665e-01 -3.59855528e-01
   1.91650506e-01 -2.87473145e-02  9.07553556e-02  4.75413139e-01
  -2.46645397e-01 -2.89142742e-01  3.44944458e-01 -3.06160423e-01
  -1.68222383e-01  1.08493473e-02 -3.27527212e-02 -3.45004006e-02
   4.39630156e-02 -9.85866201e-03 -3.60098518e-02  1.07103919e-03
  -8.77627920e-03  5.62261069e-03 -1.00928890e-02  1.12372419e-02
  -5.69864778e-02  2.42672970e-02  8.73880467e-04 -4.84569345e-02
  -7.16599878e-02 -2.17071967e-01]
 [ 2.11325916e-01 -8.94572342e-02  2.66645367e-01  8.89924146e-02
   1.20990220e-01  1.81071500e-03 -3.14640390e-01  1.18966905e-02
   2.27154024e-01 -1.14508236e-01 -1.67318771e-01 -1.01446828e-01
  -3.78439858e-02  4.55237175e-02  8.26808881e-03  2.65166513e-02
  -2.46356391e-02 -2.59288003e-02 -3.65701538e-01 -1.37293906e-02
  -6.10021933e-03 -2.63191868e-01  4.16915529e-02 -1.03653282e-01
  -7.29276412e-02  5.16750385e-01 -9.00742110e-02 -1.59352804e-01
  -1.77250625e-01  3.04950158e-01]
 [ 2.02869635e-01 -1.52292628e-01  2.16006528e-01  1.08205039e-01
   1.27574432e-01 -4.28639079e-02 -3.46679003e-01 -8.58051345e-02
   2.29160015e-01 -9.19278886e-02  5.16194632e-02 -1.76792177e-02
   5.60649318e-02 -8.35707181e-02  4.60243656e-02  4.11532265e-02
   3.34418173e-01  3.04906903e-01  4.16572314e-01 -1.10532603e-03
  -8.59259138e-02  4.20681051e-02 -3.13358657e-01  1.55304589e-01
  -1.31850405e-01 -2.24607172e-02 -9.82150746e-02 -6.42326151e-02
   2.74201148e-01 -1.92587786e-01]
 [ 1.45314521e-02  2.04430453e-01  3.08838979e-01  4.46641797e-02
   2.32065676e-01 -3.42917393e-01  2.44024056e-01 -5.73410232e-01
  -1.41924890e-01  1.60884609e-01  8.42062106e-02 -2.94710053e-01
   1.50441434e-01  2.01152530e-01 -1.85594647e-02 -5.80390613e-02
   1.39595006e-01 -2.31259943e-01  1.32600886e-02  1.60821086e-03
   1.77638619e-03 -9.79296328e-03 -9.05215355e-03  7.71755717e-03
  -3.12107028e-02  1.56311888e-02  5.98177179e-02 -5.05449015e-02
   9.00614773e-02  7.20987261e-02]
 [ 1.70393451e-01  2.32715896e-01  1.54779718e-01 -2.74693632e-02
  -2.79968156e-01  6.91975186e-02 -2.34635340e-02 -1.17460157e-01
  -1.45322810e-01  4.35048658e-02 -2.06885680e-01 -2.63456509e-01
   1.00401699e-02 -4.91755932e-01 -1.68209315e-01  1.89830896e-01
  -8.24647717e-03  1.00474235e-01  2.42448176e-01 -1.91562235e-03
   3.15813441e-03  1.53955481e-02  4.65360884e-02  4.97276317e-02
  -1.73164553e-01 -1.21777792e-01 -9.10387102e-03  4.52876920e-02
  -4.61098220e-01  1.40386572e-01]
 [ 1.53589790e-01  1.97207283e-01  1.76463743e-01  1.31687997e-03
  -3.53982091e-01  5.63432386e-02  2.08823790e-01 -6.05665008e-02
   3.58107079e-01 -1.41276243e-01  3.49517943e-01  2.51146975e-01
   1.58783192e-01 -1.34586924e-01 -2.50471408e-01 -1.25420649e-01
   8.46167156e-02 -1.95485228e-04 -1.26381025e-01  8.92652653e-03
   1.60785207e-02 -5.82097800e-03 -8.42247975e-02 -9.14549680e-02
  -1.59399802e-02  1.88205036e-01  3.87542329e-01  2.05212693e-01
   6.69461742e-02 -6.30479298e-02]
 [ 1.83417397e-01  1.30321560e-01  2.24657567e-01  7.40673350e-02
  -1.95548089e-01 -3.12244482e-02  3.69645937e-01  1.08319309e-01
   2.72519886e-01  8.62408470e-02 -3.42375908e-01 -6.45875122e-03
  -4.94026741e-01  1.99666719e-01 -6.20793442e-02 -1.98810346e-01
   1.08132263e-01  4.60549116e-02  1.21642969e-02  2.16019727e-03
  -2.39377870e-02  2.90093001e-02 -1.11655093e-02  1.79419192e-02
   1.29546547e-01 -1.09668978e-01 -3.51755074e-01  7.25453753e-02
   6.88682942e-02 -3.43753236e-02]
 [ 4.24984216e-02  1.83848000e-01  2.88584292e-01  4.40733510e-02
   2.52868765e-01  4.90245643e-01  8.03822539e-02 -2.20149279e-01
  -3.04077200e-01 -3.16529830e-01 -1.87844043e-01  3.20571348e-01
   1.03327412e-02  4.68643826e-02  1.13383199e-01 -1.57711497e-01
  -2.74059129e-01  1.87014764e-01  8.90392949e-02 -3.29389752e-04
  -5.22329189e-03  7.63652550e-03 -1.99759830e-02  1.72678486e-02
   1.95149333e-02  3.22620011e-03  4.23628949e-02  8.46544307e-02
   1.07385289e-01  9.76995265e-02]
 [ 1.02568322e-01  2.80092027e-01  2.11503764e-01  1.53047496e-02
  -2.63297438e-01 -5.31952674e-02 -1.91394973e-01 -1.11681884e-02
  -2.13722716e-01  3.67541918e-01  2.50624789e-01  2.76165974e-01
  -2.40458323e-01 -1.45652466e-01  3.53232211e-01  2.68553878e-01
  -1.22733398e-01 -5.98230982e-02 -8.66008430e-02 -1.79895682e-03
  -8.34191154e-03 -1.97564555e-02 -1.20365640e-02 -3.54889745e-02
   8.41712034e-02  7.51944193e-02 -8.57810992e-02 -2.44705083e-01
   2.22345297e-01 -6.28432814e-02]
 [ 2.27996634e-01 -2.19866379e-01 -4.75069900e-02  1.54172396e-02
   4.40659209e-03 -2.90684919e-04  9.70993602e-03 -4.26194163e-02
  -1.12141463e-01  7.73616428e-02  1.05067333e-01  3.96796652e-02
  -1.37890527e-01 -2.31012813e-02 -1.66567074e-01 -8.15605686e-02
  -2.40049982e-01 -2.16101353e-01 -1.36613039e-02  1.35643056e-01
  -6.35724917e-01 -4.12639581e-01 -1.78666740e-01  1.97054744e-01
  -7.07097238e-02 -1.56830365e-01  5.56767923e-02  9.62982088e-02
  -5.62690874e-03 -7.29389953e-03]
 [ 1.04469325e-01 -4.54672983e-02 -4.22978228e-02 -6.32807885e-01
   9.28834001e-02 -5.00080613e-02 -9.87074388e-03 -3.62516360e-02
   1.03341204e-01  2.95509413e-02  1.31572736e-02  7.97974499e-02
  -8.01454315e-02 -5.34307917e-02 -1.01115399e-01  1.85557852e-01
   6.93651855e-02  5.83984505e-02  7.58669276e-02 -1.02053601e-03
   1.72354925e-02  3.90250926e-04  2.14106944e-02 -3.64694332e-02
   1.18189721e-01 -1.18484602e-01  8.92289971e-03  1.11112024e-01
   3.00599798e-01  5.94440143e-01]
 [ 2.36639681e-01 -1.99878428e-01 -4.85465083e-02  1.38027944e-02
  -7.45415100e-03  8.50098715e-03  4.45726717e-04 -3.05585340e-02
  -1.09614364e-01  5.05083335e-02  5.10762807e-02 -8.98773800e-03
  -9.69657077e-02 -1.22193824e-02 -1.82755198e-01 -5.48570473e-02
  -2.34164147e-01 -1.88543592e-01 -9.08132490e-02 -7.97438536e-02
   2.29218029e-02  7.28680898e-01 -2.41031046e-01  2.44103670e-01
  -1.18034029e-01  2.37113167e-01 -6.33448296e-02 -1.72216251e-02
   1.10038577e-02  9.20235990e-02]
 [ 2.24870533e-01 -2.19351858e-01 -1.19023182e-02  2.58947492e-02
   2.73909030e-02 -2.51643821e-02 -6.78316595e-02 -7.93942456e-02
  -8.07324609e-02  6.99211523e-02  1.84598937e-01  4.80886567e-02
  -1.01160611e-01  6.68546458e-03 -3.14993600e-01 -9.06533944e-02
  -2.73399584e-01 -1.42064856e-01  4.10047202e-01 -3.97422838e-02
   4.44935933e-01 -2.38960316e-01  2.37162466e-01 -2.31359525e-01
   3.82899511e-02  1.44063033e-01 -1.90889625e-01  9.69598236e-02
   6.00473870e-02 -1.46790132e-01]
 [ 1.27952561e-01  1.72304352e-01 -2.59797613e-01  1.76522161e-02
   3.24435445e-01 -3.69255370e-01  1.08830886e-01 -2.05852191e-01
   1.12315904e-01 -1.28304659e-01  1.43890349e-01  5.65148662e-02
  -2.05130344e-01 -1.62235443e-01 -4.61258656e-02  1.45551659e-01
  -2.78030197e-01  5.01551675e-01 -2.34513845e-01 -4.58327731e-03
   7.38549171e-03  1.53524821e-03 -4.08535683e-02 -1.26024637e-02
   4.79647647e-02 -1.09901386e-02 -9.36901494e-02  6.82540931e-02
  -1.29723903e-01 -1.64849237e-01]
 [ 2.10095880e-01  1.43593173e-01 -2.36075625e-01 -9.13284153e-02
  -1.21804107e-01  4.77057929e-02 -1.40472938e-01 -8.40196588e-02
  -1.00677822e-01 -1.72133632e-01 -1.97420469e-01 -3.71662503e-01
   1.22793095e-02 -1.66470250e-01  4.99560142e-02 -1.53734861e-01
  -4.03712272e-03 -7.35745143e-02 -2.02007041e-02  1.28415624e-02
   3.56690391e-06 -4.86918180e-02 -7.05054136e-02  1.00463424e-01
   6.24384938e-01  1.86749953e-01  1.47920925e-01 -2.96764124e-02
   2.29280589e-01 -1.81374867e-01]
 [ 2.28767533e-01  9.79641143e-02 -1.73057335e-01 -7.39511797e-02
  -1.88518727e-01  2.83792555e-02  6.04880561e-02 -7.24678714e-02
   1.61908621e-01 -3.11638520e-01  1.85016760e-01 -8.70345324e-02
   2.17984329e-01  6.67989309e-02  2.04835886e-01 -2.15021948e-01
  -1.91313419e-01 -1.03907980e-01  4.57861197e-02 -4.02139168e-04
  -1.26757226e-02  1.76408967e-02 -1.42905801e-01 -2.66853781e-01
  -1.15770341e-01 -2.88852570e-01 -2.86433135e-01 -4.60426186e-01
  -4.64827918e-02  1.32100595e-01]
 [ 2.50885971e-01 -8.25723507e-03 -1.70344076e-01  6.00699571e-03
  -4.33320687e-02 -3.08734498e-02  1.67966619e-01  3.61707954e-02
   6.04884615e-02 -7.66482910e-02 -1.17772055e-01 -6.81253543e-02
  -2.54387490e-01  2.76418891e-01  1.69499607e-01  1.78141741e-01
  -7.54853164e-02  7.58138963e-02  2.60229625e-01  2.28844179e-03
   3.52404543e-02 -2.24756680e-02  2.30901389e-01  1.33574507e-01
  -2.63196337e-01  1.07340243e-01  5.67527797e-01 -2.99840557e-01
   3.30223397e-02 -8.86081478e-04]
 [ 1.22904556e-01  1.41883349e-01 -2.71312642e-01 -3.62506947e-02
   2.44558663e-01  4.98926784e-01  1.84906298e-02 -2.28225053e-01
   6.46378061e-02 -2.95630751e-02  1.57560248e-01  4.40335026e-02
  -2.56534905e-01 -5.35557351e-03 -1.39888394e-01  2.57894009e-01
   4.30658116e-01 -2.78713843e-01 -1.17250532e-01 -3.95443454e-04
   1.34042283e-02 -4.92048082e-03  2.27904438e-02 -2.81842956e-02
  -4.52996243e-02 -1.43818093e-02 -1.21343451e-01 -9.71448437e-02
  -1.16759236e-01 -1.62708549e-01]
 [ 1.31783943e-01  2.75339469e-01 -2.32791313e-01 -7.70534703e-02
  -9.44233510e-02 -8.02235245e-02 -3.74657626e-01 -4.83606666e-02
  -1.34174175e-01  1.26095791e-02  1.18283551e-01 -3.47316933e-02
  -1.72814238e-01  2.12104110e-01  2.56173195e-01 -4.05556492e-01
   1.59394300e-01  2.35647497e-02  1.14944811e-02 -1.89429245e-03
   1.14776603e-02  2.35621424e-02  5.99859979e-02 -4.52048188e-03
  -2.80133485e-01  3.78254532e-02 -7.62533821e-03  4.69471147e-01
  -1.04991974e-01  9.23439434e-02]]

PCA Visualization

# importing PCA module 
from sklearn.decomposition import PCA
pca = PCA (n_components=6)
pca.fit(scaled_data)
x_pca = pca.transform(scaled_data)

Scree Plot

sns.set_style ('whitegrid')
PC_values = np.arange(pca.n_components_) + 1
plt.plot(PC_values, pca.explained_variance_ratio_, 'o-', linewidth=2, color='green')
plt.title('Scree Plot')
plt.xlim(0,7)
plt.xlabel('Principal Component')
plt.ylabel('Variance Explained')
plt.xticks(ticks=range(1, 8, 1))  # Set the x-ticks with an interval of 1
plt.show()

Cumulative Variance Plot

sns.set_style('whitegrid')
# Plot the cumulative variance for each component
plt.figure(figsize = (8, 4))
components = np.arange(1, 7, step=1)
variance = np.cumsum(pca.explained_variance_ratio_)
plt.ylim(0.0,1.1)
plt.plot(components, variance, marker='o', color='green')
# plt.plot(components, variance, marker='o', linestyle='--', color='green')

plt.xlabel('Number of Components')
plt.ylabel('Cumulative variance (%)')
plt.title('The number of components needed to explain variance')
plt.show()

pca.explained_variance_ratio_.sum()

np.float64(0.887587963566906)

pca.get_covariance()

array([[ 1.08156343e+00,  3.25564716e-01,  9.41407673e-01,
         9.41300789e-01,  1.56852511e-01,  4.96533554e-01,
         6.72530754e-01,  7.98934415e-01,  1.46343436e-01,
        -2.91918938e-01,  7.22825981e-01, -1.14344516e-01,
         7.19123684e-01,  7.77820405e-01, -2.47738498e-01,
         2.00129486e-01,  2.03550841e-01,  3.70133453e-01,
        -1.24463551e-01, -5.83643764e-02,  9.44932010e-01,
         3.06532654e-01,  9.44600182e-01,  9.33429194e-01,
         1.26030234e-01,  4.25042454e-01,  5.41489854e-01,
         7.39922977e-01,  1.69140312e-01,  2.49400180e-02],
       [ 3.25564716e-01,  9.88571103e-01,  3.30461041e-01,
         3.23338871e-01, -2.51875070e-02,  2.27744152e-01,
         3.05940167e-01,  2.95738924e-01,  5.44863595e-02,
        -8.43105441e-02,  2.66330148e-01,  4.74127974e-01,
         2.74473807e-01,  2.55692355e-01, -1.50720779e-02,
         1.89430823e-01,  1.44939552e-01,  1.50198488e-01,
         1.23914887e-06,  4.88161300e-02,  3.59291001e-01,
         8.62537569e-01,  3.64882515e-01,  3.51942002e-01,
         8.98529661e-02,  2.89110719e-01,  3.17026535e-01,
         3.06968043e-01,  1.15160075e-01,  1.29585260e-01],
       [ 9.41407673e-01,  3.30461041e-01,  1.08570247e+00,
         9.42417263e-01,  1.93441607e-01,  5.39710737e-01,
         7.08126599e-01,  8.25205188e-01,  1.81616694e-01,
        -2.46225009e-01,  7.34953461e-01, -1.04023567e-01,
         7.33462019e-01,  7.84871291e-01, -2.24788884e-01,
         2.43031361e-01,  2.40813668e-01,  4.04049201e-01,
        -1.01692897e-01, -1.81284390e-02,  9.48135074e-01,
         3.13239505e-01,  9.50841073e-01,  9.36319032e-01,
         1.59799654e-01,  4.64086582e-01,  5.77792027e-01,
         7.67968363e-01,  1.97765195e-01,  6.89850961e-02],
       [ 9.41300789e-01,  3.23338871e-01,  9.42417263e-01,
         1.08824164e+00,  1.78234056e-01,  4.98605582e-01,
         6.77898970e-01,  8.07444956e-01,  1.55394300e-01,
        -2.81179851e-01,  7.62820680e-01, -7.44559147e-02,
         7.57140226e-01,  8.11259695e-01, -1.95160988e-01,
         2.10129486e-01,  2.12555582e-01,  3.93853441e-01,
        -9.09746465e-02, -3.94979415e-02,  9.43600212e-01,
         2.95412154e-01,  9.42990092e-01,  9.37308693e-01,
         1.28265771e-01,  4.01347136e-01,  5.21940187e-01,
         7.28712432e-01,  1.45968333e-01,  5.72488968e-03],
       [ 1.56852511e-01, -2.51875070e-02,  1.93441607e-01,
         1.78234056e-01,  9.66053415e-01,  6.34287058e-01,
         5.07373062e-01,  5.20133714e-01,  5.46983019e-01,
         5.81831357e-01,  3.24111478e-01,  2.98263375e-02,
         3.21839377e-01,  2.78052145e-01,  3.98533353e-01,
         3.33705387e-01,  2.30915292e-01,  3.39769952e-01,
         1.91703094e-01,  2.98775901e-01,  2.20766641e-01,
         4.14956784e-02,  2.48259261e-01,  2.28840783e-01,
         7.88325477e-01,  5.00388699e-01,  4.44807972e-01,
         4.97349918e-01,  4.24980666e-01,  5.47267058e-01],
       [ 4.96533554e-01,  2.27744152e-01,  5.39710737e-01,
         4.98605582e-01,  6.34287058e-01,  1.03980984e+00,
         8.67025617e-01,  8.01246021e-01,  5.97134962e-01,
         5.20710155e-01,  5.09725935e-01,  3.24016491e-02,
         5.40181563e-01,  4.72220256e-01,  1.60757176e-01,
         7.04832049e-01,  6.20790369e-01,  6.51462538e-01,
         2.22509336e-01,  5.23984861e-01,  5.42760449e-01,
         2.61107074e-01,  5.87057057e-01,  5.26855539e-01,
         6.01083696e-01,  8.26078562e-01,  8.36322404e-01,
         8.18754543e-01,  5.46918841e-01,  6.92301471e-01],
       [ 6.72530754e-01,  3.05940167e-01,  7.08126599e-01,
         6.77898970e-01,  5.07373062e-01,  8.67025617e-01,
         1.05193692e+00,  8.72611154e-01,  4.92262441e-01,
         3.38523606e-01,  6.50873010e-01,  5.47255199e-02,
         6.78211471e-01,  6.28317067e-01,  9.48027231e-02,
         6.93486270e-01,  6.35407645e-01,  6.97609842e-01,
         1.71845962e-01,  4.79589796e-01,  7.01628418e-01,
         3.05917128e-01,  7.39111593e-01,  6.89045977e-01,
         4.52959684e-01,  7.73180063e-01,  8.31402906e-01,
         8.55587912e-01,  4.25380263e-01,  5.49098934e-01],
       [ 7.98934415e-01,  2.95738924e-01,  8.25205188e-01,
         8.07444956e-01,  5.20133714e-01,  8.01246021e-01,
         8.72611154e-01,  1.05866340e+00,  4.48946016e-01,
         1.67137218e-01,  7.32771858e-01, -1.22764949e-02,
         7.43327404e-01,  7.35144357e-01,  2.90355316e-02,
         5.03806868e-01,  4.50759409e-01,  5.86944269e-01,
         8.54015499e-02,  2.70915784e-01,  8.30855199e-01,
         3.03154423e-01,  8.55272436e-01,  8.21748313e-01,
         4.68049232e-01,  6.88157336e-01,  7.55522068e-01,
         8.74372333e-01,  3.97306007e-01,  4.04193473e-01],
       [ 1.46343436e-01,  5.44863595e-02,  1.81616694e-01,
         1.55394300e-01,  5.46983019e-01,  5.97134962e-01,
         4.92262441e-01,  4.48946016e-01,  8.83405661e-01,
         4.88557243e-01,  3.07708353e-01,  1.19614367e-01,
         3.28534772e-01,  2.40512688e-01,  1.91638752e-01,
         4.33086475e-01,  3.27051251e-01,  3.54183200e-01,
         5.49847100e-01,  3.20117786e-01,  1.90936684e-01,
         9.20377145e-02,  2.25592717e-01,  1.84397469e-01,
         4.54625515e-01,  5.10172452e-01,  4.52985364e-01,
         4.34924218e-01,  7.01633270e-01,  4.72514568e-01],
       [-2.91918938e-01, -8.43105441e-02, -2.46225009e-01,
        -2.81179851e-01,  5.81831357e-01,  5.20710155e-01,
         3.38523606e-01,  1.67137218e-01,  4.88557243e-01,
         9.66462680e-01, -3.44801269e-02,  1.59799780e-01,
        -3.57752538e-03, -1.27894263e-01,  4.73666545e-01,
         5.79282441e-01,  4.90602813e-01,  4.04945499e-01,
         3.51634381e-01,  6.35522779e-01, -2.50401889e-01,
        -4.56616514e-02, -2.04335461e-01, -2.48501660e-01,
         5.45853737e-01,  4.62560491e-01,  3.81101060e-01,
         2.07626742e-01,  3.59262141e-01,  6.84002885e-01],
       [ 7.22825981e-01,  2.66330148e-01,  7.34953461e-01,
         7.62820680e-01,  3.24111478e-01,  5.09725935e-01,
         6.50873010e-01,  7.32771858e-01,  3.07708353e-01,
        -3.44801269e-02,  1.00899957e+00,  2.44763092e-01,
         8.61791821e-01,  8.45504685e-01,  2.13415127e-01,
         3.64712328e-01,  3.43881042e-01,  5.51225706e-01,
         2.68694449e-01,  2.08803508e-01,  7.16881470e-01,
         1.88297271e-01,  7.24537446e-01,  7.41678001e-01,
         1.47738858e-01,  2.69187785e-01,  3.80075463e-01,
         5.54287963e-01,  9.14861462e-02, -5.65308889e-03],
       [-1.14344516e-01,  4.74127974e-01, -1.04023567e-01,
        -7.44559147e-02,  2.98263375e-02,  3.24016491e-02,
         5.47255199e-02, -1.22764949e-02,  1.19614367e-01,
         1.59799780e-01,  2.44763092e-01,  8.61307733e-01,
         2.47601418e-01,  1.53873516e-01,  4.64051890e-01,
         2.46084921e-01,  2.06120157e-01,  2.28231596e-01,
         4.20636142e-01,  2.91439338e-01, -1.14378089e-01,
         4.06918662e-01, -1.06072420e-01, -7.85921724e-02,
        -5.19054264e-02, -9.33197877e-02, -7.87951059e-02,
        -1.33320547e-01, -9.39008741e-02, -3.96787689e-02],
       [ 7.19123684e-01,  2.74473807e-01,  7.33462019e-01,
         7.57140226e-01,  3.21839377e-01,  5.40181563e-01,
         6.78211471e-01,  7.43327404e-01,  3.28534772e-01,
        -3.57752538e-03,  8.61791821e-01,  2.47601418e-01,
         1.00057777e+00,  8.35599994e-01,  2.08695664e-01,
         4.13408357e-01,  3.91000450e-01,  5.82444702e-01,
         2.87569036e-01,  2.51551602e-01,  7.13001161e-01,
         1.96067208e-01,  7.23943449e-01,  7.35384613e-01,
         1.45931676e-01,  3.07053467e-01,  4.17381863e-01,
         5.73235317e-01,  1.17021610e-01,  3.31069496e-02],
       [ 7.77820405e-01,  2.55692355e-01,  7.84871291e-01,
         8.11259695e-01,  2.78052145e-01,  4.72220256e-01,
         6.28317067e-01,  7.35144357e-01,  2.40512688e-01,
        -1.27894263e-01,  8.45504685e-01,  1.53873516e-01,
         8.35599994e-01,  9.84800040e-01,  1.13965438e-01,
         2.85044327e-01,  2.74865624e-01,  4.88147312e-01,
         1.60075041e-01,  1.14004534e-01,  7.71498680e-01,
         1.86969151e-01,  7.74041231e-01,  7.91001801e-01,
         1.28252762e-01,  2.58833708e-01,  3.75341812e-01,
         5.72583691e-01,  6.79431393e-02, -4.59878726e-02],
       [-2.47738498e-01, -1.50720779e-02, -2.24788884e-01,
        -1.95160988e-01,  3.98533353e-01,  1.60757176e-01,
         9.48027231e-02,  2.90355316e-02,  1.91638752e-01,
         4.73666545e-01,  2.13415127e-01,  4.64051890e-01,
         2.08695664e-01,  1.13965438e-01,  8.42475322e-01,
         2.99565201e-01,  2.55078074e-01,  3.18304567e-01,
         3.68793154e-01,  4.41654881e-01, -2.42751856e-01,
        -6.79449217e-02, -2.26751640e-01, -1.95234582e-01,
         2.55674334e-01, -5.98422790e-02, -7.08164006e-02,
        -1.05881968e-01, -1.40029269e-01,  1.35320730e-01],
       [ 2.00129486e-01,  1.89430823e-01,  2.43031361e-01,
         2.10129486e-01,  3.33705387e-01,  7.04832049e-01,
         6.93486270e-01,  5.03806868e-01,  4.33086475e-01,
         5.79282441e-01,  3.64712328e-01,  2.46084921e-01,
         4.13408357e-01,  2.85044327e-01,  2.99565201e-01,
         1.01330522e+00,  8.26802157e-01,  7.49986132e-01,
         3.80099719e-01,  7.86990801e-01,  2.04514063e-01,
         1.46994116e-01,  2.55083945e-01,  2.00856437e-01,
         2.36690373e-01,  6.18751609e-01,  6.53660380e-01,
         4.97101621e-01,  2.81942181e-01,  5.93173547e-01],
       [ 2.03550841e-01,  1.44939552e-01,  2.40813668e-01,
         2.12555582e-01,  2.30915292e-01,  6.20790369e-01,
         6.35407645e-01,  4.50759409e-01,  3.27051251e-01,
         4.90602813e-01,  3.43881042e-01,  2.06120157e-01,
         3.91000450e-01,  2.74865624e-01,  2.55078074e-01,
         8.26802157e-01,  9.43794078e-01,  7.22973450e-01,
         3.18508738e-01,  7.52285154e-01,  1.95058415e-01,
         8.71234385e-02,  2.40829209e-01,  1.92388920e-01,
         1.28984370e-01,  5.38313590e-01,  5.90414620e-01,
         4.38915556e-01,  1.74834959e-01,  5.03689511e-01],
       [ 3.70133453e-01,  1.50198488e-01,  4.04049201e-01,
         3.93853441e-01,  3.39769952e-01,  6.51462538e-01,
         6.97609842e-01,  5.86944269e-01,  3.54183200e-01,
         4.04945499e-01,  5.51225706e-01,  2.28231596e-01,
         5.82444702e-01,  4.88147312e-01,  3.18304567e-01,
         7.49986132e-01,  7.22973450e-01,  8.82372072e-01,
         3.24682269e-01,  6.59729956e-01,  3.63365585e-01,
         8.16377201e-02,  4.01083311e-01,  3.73114340e-01,
         1.96144624e-01,  4.91018623e-01,  5.63737126e-01,
         5.11802029e-01,  1.42240798e-01,  3.97510384e-01],
       [-1.24463551e-01,  1.23914887e-06, -1.01692897e-01,
        -9.09746465e-02,  1.91703094e-01,  2.22509336e-01,
         1.71845962e-01,  8.54015499e-02,  5.49847100e-01,
         3.51634381e-01,  2.68694449e-01,  4.20636142e-01,
         2.87569036e-01,  1.60075041e-01,  3.68793154e-01,
         3.80099719e-01,  3.18508738e-01,  3.24682269e-01,
         9.32906041e-01,  3.80211484e-01, -1.31204117e-01,
        -6.29009408e-02, -1.06795787e-01, -1.08267888e-01,
        -2.14383602e-02,  5.26472002e-02,  3.10215466e-02,
        -3.21744943e-02,  3.55699823e-01,  9.06561869e-02],
       [-5.83643764e-02,  4.88161300e-02, -1.81284390e-02,
        -3.94979415e-02,  2.98775901e-01,  5.23984861e-01,
         4.79589796e-01,  2.70915784e-01,  3.20117786e-01,
         6.35522779e-01,  2.08803508e-01,  2.91439338e-01,
         2.51551602e-01,  1.14004534e-01,  4.41654881e-01,
         7.86990801e-01,  7.52285154e-01,  6.59729956e-01,
         3.80211484e-01,  9.43672839e-01, -6.28411103e-02,
        -5.71125775e-03, -1.64568483e-02, -5.34964118e-02,
         1.86175086e-01,  4.16566296e-01,  4.34588792e-01,
         2.48422954e-01,  1.06344662e-01,  5.14666043e-01],
       [ 9.44932010e-01,  3.59291001e-01,  9.48135074e-01,
         9.43600212e-01,  2.20766641e-01,  5.42760449e-01,
         7.01628418e-01,  8.30855199e-01,  1.90936684e-01,
        -2.50401889e-01,  7.16881470e-01, -1.14378089e-01,
         7.13001161e-01,  7.71498680e-01, -2.42751856e-01,
         2.04514063e-01,  1.95058415e-01,  3.63365585e-01,
        -1.31204117e-01, -6.28411103e-02,  1.10039923e+00,
         3.57181555e-01,  9.61145815e-01,  9.45546817e-01,
         2.09258938e-01,  4.81685560e-01,  5.85538209e-01,
         7.84701505e-01,  2.30434164e-01,  8.57774446e-02],
       [ 3.06532654e-01,  8.62537569e-01,  3.13239505e-01,
         2.95412154e-01,  4.14956784e-02,  2.61107074e-01,
         3.05917128e-01,  3.03154423e-01,  9.20377145e-02,
        -4.56616514e-02,  1.88297271e-01,  4.06918662e-01,
         1.96067208e-01,  1.86969151e-01, -6.79449217e-02,
         1.46994116e-01,  8.71234385e-02,  8.16377201e-02,
        -6.29009408e-02, -5.71125775e-03,  3.57181555e-01,
         1.05460395e+00,  3.63881334e-01,  3.41045851e-01,
         2.06551608e-01,  3.66305762e-01,  3.67770993e-01,
         3.55007849e-01,  2.13875270e-01,  2.18953086e-01],
       [ 9.44600182e-01,  3.64882515e-01,  9.50841073e-01,
         9.42990092e-01,  2.48259261e-01,  5.87057057e-01,
         7.39111593e-01,  8.55272436e-01,  2.25592717e-01,
        -2.04335461e-01,  7.24537446e-01, -1.06072420e-01,
         7.23943449e-01,  7.74041231e-01, -2.26751640e-01,
         2.55083945e-01,  2.40829209e-01,  4.01083311e-01,
        -1.06795787e-01, -1.64568483e-02,  9.61145815e-01,
         3.63881334e-01,  1.10692507e+00,  9.45981290e-01,
         2.34356525e-01,  5.25110744e-01,  6.26802904e-01,
         8.13272814e-01,  2.61357456e-01,  1.33191039e-01],
       [ 9.33429194e-01,  3.51942002e-01,  9.36319032e-01,
         9.37308693e-01,  2.28840783e-01,  5.26855539e-01,
         6.89045977e-01,  8.21748313e-01,  1.84397469e-01,
        -2.48501660e-01,  7.41678001e-01, -7.85921724e-02,
         7.35384613e-01,  7.91001801e-01, -1.95234582e-01,
         2.00856437e-01,  1.92388920e-01,  3.73114340e-01,
        -1.08267888e-01, -5.34964118e-02,  9.45546817e-01,
         3.41045851e-01,  9.45981290e-01,  1.07741126e+00,
         2.01099729e-01,  4.43643983e-01,  5.51148377e-01,
         7.57517859e-01,  1.94402209e-01,  5.62764752e-02],
       [ 1.26030234e-01,  8.98529661e-02,  1.59799654e-01,
         1.28265771e-01,  7.88325477e-01,  6.01083696e-01,
         4.52959684e-01,  4.68049232e-01,  4.54625515e-01,
         5.45853737e-01,  1.47738858e-01, -5.19054264e-02,
         1.45931676e-01,  1.28252762e-01,  2.55674334e-01,
         2.36690373e-01,  1.28984370e-01,  1.96144624e-01,
        -2.14383602e-02,  1.86175086e-01,  2.09258938e-01,
         2.06551608e-01,  2.34356525e-01,  2.01099729e-01,
         1.00772979e+00,  5.74461261e-01,  4.93771711e-01,
         5.24517068e-01,  4.53822310e-01,  6.30863597e-01],
       [ 4.25042454e-01,  2.89110719e-01,  4.64086582e-01,
         4.01347136e-01,  5.00388699e-01,  8.26078562e-01,
         7.73180063e-01,  6.88157336e-01,  5.10172452e-01,
         4.62560491e-01,  2.69187785e-01, -9.33197877e-02,
         3.07053467e-01,  2.58833708e-01, -5.98422790e-02,
         6.18751609e-01,  5.38313590e-01,  4.91018623e-01,
         5.26472002e-02,  4.16566296e-01,  4.81685560e-01,
         3.66305762e-01,  5.25110744e-01,  4.43643983e-01,
         5.74461261e-01,  1.02619322e+00,  8.69102813e-01,
         8.00543973e-01,  6.11538700e-01,  7.57990415e-01],
       [ 5.41489854e-01,  3.17026535e-01,  5.77792027e-01,
         5.21940187e-01,  4.44807972e-01,  8.36322404e-01,
         8.31402906e-01,  7.55522068e-01,  4.52985364e-01,
         3.81101060e-01,  3.80075463e-01, -7.87951059e-02,
         4.17381863e-01,  3.75341812e-01, -7.08164006e-02,
         6.53660380e-01,  5.90414620e-01,  5.63737126e-01,
         3.10215466e-02,  4.34588792e-01,  5.85538209e-01,
         3.67770993e-01,  6.26802904e-01,  5.51148377e-01,
         4.93771711e-01,  8.69102813e-01,  1.02803607e+00,
         8.40711840e-01,  5.23773326e-01,  6.89880957e-01],
       [ 7.39922977e-01,  3.06968043e-01,  7.67968363e-01,
         7.28712432e-01,  4.97349918e-01,  8.18754543e-01,
         8.55587912e-01,  8.74372333e-01,  4.34924218e-01,
         2.07626742e-01,  5.54287963e-01, -1.33320547e-01,
         5.73235317e-01,  5.72583691e-01, -1.05881968e-01,
         4.97101621e-01,  4.38915556e-01,  5.11802029e-01,
        -3.21744943e-02,  2.48422954e-01,  7.84701505e-01,
         3.55007849e-01,  8.13272814e-01,  7.57517859e-01,
         5.24517068e-01,  8.00543973e-01,  8.40711840e-01,
         1.05149800e+00,  4.90489653e-01,  5.36938333e-01],
       [ 1.69140312e-01,  1.15160075e-01,  1.97765195e-01,
         1.45968333e-01,  4.24980666e-01,  5.46918841e-01,
         4.25380263e-01,  3.97306007e-01,  7.01633270e-01,
         3.59262141e-01,  9.14861462e-02, -9.39008741e-02,
         1.17021610e-01,  6.79431393e-02, -1.40029269e-01,
         2.81942181e-01,  1.74834959e-01,  1.42240798e-01,
         3.55699823e-01,  1.06344662e-01,  2.30434164e-01,
         2.13875270e-01,  2.61357456e-01,  1.94402209e-01,
         4.53822310e-01,  6.11538700e-01,  5.23773326e-01,
         4.90489653e-01,  1.00780968e+00,  5.27336361e-01],
       [ 2.49400180e-02,  1.29585260e-01,  6.89850961e-02,
         5.72488968e-03,  5.47267058e-01,  6.92301471e-01,
         5.49098934e-01,  4.04193473e-01,  4.72514568e-01,
         6.84002885e-01, -5.65308889e-03, -3.96787689e-02,
         3.31069496e-02, -4.59878726e-02,  1.35320730e-01,
         5.93173547e-01,  5.03689511e-01,  3.97510384e-01,
         9.06561869e-02,  5.14666043e-01,  8.57774446e-02,
         2.18953086e-01,  1.33191039e-01,  5.62764752e-02,
         6.30863597e-01,  7.57990415e-01,  6.89880957e-01,
         5.36938333e-01,  5.27336361e-01,  9.67589384e-01]])

x_pca.shape

(569, 6)

x_pca

array([[ 9.19283683,  1.94858307, -1.12316616, -3.6337309 ,  1.19511012,
         1.41142445],
       [ 2.3878018 , -3.76817174, -0.52929269, -1.11826386, -0.62177498,
         0.02865635],
       [ 5.73389628, -1.0751738 , -0.55174759, -0.91208267,  0.1770859 ,
         0.54145215],
       ...,
       [ 1.25617928, -1.90229671,  0.56273053,  2.08922702, -1.80999133,
        -0.53444719],
       [10.37479406,  1.67201011, -1.87702933,  2.35603113,  0.03374193,
         0.56793647],
       [-5.4752433 , -0.67063679,  1.49044308,  2.29915714,  0.18470331,
         1.61783736]], shape=(569, 6))

# Convert to dataframe
component_names = [f"PC{i+1}" for i in range(x_pca.shape[1])]
x_pca_df = pd.DataFrame(x_pca, columns=component_names)

x_pca_df.head()

	PC1	PC2	PC3	PC4	PC5	PC6
0	9.192837	1.948583	-1.123166	-3.633731	1.195110	1.411424
1	2.387802	-3.768172	-0.529293	-1.118264	-0.621775	0.028656
2	5.733896	-1.075174	-0.551748	-0.912083	0.177086	0.541452
3	7.122953	10.275589	-3.232790	-0.152547	2.960878	3.053422
4	3.935302	-1.948072	1.389767	-2.940639	-0.546747	-1.226495

loadings = pd.DataFrame(
    pca.components_.T,  # transpose the matrix of loadings
    columns=component_names,  # so the columns are the principal components
    index=cancer.feature_names,  # and the rows are the original features
)

loadings

	PC1	PC2	PC3	PC4	PC5	PC6
mean radius	0.218902	-0.233857	-0.008531	-0.041409	-0.037786	0.018741
mean texture	0.103725	-0.059706	0.064550	0.603050	0.049469	-0.032179
mean perimeter	0.227537	-0.215181	-0.009314	-0.041983	-0.037375	0.017308
mean area	0.220995	-0.231077	0.028700	-0.053434	-0.010331	-0.001888
mean smoothness	0.142590	0.186113	-0.104292	-0.159383	0.365089	-0.286374
mean compactness	0.239285	0.151892	-0.074092	-0.031795	-0.011704	-0.014131
mean concavity	0.258400	0.060165	0.002734	-0.019123	-0.086375	-0.009344
mean concave points	0.260854	-0.034768	-0.025564	-0.065336	0.043861	-0.052050
mean symmetry	0.138167	0.190349	-0.040240	-0.067125	0.305941	0.356458
mean fractal dimension	0.064363	0.366575	-0.022574	-0.048587	0.044424	-0.119431
radius error	0.205979	-0.105552	0.268481	-0.097941	0.154456	-0.025603
texture error	0.017428	0.089980	0.374634	0.359856	0.191651	-0.028747
perimeter error	0.211326	-0.089457	0.266645	-0.088992	0.120990	0.001811
area error	0.202870	-0.152293	0.216007	-0.108205	0.127574	-0.042864
smoothness error	0.014531	0.204430	0.308839	-0.044664	0.232066	-0.342917
compactness error	0.170393	0.232716	0.154780	0.027469	-0.279968	0.069198
concavity error	0.153590	0.197207	0.176464	-0.001317	-0.353982	0.056343
concave points error	0.183417	0.130322	0.224658	-0.074067	-0.195548	-0.031224
symmetry error	0.042498	0.183848	0.288584	-0.044073	0.252869	0.490246
fractal dimension error	0.102568	0.280092	0.211504	-0.015305	-0.263297	-0.053195
worst radius	0.227997	-0.219866	-0.047507	-0.015417	0.004407	-0.000291
worst texture	0.104469	-0.045467	-0.042298	0.632808	0.092883	-0.050008
worst perimeter	0.236640	-0.199878	-0.048547	-0.013803	-0.007454	0.008501
worst area	0.224871	-0.219352	-0.011902	-0.025895	0.027391	-0.025164
worst smoothness	0.127953	0.172304	-0.259798	-0.017652	0.324435	-0.369255
worst compactness	0.210096	0.143593	-0.236076	0.091328	-0.121804	0.047706
worst concavity	0.228768	0.097964	-0.173057	0.073951	-0.188519	0.028379
worst concave points	0.250886	-0.008257	-0.170344	-0.006007	-0.043332	-0.030873
worst symmetry	0.122905	0.141883	-0.271313	0.036251	0.244559	0.498927
worst fractal dimension	0.131784	0.275339	-0.232791	0.077053	-0.094423	-0.080224

sns.set_style('whitegrid')
plt.figure(figsize=(12,8))
plt.scatter(x_pca[:,0], x_pca[:,1],
            c = cancer['target'], cmap='plasma')
plt.xlabel('PCA1')
plt.ylabel('PCA2')
plt.show()

Using Principal Component Scores for Other Machine Learning Algorithm

x_pca_df2 = x_pca_df.copy()
x_pca_df2['target'] = cancer.target # Adding the target variable with the PCA values 

x_pca_df2.sample(5)

	PC1	PC2	PC3	PC4	PC5	PC6	target
523	-0.619839	0.636349	-0.488851	0.015154	0.106407	-0.600716	1
494	-2.793331	-1.077888	0.844542	1.110208	-0.004870	0.565603	1
427	-2.314247	0.402031	0.274623	1.670463	0.536711	0.508375	1
101	-4.555029	3.528786	-0.215264	-1.091886	2.758349	-0.900493	1
269	-0.739743	3.152521	1.469858	0.314894	-0.947689	-0.531441	1

x_pca_df2['target'].value_counts()

target
1    357
0    212
Name: count, dtype: int64

final_df = x_pca_df2.copy()

from sklearn.linear_model import LogisticRegression
from sklearn.model_selection import train_test_split
logreg = LogisticRegression()

# Feature and Target Vectors 
Xlog = final_df.drop(['target'], axis=1)
ylog = final_df['target']

# Training and Testing Split 
Xlog_train, Xlog_test, ylog_train, ylog_test = train_test_split(Xlog, ylog, test_size=0.20, random_state=420)

logreg.fit(Xlog_train, ylog_train)

LogisticRegression()

In a Jupyter environment, please rerun this cell to show the HTML representation or trust the notebook.
On GitHub, the HTML representation is unable to render, please try loading this page with nbviewer.org.

ylog_predict = logreg.predict(Xlog_test)

from sklearn.metrics import accuracy_score
accuracy_score(ylog_test, ylog_predict)

0.9649122807017544

Logistic Regression with Original Dataset

# For original dataset 
Xorg = cancer['data']
yorg = cancer['target']


# for original dataset 
Xorg_train, Xorg_test, yorg_train, yorg_test = train_test_split(Xorg, yorg, test_size=0.20, random_state=500) # for original dataset 


# for original dataset 
logreg.fit(Xorg_train, yorg_train)


# for original dataset 
yorg_predict = logreg.predict(Xorg_test)

C:\Users\mshar\OneDrive - Southern Illinois University\BSAN405_MLINBUSINESS_WEBSITE\mlbusiness2\Lib\site-packages\sklearn\linear_model\_logistic.py:465: ConvergenceWarning:

lbfgs failed to converge (status=1):
STOP: TOTAL NO. OF ITERATIONS REACHED LIMIT.

Increase the number of iterations (max_iter) or scale the data as shown in:
    https://scikit-learn.org/stable/modules/preprocessing.html
Please also refer to the documentation for alternative solver options:
    https://scikit-learn.org/stable/modules/linear_model.html#logistic-regression

# for original dataset 
accuracy_score(yorg_test, yorg_predict)

0.9473684210526315

The accuracy score in original dataset is about 95%, which is less than principal component score result where accuracy is about 98%.

Conclusion