Project 2

Assume you are appointed as a Data scientist in any international humanitarian NGO, after the recent funding programmes, have been able to raise around $ 120 million. Now the CEO of the NGO call you to choose how to use this money strategically and effectively. The significant issues that comes while making this conclusion are mostly related to choosing the countries that are in the direst need of aid. Your job is to classify the countries using some socio-economic and health factors that determine the overall development of the country. Then you need to suggest the countries which the CEO needs to focus on the most. Apply Principal component analysis, K-Means Clustering & Hierarchical Clustering.

import pandas as pd

import numpy as np

import matplotlib.pyplot as plt

from sklearn.cluster import AgglomerativeClustering

from sklearn.preprocessing import StandardScaler

from scipy.cluster.hierarchy import dendrogram, linkage, fcluster

data = pd.read_csv('Country_data.csv', index_col=False, na_values=["?"])

df1 = pd.read_csv('Country_data.csv', index_col=False, na_values=["?"])

df = df1.copy()

print(data)

# Collecting features

features = list(df.columns)

print(features)

X1 = df[features]

# Data Preprocessing

X = pd.get_dummies(X1)

X = StandardScaler().fit_transform(X)

dendrogram = dendrogram(linkage(X,method='ward'))

clf = AgglomerativeClustering(n_clusters=2, affinity='euclidean', linkage='ward')

clf.fit(X)

labels = clf.labels_

data['anomaly'] = labels

outliers = data.loc[data['anomaly']==1]

outliers_index = list(outliers.index)

print(outliers_index)

from sklearn.decomposition import PCA

from sklearn.preprocessing import StandardScaler

from mpl_toolkits.mplot3d import Axes3D

pca = PCA(n_components=3)

scaler = StandardScaler()

X = scaler.fit_transform(X)

X_reduce = pca.fit_transform(X)

fig = plt.figure()

ax = fig.add_subplot(111, projection = '3d')

ax.scatter(X_reduce[:,0], X_reduce[:,1], zs = X_reduce[:,2], s=4, lw=1,

label="inlines", c="green")

ax.scatter(X_reduce[outliers_index,0], X_reduce[outliers_index,1],X_reduce[outliers_index,2],

lw=2, s=60, marker="x", c="red", label="outliers")

ax.legend()

plt.show()