Project 1

To predict whether a patient has heart disease based on clinical parameters, we'll proceed by applying the following machine learning models:

1. Logistic Regression
2. K-Nearest Neighbors (KNN) Classifier
3. Support Vector Machine (SVM)
4. Decision Tree Classifier
5. Random Forest Classifier

We will follow these steps:

1. Data Preprocessing: Load and clean the data, handle missing values, encode categorical variables, and split the data into training and testing sets.
2. Modeling: Build the models using the respective algorithms.
3. Model Evaluation: Evaluate each model's accuracy and performance.
4. Comparison: Compare the accuracy of all models to find the best performer.

Steps in Python:

We'll assume that the data is in CSV format, which can be processed efficiently.

Code Implementation:

import pandas as pd

from sklearn.model_selection import train_test_split

from sklearn.preprocessing import LabelEncoder

from sklearn.linear_model import LogisticRegression

from sklearn.neighbors import KNeighborsClassifier

from sklearn.svm import SVC

from sklearn.tree import DecisionTreeClassifier

from sklearn.ensemble import RandomForestClassifier

from sklearn.metrics import accuracy_score

# Step 1: Load and inspect the dataset

# Assume the dataset is loaded as 'heart_data.csv' (you will replace this with your file path)

file_path = 'heart_data.csv'

data = pd.read_csv(file_path)

# Check the first few rows and info to ensure data is loaded correctly

print(data.head())

print(data.info())

# Step 2: Preprocessing

# Handle missing values (if any)

data = data.dropna() # Alternatively, you can use imputation

# Encode categorical variables (e.g., sex, cp, restecg, exng, slp, thall)

label_encoder = LabelEncoder()

data['sex'] = label_encoder.fit_transform(data['sex'])

data['cp'] = label_encoder.fit_transform(data['cp'])

data['restecg'] = label_encoder.fit_transform(data['restecg'])

data['exng'] = label_encoder.fit_transform(data['exng'])

data['slp'] = label_encoder.fit_transform(data['slp'])

data['thall'] = label_encoder.fit_transform(data['thall'])

# Define features (X) and target (y)

X = data.drop('output', axis=1)

y = data['output']

# Split the data into training and testing sets

X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)

# Step 3: Model Training and Evaluation

# 1. Logistic Regression

logreg = LogisticRegression(max_iter=1000)

logreg.fit(X_train, y_train)

y_pred_logreg = logreg.predict(X_test)

accuracy_logreg = accuracy_score(y_test, y_pred_logreg)

# 2. K-Nearest Neighbors

knn = KNeighborsClassifier(n_neighbors=5)

knn.fit(X_train, y_train)

y_pred_knn = knn.predict(X_test)

accuracy_knn = accuracy_score(y_test, y_pred_knn)

# 3. Support Vector Machine (SVM)

svm = SVC(kernel='linear') # Linear kernel

svm.fit(X_train, y_train)

y_pred_svm = svm.predict(X_test)

accuracy_svm = accuracy_score(y_test, y_pred_svm)

# 4. Decision Tree Classifier

dtree = DecisionTreeClassifier(random_state=42)

dtree.fit(X_train, y_train)

y_pred_dtree = dtree.predict(X_test)

accuracy_dtree = accuracy_score(y_test, y_pred_dtree)

# 5. Random Forest Classifier

rf = RandomForestClassifier(n_estimators=100, random_state=42)

rf.fit(X_train, y_train)

y_pred_rf = rf.predict(X_test)

accuracy_rf = accuracy_score(y_test, y_pred_rf)

# Step 4: Compare accuracies

print(f"Logistic Regression Accuracy: {accuracy_logreg * 100:.2f}%")

print(f"K-Nearest Neighbors Accuracy: {accuracy_knn * 100:.2f}%")

print(f"Support Vector Machine Accuracy: {accuracy_svm * 100:.2f}%")

print(f"Decision Tree Classifier Accuracy: {accuracy_dtree * 100:.2f}%")

print(f"Random Forest Classifier Accuracy: {accuracy_rf * 100:.2f}%")

Explanation:

1. Data Preprocessing:

   • We handle missing data using dropna() (you can also use imputation if necessary).
   • Categorical variables (such as sex, cp, restecg, exng, slp, thall) are encoded using LabelEncoder to convert them into numeric format.
   • The dataset is then split into training and testing sets (80% for training, 20% for testing).

2. Modeling:

   • Logistic Regression, K-Nearest Neighbors, Support Vector Machine, Decision Tree, and Random Forest classifiers are trained on the dataset using fit().

3. Model Evaluation:

   • We use the accuracy score to evaluate the performance of each model.

4. Comparison:

   • The accuracy scores for all models are printed, allowing us to compare their performance.

Output:

The model will output the accuracy for each of the classifiers, like:

Logistic Regression Accuracy: 85.62%

K-Nearest Neighbors Accuracy: 90.15%

Support Vector Machine Accuracy: 88.10%

Decision Tree Classifier Accuracy: 82.45%

Random Forest Classifier Accuracy: 92.35%

You can further improve the models by tuning hyperparameters (e.g., adjusting the number of neighbors in KNN or the depth of the Decision Tree).