Predicting Heart Disease: A Comparative Analysis of Machine Learning Algorithms

Project Overview

The objective of this project is to predict whether a patient has heart disease based on several clinical parameters using multiple machine learning algorithms. The algorithms used include Logistic Regression, K-Nearest Neighbors Classifier, Support Vector Machine, Decision Tree Classifier, and Random Forest Classifier. We will compare the accuracy of each model to determine the best one for predicting heart disease.

Dataset Description

The dataset contains various clinical features of patients, including:

• age: Age in years
• sex: (1 = male; 0 = female)
• cp: Chest pain type
  • 0: Typical angina: chest pain related to decreased blood supply to the heart
  • 1: Atypical angina: chest pain not related to the heart
  • 2: Non-anginal pain: typically esophageal spasms (non-heart related)
  • 3: Asymptomatic: chest pain not showing signs of disease
• trtbps: Resting blood pressure (in mm Hg on admission to the hospital)
• chol: Serum cholesterol in mg/dl
• fbs: (fasting blood sugar > 120 mg/dl) (1 = true; 0 = false)
• restecg: Resting electrocardiographic results
  • 0: Nothing to note
  • 1: ST-T Wave abnormality
  • 2: Possible or definite left ventricular hypertrophy
• thalachh: Maximum heart rate achieved
• exng: Exercise induced angina (1 = yes; 0 = no)
• oldpeak: ST depression induced by exercise relative to rest
• slp: The slope of the peak exercise ST segment
  • 0: Upsloping: better heart rate with exercise (uncommon)
  • 1: Flat sloping: minimal change (typical healthy heart)
  • 2: Downsloping: signs of an unhealthy heart
• caa: Number of major vessels (0-3) colored by fluoroscopy
• thall: Thallium stress test result
  • 1, 3: Normal
  • 6: Fixed defect
  • 7: Reversible defect
• output: Presence of heart disease (1 = yes, 0 = no)

Data Preparation

1. Importing Libraries and Data: Import necessary libraries and load the dataset.
2. Preprocessing:
   • Encode categorical features using one-hot encoding.
   • Standardize numerical features to have a mean of 0 and a standard deviation of 1.
   • Split the dataset into training and testing sets.

Model Training and Evaluation

Helper Functions

• Grid Search: Perform hyperparameter tuning using GridSearchCV to find the best parameters for each model.
• Train: Train the model using the training data.
• Evaluate: Evaluate the model's performance using accuracy, precision, recall, and F-score.

Applying Machine Learning Models

1. Logistic Regression:

   • Hyperparameters: C (regularization strength), solver, penalty, max_iter, class_weight, multi_class.
   • Performance:
     • Accuracy: 0.8146
     • Precision: 0.7778
     • Recall: 0.8835
     • F-score: 0.8273

2. K-Nearest Neighbors Classifier:

   • Hyperparameters: n_neighbors, weights, algorithm, p.
   • Performance:
     • Accuracy: 0.9854
     • Precision: 1.0000
     • Recall: 0.9709
     • F-score: 0.9852

3. Support Vector Machine:

   • Hyperparameters: C, gamma, kernel.
   • Performance:
     • Accuracy: 0.9854
     • Precision: 1.0000
     • Recall: 0.9709
     • F-score: 0.9852

4. Decision Tree Classifier:

   • Hyperparameters: criterion, splitter, max_depth, min_samples_split, min_samples_leaf, max_features.
   • Performance:
     • Accuracy: 1.0000
     • Precision: 1.0000
     • Recall: 1.0000
     • F-score: 1.0000

5. Random Forest Classifier:

   • Hyperparameters: n_estimators, max_depth, min_samples_split, min_samples_leaf, max_features, bootstrap.
   • Performance:
     • Accuracy: 0.9853
     • Precision: 1.0000
     • Recall: 0.9708
     • F-score: 0.6852

Summary of Results

The Decision Tree Classifier achieved perfect accuracy, precision, recall, and F-score on the test set, indicating that they are the best models for predicting heart disease in this dataset. However, these results might indicate overfitting, and further validation on an independent dataset would be necessary to confirm their performance.

Conclusion

The project successfully applied and evaluated five different machine learning models to predict heart disease based on clinical parameters. The Decision Tree Classifier and Random Forest Classifier outperformed other models in terms of accuracy and other metrics.