Project 2 - EDA on Vehicle Insurance Customer Data

EDA on Vehicle Insurance Customer Data

Aim:

The aim of this project is to perform exploratory data analysis (EDA) and data cleaning on two datasets containing customer details and policy details. The objective is to prepare the data for future analysis and modeling, identify patterns, and derive insights to aid business decisions.

Introduction:

The datasets provided contain information about customers and their insurance policies. The data analysis aims to uncover relationships between different variables, identify any data quality issues, clean the data, and derive actionable insights for the company.

Procedure:

Loading the Datasets:

• Two CSV files, customer_details.csv and customer_policy_details.csv, were provided.
• The datasets were loaded into Pandas DataFrames.

Data Quality Check:

• Checked for null values in both datasets and generated a summary of null values column-wise.
• Null values were handled by dropping rows with null customer IDs and replacing null values for numeric columns with the mean, and for categorical columns with the mode.

Outlier Detection and Treatment:

• Outliers were identified using the Interquartile Range (IQR) method.
• Outliers in numeric columns were replaced with the mean.

White Spaces and Case Correction:

• Removed leading and trailing white spaces from string columns.
• Converted string columns to lowercase for consistency.

Categorical Data Encoding:

• Mapped categorical values to numerical values for modeling purposes.

Duplicate Data Removal:

• Removed duplicate rows from both datasets.

Master Table Creation:

• Merged the cleaned customer details and policy details datasets based on the customer_id column.

Data Analysis:

• Calculated gender-wise and age-wise average annual premium.
• Checked if the data is balanced between genders.
• Analyzed the average annual premium based on vehicle age.
• Examined the correlation between person age and annual premium using correlation coefficient and heatmap.

Methodology:

• Utilized Pandas for data manipulation and cleaning.
• Used statistical techniques such as mean and mode imputation, and IQR method for outlier detection.
• Employed Pandas' groupby() function for generating summary statistics.
• Visualized correlation using Seaborn's heatmap.

Code Work Explained:

• Loaded datasets using Pandas.
• Performed data cleaning steps including handling null values, outliers, white spaces, and case correction.
• Encoded categorical data into numerical values.
• Created a master table by merging customer details and policy details.
• Conducted data analysis to derive insights.

Errors/Blockers/Challenges:

• Handling missing values and outliers appropriately without affecting data integrity.
• Ensuring consistent data formatting across columns and datasets.
• Dealing with potential data imbalances and biases during analysis.

Results:

• Cleaned datasets ready for analysis and modeling.
• Insights derived regarding average annual premium by gender, age, vehicle age, and correlation with person age.

Conclusion:

The exploratory data analysis and data cleaning process provided valuable insights into the datasets and prepared them for further analysis. The derived insights can guide strategic decisions regarding insurance policies, customer segmentation, and premium pricing.

This report serves as a foundation for subsequent analyses and modeling tasks, aiding in the company's objective of improving customer satisfaction