Python: Data Analysis
AIM: To perform data analysis on engine data using python.
OBJECTIVES:
To perform data analysis and
- plot a graph between any two quantities (like pressure vs volume, pressure vs crank, crank vs volume, etc.)
- calculate the area under P-V diagram
- calculate the power output of an engine running at a speed of 1500 RPM
- calculate the specific fuel consumption for an engine which consumes 20 micro grams of fuel (per 1 stroke cycle)
THEORY:
Data analysis is a process of inspecting, cleansing, transforming and modelling data with the goal of discovering useful information, informing conclusions and supporting decision-making.
Data Analysis consists of the following phases:
- a) Data Requirement Gathering
The data are necessary as inputs to the analysis, which is specified based upon the requirements of those directing the analysis. The general type of entity upon which the data will be collected is referred to as an experimental unit. Data may be numerical or categorical.
- b) Data Collection
Data are collected from a variety of sources. The requirements may be communicated by analysts to custodians of the data, such as information technology personnel within an organization. The data may also be collected from sensors in the environment, such as traffic cameras, satellites, recording devices, etc. It may also be obtained through interviews, downloads from online sources, or reading documentation.
- c) Data Processing
Data initially obtained must be processed or organised for analysis. For instance, these may involve placing data into rows and columns in a table format (i.e., structured data) for further analysis, such as within a spreadsheet or statistical software.
- d) Data Cleaning
Once processed and organised, the data may be incomplete, contain duplicates, or contain errors. The need for data cleaning will arise from problems in the way that data are entered and stored. Data cleaning is the process of preventing and correcting these errors.
- e) Data Analysis
Once the data are cleaned, it can be analyzed. Analysts may apply a variety of techniques referred to as exploratory data analysis to begin understanding the messages contained in the data. The process of exploration may result in additional data cleaning or additional requests for data, so these activities may be iterative in nature.
- f) Data Interpretation
After analyzing your data, it's finally time to interpret your results. You can choose the way to express or communicate your data analysis either you can use simply in words or maybe a table or chart. Then use the results of your data analysis process to decide your best course of action.
- g) Data Visualization
Data visualization is very common in your day to day life; they often appear in the form of charts and graphs. In other words, data shown graphically so that it will be easier for the human brain to understand and process it. Data visualization often used to discover unknown facts and trends. By observing relationships and comparing datasets, you can find a way to find out meaningful information.
GOVERNING EQUATION:
Indicated (Total) Power Output without consideration of friction losses which is equal to Brake Power Output,
Specific Fuel Consumption,
INPUTS: Link to data file
STEPS (WITH EXPLAINATION):
FOR DATA VISUALISATION AND COMPATIBILITY CHECK
First we import the basic modules viz. math, numpy (as np) and matplotlib.pyplot (as plt).Now, we will check compatibility of the data file with the help of try command and except for execution of any error in the data file. In the else command, desired inputs are assigned to read column number, column name and column units as an empty array.
Now, we will execute for loop is used for reading the file line by line and storing the required data using open command. Also, one more variable count is used for keeping the count of the line number which is incremented at the end of every iteration. Inside the primary for loop, two secondary for loops one for storing the column number and the other for storing the column name and units are provided.
Now, the first for loop which varies from 0 to 18 with 19 iterations, a if condition is used for value greater than or equal to 2 to avoid storing of the non-required characters. Then secondary if condition is used, when count is equal to 2 then, an empty array 'column_num' gets appended with the values having integer data.
Similarly, another for loops execute in the first for loop from 0 to 17 with 18 iterations. The for loop runs only for 17 times due to condition provided in the primary if condition provide inside this for loop. Inside this loop, an if condition is used for greater than or equal to 1 to avoid storing of the non-required characters. Then, an if else condition is used, firstly an if condition used which executes 'lcount' which is equal to 3 then, an empty array 'column_name' gets appended with the values having string data type. Now an elseif condition is used to execute 'lcount' when equal to 4 then, an empty array 'column_units' gets appended with the values having string data type.
Now, another loop of if condition executes which says that if there is not a hashtag ('#') symbol at the start of the line then, then inside this loop secondary if condition is used, which says that if the variable count is greater than or equal to 6 then, the empty arrays term1 and term2 gets appended with the required values having float data type which means that the empty arrays gets appended with values (for the required two parameters) only from the corresponding line of the data file.
Then the count variable is incremented at the end of every iteration. Then a plot between the two required parameters is obtained using the plt.plot command. The plot is titled using the plt.title command. Also, the axes are labelled using the plt.xlabel and plt.ylabel commands. The figure window or plot is saved using the plt.savefig command.
FOR BASIC PERFORMANCE CALCULATION
First we import the basic modules viz. math, numpy (as np), matplotlib.pyplot (as plt) and from scipy.integrate we import simps.Now, we will check compatibility of the data file with the help of try command and except for execution of any error in the data file.In the else command, desired inputs are assigned to read column number, column name and column units as an empty array.
Now, we will execute for loop for reading the file line by line and storing the required data using open command. Also, one more variable ‘count’ is used for keeping the count of the line number which is incremented at the end of every iteration. Inside the primary for loop, two secondary for loops are created, one for storing the column number and the other for storing the column name and units and these loops if is used for storing desired data.
Now, the first for loop which varies from 0 to 18 with 19 iterations, a if condition is used for value greater than or equal to 2 to avoid storing of the non-required characters. Then secondary if condition is used, when 'count' is equal to 2. Then, an empty array 'column_num' gets appended with the values having integer data.
Similarly, another for loops execute in the first for loop upto 0 to 17 with 18 iterations. The for loop runs only for 17 times due to condition provided in the primary if condition provide inside this for loop. Inside this' loop, a if condition is used upto greater than or equal to 1 to avoid storing of the non-required characters. Then, an 'if else' condition is used; Firstly, an if condition is used which executes 'lcount' which is equal to 3 then, an empty array 'column_name' gets appended with the values having string data type. Now an elseif condition used to executes 'lcount' which is equal to 4. Then, an empty array 'column_units' gets appended with the values having string data type.
Now, another loop of if condition executes which says that if there is not a hashtag ('#') symbol at the start of the line then the inside secondary if condition is used, which says that if the variable count is greater than or equal to 6 then the empty arrays term1 and term2 gets appended with the required values having float data type which means that the empty arrays gets appended with values (for the required two parameters) only from the corresponding line of the data file.
Then the count variable is incremented at the end of every iteration. Then a plot between the two required parameters is obtained using the plt.plot command. The plot is titled using the plt.title command. Also, the axes are labelled using the plt.xlabel and plt.ylabel commands. The figure window or plot is saved using the plt.savefig command.
Now, we will calculate area under PV daigram using scipy.integrate module namely as simps in which we pass both the term1 and term2 arguments with desired initialisation. Then we will calculate and print the value of the power output of the engine and the specific fuel consumption of the engine.
CODE:
FOR DATA VISUALISATION AND COMPATIBILITY CHECK
'''
Program to perfrom data analysis on engine data using python
By Utkarsh Palwekar
'''
import math
import numpy as np
import matplotlib.pyplot as plt
# Compatibilty check of the file with the program
try:
open('engine_data.out')
except:
print('File not recognized. Please provide a valid CONVERGE output file')
else:
# Inputs
# Initiating line count
line_count = 1
column_number = [] # to store the value of column number
name_of_column = [] # to store the column name in the form of string
units = [] # to store the units in the form of string
X = [] # to store the value of x
Y = [] # to store the value of y
# Initialization of value of x and y respectively
x = 0
y = 7
# For loop for reading the file line by line and storing the required data
for line in open('engine_data.out'):
# For loop defined for appending an empty array 'column_number' to store the the value of column number from the line number 2 of the data file
for i in range(0, 19):
# If loop to avoid the storing of non-required characters like the string 'column' and the symbol '#'
if i >= 2:
# If loop to store the values of column number in an empty array 'column_number' from only the line number 2
if line_count == 2:
column_number.append(int(line.split()[i]))
# For loop defined for appending empty arrays 'name_of_column' and 'units' to store the column name and units in the form of string from the line number 3 and 4 respectively of the data file
for j in range(0, 18):
# If loop to avoid the storing of non-required characters like the symbol '#'
if j >= 1:
# If and else if loop
# Storing of the character column name in an empty array 'name_of_column' from the line number 3 if the if condition is true
if line_count == 3:
name_of_column.append(line.split()[j])
# Storing of the character units in an empty array 'units' from the line number 4 if the else if condition is true
elif line_count == 4:
units.append(line.split()[j])
# Storing the values from line having numerical values only after the comments
if '#' not in line:
# If loop for storing of numeric data i.e quantity 1 and quantity 2 from the line number 6 and beyond that line
if line_count >= 6:
X.append(float(line.split()[x]))
Y.append(float(line.split()[y]))
line_count = line_count + 1
# Plotting
plt.figure()
plt.plot(X, Y, color='r', linewidth=2)
plt.title('Plot of ' + str(name_of_column[y]) + ' versus ' + str(name_of_column[x]) + ' for FI Engine', color='k',
fontsize=12, fontweight='bold')
plt.xlabel(str(name_of_column[x]) + str(units[x]), fontweight='bold')
plt.ylabel(str(name_of_column[y]) + str(units[y]), fontweight='bold')
# Saving the plot as an image in PNG format
plt.savefig(str(name_of_column[y]) + ' vs ' + str(name_of_column[x]) + ' for FI Engine.png')
plt.show()
FOR BASIC PERFORMANCE CALCULATION
'''
Program to perform data analysis and
calculate the area under P-V diagram, power output and specific fuel consumption
By Utkarsh Palwekar
'''
import math
import numpy as np
import matplotlib.pyplot as plt
from scipy.integrate import simps
# Compatibilty check of the file with the program
try:
open('engine_data.out')
except:
print('File not recognized. Please provide a valid CONVERGE output file')
else:
# Initial Inputs
line_count = 1 # to keep the line count
column_number = [] # to store the value of column number
name_of_column = [] # to store the column name in the form of string
units = [] # to store the units in the form of string
X = [] # to store the value of x
Y = [] # to store the value of y
# Initialization of value of x as 7 (i.e volume) and y as 1 (i.e pressure) respectively
x = 7
y = 1
# For loop for reading the file line by line and storing the required data
for line in open('engine_data.out'):
# For loop defined for appending an empty array 'column_number' to store the the value of column number from the line number 2 of the data file
for i in range(0, 19):
# If loop to avoid the storing of non-required characters like the string 'column' and the symbol '#'
if i >= 2:
# If loop to store the values of column number in an empty array 'column_number' from only the line number 2
if line_count == 2:
column_number.append(int(line.split()[i]))
# For loop defined for appending empty arrays 'name_of_column' and 'units' to store the column name and units in the form of string from the line number 3 and 4 respectively of the data file
for j in range(0, 18):
# If loop to avoid the storing of non-required characters like the symbol '#'
if j >= 1:
# If and else if loop
# Storing of the character column name in an empty array 'name_of_column' from the line number 3 if the if condition is true
if line_count == 3:
name_of_column.append(line.split()[j])
# Storing of the character units in an empty array 'units' from the line number 4 if the else if condition is true
elif line_count == 4:
units.append(line.split()[j])
# Storing the values from line having numerical values only after the comments
if '#' not in line:
# If loop for storing of numeric data i.e quantity 1 and quantity 2 from the line number 6 and beyond that line
if line_count >= 6:
X.append(float(line.split()[x]))
Y.append(float(line.split()[y]))
# Updating the counter variable
line_count = line_count + 1
# Plotting
plt.figure()
plt.plot(X, Y, color='r', linewidth=2)
plt.title('Plot of ' + str(name_of_column[y]) + ' versus ' + str(name_of_column[x]) + ' for FI Engine', color='k',
fontsize=12, fontweight='bold')
plt.xlabel(str(name_of_column[x]) + str(units[x]), fontweight='bold')
plt.ylabel(str(name_of_column[y]) + str(units[y]), fontweight='bold')
# Saving the plot as an image in PNG format
plt.savefig(str(name_of_column[y]) + ' vs ' + str(name_of_column[x]) + ' for FI Engine.png')
plt.show()
# Printing the required results statement
print('Results:')
print('\t')
# Calculation of area under PV diagram (i.e workdone) using the simpsons rule
# Area i.e workdone in mega joule
area = simps(Y, X)
print('Workdone (i.e Area under PV diagram) calculated using simpsons rule, w = ' + str(area) + ' MJ')
print('\t')
# Workdone in joule
Area = simps(Y, X) * pow(10, 6)
print('Workdone (i.e Area under PV diagram) calculated using simpsons rule, W = ' + str(Area) + ' J')
print('\t')
# Inputs given in the challenge for calculation of power output and specific fuel consumption
N_rpm = 1500 # Speed of an engine (RPM)
# Fuel consumption (micro grams per 1 stroke cycle) i.e 20 micro gram fuel is burnt during 1 complete cycle i.e in 2 revolutions of crankshaft
fc_micro_gram_per_2_rev = 20
# Engine Specifications
no_of_stroke = 4 # Strokes
n = 1 # Number of cylinder
# In a 4 stroke engine there is a power stroke for every two revolutions and therefore there is a factor k which is considered as equal to 2
k = 2
# Power Output (KW)
# Power output is equal to total power output as friction losses are considered to be negligible/almost zero as there is no data given in the data file about friction losses
# power_output = no of cylinder*Area under PV diagram*speed of an engine/(60*1000*k) [The numerator is divided by 1000 to get the power output in kilowatt]
power_output = n * Area * N_rpm / (60 * 1000 * k)
print('Power Ouput at 1500 RPM, power_output = ' + str(power_output) + ' KW')
print('\t')
# Convertion of fuel consumption in micro grams per 1 stroke cycle to fuel consumption in kilogram per hour
# We need to first calculate the speed of an engine in revolutions per hour and then fuel consumption in kilogram per revolution of crankshaft and then multiply both the quantities to get the fuel consumption in kilogram per hour
# Speed of an engine (revolutions per hour) [Mutliplying N_rpm by 60 ]
N_rev_per_hr = N_rpm * 60
# Fuel consumption (kilogram per revolution) [Mutltiplying the fc_micro_gram_per_2_rev by 10^(-9) and dividing by 2]
fc_kg_per_rev = fc_micro_gram_per_2_rev * pow(10, -9) / 2
# Fuel consumption (kilogram per hour)
fc_kg_per_hr = N_rev_per_hr * fc_kg_per_rev
# Specific fuel consumption ((kg/hr)/KW) [Dividing fc_kg_per_hr by power_output]
SFC = fc_kg_per_hr / power_output
print('Specific Fuel Consumption, SFC = ' + str(SFC) + ' (kg/hr)/KW')
RESULTS/OUTPUT:
FOR DATA VISUALISATION AND COMPATIBILITY CHECK
After running our 1st code which is just to visualise data and compatibility checking of the datafile following plot is obtained-
Volume vs Crank for FI Engine Plot
Some other plots can also be obtained by changing the initialisation values of x and y in code line no 34 and 35. Other plots obtained by changing the values of x and y are shown below
Pressure vs Crank Plot
Mean Temperature vs Crank Plot
Mean Temperature vs Pressure Plot
FOR BASIC PERFORMANCE CALCULATION
After running 2nd code which is basically for calculation of basic performance of the engine whose data is given as input via datafile.
Plot for Pressure vs Volume (PV Diagram)
Basic Performance Calculation Results
CONCLUSION:
Finally, we conclude that the program run successfully and the output obtained are right. Thus, python can be used for quick data analysis as it requires less time and lines for coding.
ERRORS & SOLUTIONS:
Minor typo errors were faced which was corrected when encountered.
The flow of program was getting complex so I have to rearrange code many times.
A lot of confusion was created due to so many loops were used. A lot of indentation errors were faced which were corrected by checking the loop, by checking the code and giving proper spacing.
REFERENCES:
https://www.guru99.com/what-is-data-analysis.html
https://en.wikipedia.org/wiki/Data_analysis#Data_collection
https://scipy.github.io/devdocs/generated/scipy.integrate.simps.html?highlight=simp