Week 6 - Data analysis

Data Analysis and file parsing using Python:

Objective: To write a Python program that takes in CONVERGE output files and analyses the data and helps represent it in a simpler manner by fulfilling visualization, compatibility criteria and performing simple performance calculations with user inputs.

Solution:

Firstly, using the ‘for’ loop, and ‘open(‘’)’ command, the file is read line by line. But to make sure the file is input by the user, the ‘input(‘’)’ prompt was used. To make sure the input file is compatible with the code the prompt is kept informative. In order for the code to exit gracefully, the ‘try:.....except:..’ loop format was used such that, the code runs the ‘try’ loop if the file is compatible, otherwise it skips to the ‘except’ loop where an informative command is printed to make the user know the possible error.

Now to store all the data in the file ‘for’ loops are used. As mentioned, we can only read the data line by line, hence to store the data column-wise, ‘line. split()’ command is used to isolate each parameter in its line. Now, these split parameters are stored as a string in an array using an ‘append’ operation, such that, every ‘n^th ’ parameter in all the lines is appended into the same array. This way, all the column parameters are stored in separate arrays.

To make things easy when calling for certain column parameters, the same method is used to store the column numbers, names, and units in separate arrays as ‘strings’ instead of ‘float or ‘integers. Further, the ‘strings’ from the column number array are equated to the data array such that the column number ‘string’ is replaced by the data. This way by requesting the column number, all the data in that column can be accessed.

Now to get a plot, the ‘input’ prompt is used to request column numbers, these numbers are stored in 2 variables and again using the data arrays, these variables are co-related to respective columns and plotted using ‘matplotlib’ module. The arrays for column names and units were used in a similar way to give labels and titles to these plots as well as unique filenames to save separately.

Now, to calculate the performance parameters following formulas were used:

Area under the P-V curve = Work done per cycle - ‘T’

Units of the area under the curve will be a product of the axes parameters = M Pa*M³ = (1e6)N/m^2 * m^3 = M N-m. = M joules

Output Power P = Work done * speed = M N-m * rad/s = M Nm/s = M joules/s = M Watts

Assuming a 4-stroke, single-cylinder engine, with 2 revolutions per stroke-cycle, at ‘N’ RPMs of the crankshaft,

P = T*(N/2*60*1000) kW

Brake Specific Fuel Consumption is the Fuel consumed in g/s per Output power produced in joules/s or kW-h, and has units g/j or g/kW-s or g/kW-hr

Mass flow in grams/second = micro-gram/cycle*[(RPM/2*60) or cycles/s] *1e-6

Mass flow ‘F’ = f*(N/120)*1e-6 [g/s]

where ‘f’ is the mass flow in microgram/cycle

BSFC = F/P = [g/s * 1/kW] = [g/kW-s]

Python Code:

#File parsing and data analysis

import matplotlib.pyplot as plt

from numpy import trapz

import numpy as np

from scipy.integrate import simps

import math

#Defining arrays for variables

ct = 1

crank_angle = []

pressure=[]

max_pressure = []

min_pressure =[]

mean_temperature =[]

max_temperature =[]

min_temperature =[]

volume=[]

mass=[]

density =[]

integrated_HR=[]

HR_rate=[]

c_p=[]

c_v=[]

gamma=[]

kin_viscosity=[]

dyn_viscosity=[]

data =[]

x=[]

y=[]

z=[]

f=0

q = 1

# (int(len(line.split())) + 2)

j = input('Please enter the CONVERGE output file name(without the [.out] extension):')

try:

for line in open('engine_data.out'):

m = int(len(line.split()))

for n in range(q + 1, m):

if ct == 2:

a = (str(line.split()[n]))

x.append((a))

#print(n, m)

for n in range(q + 1, m + 1):

if ct == 3:

b = (str(line.split()[(n - 1)]))

y.append(b)

#print(n, m)

if ct == 4:

c = (str(line.split()[(n - 1)]))

z.append(c)

#print(n, m)

ct = ct + 1

except:

print('Please input a valid CONVERGE file name(remove the .out and try again)')

ct1 = 1

d = []

n = []

m=[]

try:

for line in open(str(j) + '.out'):

if '#' not in line:

crank_angle.append(float(line.split()[0]))

x[0] = crank_angle

pressure.append(float(line.split()[1]))

x[1] = pressure

max_pressure.append(float(line.split()[2]))

x[2] = max_pressure

min_pressure.append(float(line.split()[3]))

x[3] = min_pressure

mean_temperature.append(float(line.split()[4]))

x[4] = mean_temperature

max_temperature.append(float(line.split()[5]))

x[5] = max_temperature

min_temperature.append(float(line.split()[6]))

x[6] = min_pressure

volume.append(float(line.split()[7]))

x[7] = volume

mass.append(float(line.split()[8]))

x[8] = mass

density.append(float(line.split()[9]))

x[9] = density

integrated_HR.append(float(line.split()[10]))

x[10] = integrated_HR

HR_rate.append(float(line.split()[11]))

x[11] = HR_rate

c_p.append(float(line.split()[12]))

x[12] = c_p

c_v.append(float(line.split()[13]))

x[13] = c_v

gamma.append(float(line.split()[14]))

x[14] = gamma

kin_viscosity.append(float(line.split()[15]))

x[15] = kin_viscosity

dyn_viscosity.append(float(line.split()[16]))

x[16] = dyn_viscosity

n = int(input('Enter the column number for x-parameter of the plot:'))

m = int(input('Enter the column number for y-parameter of the plot:'))

plt.plot(x[((n) - 1)], x[((m) - 1)])

plt.grid()

plt.xlabel(str(y[((n) - 1)]) + ' in ' + str(z[((n) - 1)]))

plt.ylabel(str(y[((m) - 1)]) + ' in ' + str(z[((m) - 1)]))

plt.title(str(y[((n) - 1)]) + ' v/s ' + str(y[((m) - 1)]) + ' plot')

plt.savefig(str(y[(int(n) - 1)]) + ' vs ' + str(y[(int(m) - 1)]))

plt.show()

#engine_data

except:

print('File not found. Make sure the data file is in the same folder.Try again.')

#exit()

#Calculating the area under the P-V curve:

if n==(2) and m==(8):

#print(n,m)

print('\nCalculating the area under the P-V curve:')

k = trapz(x[((n) - 1)],x[((m) - 1)])

j = simps(x[((n) - 1)],x[((m) - 1)])

#plt.plot(x[(int(n) - 1)], x[(int(m) - 1)])

#plt.show()

#Units of the area under the curve will be product of the axes parameters = (1e6)N/m^2 * m^3 = MN-m which is 'work done per cycle

print('Area under the P-V curve or the Work done per cycle:')

print('Using Trapezoidal rule:' + str(k)+' in MN-m or '+str(float(k*1000000))+' in joules')

print('Using Simpson rule:' + str(j)+' in MN-m or '+str(float(j*1000000))+' in joules')

else:

print('Rerun for another attempt')

exit()

#Calculating output power:P = T*N/(2*60), where N - RPM, and T- work done per cycle

#Assuming at N=1500, and T as calculated above,

if n==(2) and m==(8):

N = int(input('\nEnter Crank rotation speed in RPMs:'))

T1 = k*1e06

#print(k)

P1 = (N*T1)/(2*60*1000) #assuming 4-stroke engine, 2 revolutions per cycle

print('Output Power is '+str(P1)+' in kWatts or '+str(float(P1*1000))+' in Watts')

T2 = j*1e06

#print(j)

P2 = (N*T2)/(2*60*1000)

print('Output Power is '+str(P2)+' in kWatts or '+str(float(P2*1000))+' in Watts')

#Brake Specific Fuel Consumption calculation: Fuel consumed in g/s per Output power produced in joules/s or kW-h

f = f + int(input('\nInput Fuel consumed in microgram/cycle:'))

f_actual = (int(f*N/120))*pow(1,-6) #grams/second = microg/cycle*[(RPM/2*60) or cycles/s] *10e-6

SFC1 = f_actual/P1 #g/kW-s

print(str(SFC1)+' in grams/kW-s')

print(str(float(SFC1/3600))+' in gram/kW-h') #g/kW-h

SFC2 = f_actual/P2

print('\n'+str(SFC2) + ' in grams/kW-s')

print(str(float(SFC2/3600))+' in gram/kW-h')

rocess:

1. First all the necessary modules and operations were imported to be called up later. Then all the required empty arrays were defined which are later to be filled with the column data.

2. As mentioned above, the compatibility of the file is examined using ‘try-except’ loop format with the ‘except’ loop being a promt printed letting the user know the possible error with the file name. The ‘try’ loop which runs if the file is compatibles, first stores the column name, number and units in empty arrays as ‘strings’ ,mentioned earlier using a nested for loop with a counter variable to instruct the line number for each of the parameters.

3. Now to actually store the thousands of lines of data, the empty arrays define earlier are used with a similar operation as the last step. An ‘if’ condition is used here instead of the line counter which omits any line with a ‘#’ in the start of the line making things easier.

4. By equating one such ‘column data’ array with the n^th element of the ‘number of column’ array[x], we can nest one array into another such that,

if a = ['s','f','g','e','r','t'] and b=[1, 2, 6, 5, 4, 8, 7] by doing a[0]=b, we are changing the ‘a’ array as:

a =[[1, 2, 6, 5, 4, 8, 7], 'f', 'g', 'e', 'r', 't'].
This way instead when calling the column data for plotting, instead of calling the column data array, we can call the number of column array and get the results.

5. By requesting the user to enter the column numbers for the required parameters using promt input commands, storing these inputs as integers in different variables we can produce the plots. Using these very parameters for the ‘name of column’ array[y] and ‘unit for the column parameter’ array[z], we can label the plots, title them and produce a unique file name as well while saving it.

6. For the performance calculations, as mentioned the algebraic formulas were used which are very straightforward.

7. For the area under the curve problem, we can integrate the ‘y’ function of the respective plot for ‘xmin’ to ‘xmax’ limits. To get the ‘y’ functions, we can perform curve fitting and make the problem even more complicated. Instead we can use ‘Composite Trapezoid rule’ or ‘Composite Simpson rule’ which can be accessed from ‘numpy’ and ‘scipy’ modules respectively. Both of these use composition of samples method such that, the area under the curve is broken down into intervals and the curve for all these intervals is approximated. ‘Trapezoidal rule’ uses a linear relation for approximation, defines the area under the curve as a trapezoid and calculates the area of the trapezoid. ‘Simpson rule’ uses a quadratic relation for approximation and even number of intervals forming equal subdivisions and calculation the area.

Conclusion:

1. This program can be used to parse any converged file not just the ones used for engine simulation as long as the format is the same. It is made sure that the column names are input from the file and not by the user to make this program compatible with almost any similar file with minor changes.
2. It is possible to add more of the performance calculations and further automate the analysis procedure where the user only has to input the variables such as in this case the ‘RPM’ figure and ‘fuel consumed’.
3. This program can also be used simply to visualize the overwhelming amount of data a simulation program produces in the ‘.out ’ file for a better understanding of it. A similar program can be used as a post-processor for such a simulation involving animations and further visualization tools.