Data Parsing using Python

Aim:

To parse the given data file and to automate the following tasks using Python:

(1) Get column numbers as inputs from user and plot the two columns with labels and save the file by the column name

(2) Calculate Area under P-V diagram, Power output and SFC of the engine using the given data file.

(3) Check the compatibility of the program

Introduction:

Parsing a data means reading in a data stream of some sort and building an in memory model of the semantic content of that data. This helps in many ways such as to automate specific tasks and also to arrive at the relation between various entries of the data by extracting information from a file.

In this challenge, the given file has the data such as Pressure, Volume, Mass, Density, Crank angle, Temperature, Heat Release rate, Specific heat and viscosity values of a port fuel injected IC engine.

Python Program:

"""
Data Parsing using Python
By Sree Vignesh
"""

#Import all required modules
import math
import numpy as np
import matplotlib.pyplot as plt

#To check validity of the filename (Compatability Check)
filename = input('Enter a file name with ".out": ')

if 'engine_data.out' in filename:
	data = open(filename, 'r')

	#Basic Initialization
	Crank = []
	Pressure  = []
	Max_pressure  = []
	Min_pressure = []
	Mean_Temperature = []
	Max_Temperature = []
	Min_Temperature = []
	Volume = []
	Mass = []
	Density = []
	Integrated_HR = []
	HR_Rate = []
	C_p = []
	C_v = []
	Gamma = []
	Kinematic_Viscosity = []
	Dynamic_Viscosity = []

	#Data Parsing starts here
	line_count = 1
	for line in data:
		if '#' not in line:
			Crank.append(float(line.split()[0]))
			Pressure.append(float(line.split()[1]))
			Max_pressure.append(float(line.split()[2]))
			Min_pressure.append(float(line.split()[3]))
			Mean_Temperature.append(float(line.split()[4]))
			Max_Temperature.append(float(line.split()[5]))
			Min_Temperature.append(float(line.split()[6]))
			Volume.append(float(line.split()[7]))
			Mass.append(float(line.split()[8]))
			Density.append(float(line.split()[9]))
			Integrated_HR.append(float(line.split()[10]))
			HR_Rate.append(float(line.split()[11]))
			C_p.append(float(line.split()[12]))
			C_v.append(float(line.split()[13]))
			Gamma.append(float(line.split()[14]))
			Kinematic_Viscosity.append(float(line.split()[15]))
			Dynamic_Viscosity.append(float(line.split()[16]))

			line_count = line_count+1

	#Creating lists for plotting
	Parameters = np.array([Crank,Pressure,Max_pressure,Min_pressure,Mean_Temperature,Max_Temperature,Min_Temperature,Volume,Mass,Density,Integrated_HR,HR_Rate,C_p,C_v,Gamma,Kinematic_Viscosity,Dynamic_Viscosity])
	Labels = ['Crank (Degree)','Pressure (MPa)','Maximum pressure (MPa)','Minimum pressure (MPa)','Mean temperature (K)','Maximum temperature (K)','Minimum temperature (K)','Volume (m^3)','Mass (kg)','Density (kg/m^3)','Integrated Heat release (J)','Heat release rate (J/time)','C_p (J/kg-K)','C_v (J/kg-K)','Gamma','Kinematic viscosity (m^2/s)','Dynamic viscosity (N*s/m^2)']

	#Displaying the list of parameters
	i = 1
	for i in range(1,18):
		if i<10:
			print(i,' :',Labels[i-1])

		if i>=10:
			print(i,':',Labels[i-1])

	#Receiving inputs
	x_axis = int(input("\nEnter x-axis parameter:"))
	y_axis = int(input("Enter y-axis parameter:"))
	print("\nX-axis is",Labels[x_axis-1])
	print("Y-axis is",Labels[y_axis-1])

	#Plotting and saving 
	plt.plot(Parameters[x_axis-1],Parameters[y_axis-1])
	plt.xlabel(Labels[x_axis-1])
	plt.ylabel(Labels[y_axis-1])

	title = Labels[y_axis-1]+' '+'vs'+' '+Labels[x_axis-1]

	plt.title(title)
	plt.savefig(title+' '+'.png')

	#Calculating Area under the P-V diagram
	work_done = np.trapz(np.array(Pressure)*pow(10,6),np.array(Volume))
	print('\nWork done =',work_done,'J')

	"""
	Calculating Power output of this engine
	Power = (2*pi*N*T)/60000 kW
	where, N is Engine speed and T is Torque = work done
	Assuming:
	N = 1500 rpm
	"""
	N = 1500 

	power_output = (2*math.pi*N*work_done)/60000
	print('\nPower output of the engine =',power_output,'kW')

	"""
	Calculating specific fuel consumption
	Assuming:
	20 micro grams of fuel consumed per 1 stroke cycle
	"""
	fuel = 20*10e-6*3600
	sfc = fuel/(power_output)
	print('\nSpecific fuel consumption =',sfc,'g/kW-hr')

	#Display Plot
	plt.show()

else:
	print('\nFile not recognized. Please provide a valid CONVERGE output file\n')

Explanation:

The title of the program is entered using comments and all the required modules are imported at first.

To check compatability , a variable called filename is used to store the input from the user. Using if else statement, if the correct file is entered, a variable called data holds the data of the file and if not appropriate statement is displayed.

To hold the values of each variable in the data file provided, empty arrays of the same number of variables as in the file are defined with respective names. A variable line_count is used to have a count on the number of lines and it gets incremented everytime when for loop is executed.

Using for loop and if loop the lines of the file are read one by one, each time when loop gets executed. The values are assigned to each of the empty arrays using append() command. Note that if the line has # then it is a header and so, only those lines which do not have # are taken into consideration. line.split() command is used since each column holds values of a particular variable and hence only respective values of the variables are assigned to the arrays.

There are a total of 18 variables and hence there are 18 different variables defined as arrays. Note that the array index starts from 0 and hence the index goes till 17 while defining.

In order to display a list of variables and labels, two arrays Parameters and Labels are defined. The size of both is same. Using for loop, the variables are printed out along with a number and their labels. For neat visualization, I have made use of a space before ':' for single digit numbers.

After this list prints out, the user is prompted to choose 2 parameters, one for X-axis and the other for Y-axis. Then, choices made by the user gets printed out. Based on this, a plot is created. This plot takes appropriate labels based on user inputs automatically. Title is also specified accordingly. Plot is saved with the title and in PNG format. At the end, plt.show() is used to display the plot, once all computations get over.

To calculate Area under the P-V curve, np.trapz() function is used for integration. Hence the work done in J is printed after computation

For calculating the Power output, the conventional formula is used and engine speed is assumed as 1500 rpm. After calculation, the power output is also printed out and bears the unit kW.

Given that, the engine consumes 20 micro grams of fuel per stroke cycle, SFC is calculated in such a way to arrive at g/kW-hr unit. SFC is the total fuel consumed/Power output . After this computation, the SFC is printed out in g/kW-hr.

Result:

(1) Taking column numbers as input and generate plots:

(a)

As we can see that, all the variables of the file are displayed in a way to show the choices to the user, when correct file name is entered at the first prompt.

After this, the user is again prompted to enter the number that matches his/her desired parameter which is the column number of the data file.

The first input is for X-axis and the second one for Y-axis. After this, the program automatically displays the choice made by the user.

Now, the following plot appears:

Since I had given 1 and 2 as the column number inputs, a plot between Pressure and Crank is generated.

(b)

If I give 8 and 2 as inputs, then a plot between Pressure and Volume appears as follows, which is the PV diagram for which Area under the curve has to be calculated:

(c)

Once those two plots were generated, they were automatically saved in the folder which can be seen in the above screenshot.

(2) Performance calculations:

As explained in the coding part, the performance calculations were made using conventional formulae, and the results which appeared are shown in the above screenshot.

The area under PV diagram is the work done in J . The power output is in kW and SFC is in g/kW-hr

(3) Compatability check:

Here, the correct file name was not entered and hence the program exits saying the statement, 'File not recognized. Please provide a valid CONVERGE output file'.

If the correct file is given at the prompt, then the program gets executed and all the results are displayed as explained above.

Note: All these were automatically executed and REPL environment is used to view the output from the program.

Conclusion:

A Python script to parse the given data file was coded and a detailed explanation to the program was given. Also, the results displayed by the program were attached as images and explanation was given for the same.