Data Analysis Automation using Python

INTRODUCTION:

Engineering work would most often involve enormous data that would be generated. Data would not make sense if they are not processed and and are presented in format that could be comprehended such as graphs, charts and tables. But, most often this would involve managing large data bases and arranging them in order which is highly repetitive and taxing.

Hence Automation scripts are developed to replace mundane repetitive tasks, to process data and present data in the format that could be understood.

In this code, CFD data from CONVERGE is processed. It is assumed that User has seen the data file and is running the script after reading the data file.

User interface is built to input the column number whose data is requried, once this input is receieved that column of data is stored and graph is plotted. Similarly, exception handling is built in and to smoothly exit in case of non compatible file fed to the script. If user inputs column number greater than what is in the data file, prompts are built to alert the user

Similarly, area under PV diagram is inherently calculated without any user prompt. Work, Power and Fuel consumption are calculated. This script could also be used to calculate wiebe parameters, heat release curve with minor code modifications

Python Script:

# -*- coding: utf-8 -*-
"""
Created on Mon Jul 13 16:37:34 2020

@author: sweekar
This is coded by sweekar, please reach out to him at [email protected]
This code is used to take user inputs and plot the data user requests. This is also used to calculate
engine performance from indicator diagram such as work done, power, ISFC, Indicated efficency
"""
import matplotlib.pyplot as plt
from scipy.integrate import simps
import math

#Exception handling
try:
    f = open('engine_data.out','r')
except:
    print('File not recognised, please provide valid CONVERGE output file')
    exit()

#Intialisation
line_count = 1
name_count = 1
crank = []
pressure = []
temperature = []
volume = []
x = []
y = []
label = []

#User Inputs
print('........................................DATA ANALYSER..................................')
print('This is used to analyse the data from the file named as engine_data.out')
print('Please enter the index of columns accurately to read data from that column and plot')
ncols1 = input('Enter the number of columns in the data file counting from 1: ')
m1 = input('Enter the index of data required on x axis counting from 1: ')
n1 = input('Enter the index of data required on y axis counting from 1: ')
ncols = int(ncols1)-1
m = int(m1)-1
n = int(n1)-1

#Main
for line in open('engine_data.out'):   #line could be substituted with data,line is just a variable
    if '#' not in line:
       crank.append(float(line.split()[0]))
       pressure.append(float(line.split()[1]))
       temperature.append(float(line.split()[4]))
       volume.append(float(line.split()[7]))
       if m>=0 and m<=ncols:
           if n>=0 and n<=ncols:
               x.append(float(line.split()[m]))
               y.append(float(line.split()[n]))
       else:
            print('Index is outside the maximum number of columns specified in data file')
            print('please retry with correct values')
            break
       line_count = line_count + 1
       
    else:
        if name_count==1: 
           name_count = name_count+1
        elif name_count==2:
            label.append(line.split())
            

"""
plt.figure(1)
plt.plot(crank,pressure,color='red')
plt.xlabel('Crank angle degrees(deg)')
plt.ylabel('Pressure')
plt.title('Pressure trace')
plt.figure(2)
plt.plot(crank,temperature,color='blue')
plt.xlabel('Crank angle degrees(deg)')
plt.ylabel('Temperature[K]')
plt.title('Temperature profile')
"""
#Plotting
if m>=0 and m<=ncols:
           if n>=0 and n<=ncols:
               plt.figure(3)
               plt.plot(x,y,color='black')
               plt.xlabel('%s %s'%(label[1][m+1],label[2][m+1]))
               plt.ylabel('%s %s'%(label[1][n+1],label[2][n+1]))
               plt.title('%s vs %s'%(label[1][m+1],label[1][n+1]))
               plt.savefig('%s vs %s'%(label[1][m+1],label[1][n+1]))

#Pressure Trace analysis
work = (simps(pressure,volume))*pow(10,3)  #Area under PV diagram is found by Numerical integration using Simpsons rule
print('The work done which is area under PV diagram or indicator diagram is %f kJ'%work)
N = 1500 #RPM
Power = 2*math.pi*N*work/60
print('The power generated is %f kW'%Power)
Fuel = 20*pow(10,-6) #grams per stroke
#Assuming 4 stroke engine
Fuel_ghr = Fuel*N*2*60 #g/hr
ISFC = Fuel_ghr/Power #g/kW-hr
print('The fuel consumed per hour is %f kg/hr'%(Fuel_ghr/1000))
print('ISFC for the engine is %f g/kW-hr'%ISFC)

RESULTS

For the user inputs of 17,1,5 following plot is obtained automatically

This plot is automatically stored in the working directory and labelled without user intervention

Similarly, output of the command window is as follows

DISCUSSION ON RESULTS
1) Exception handling is requried for smooth handling of the error

2) Automation reduces the manual efforts of painstakingly going through bunch of data, couple of lines could result in plotting required data without much fuss

3) With the engine data used, ISFC seems too good to be true