Week 5 - Curve fitting

• This statistic measures the total deviation of the response values from the fit to the response values. It is also called the summed square of residuals and is usually labeled as SSE.

A value closer to 0 indicates that the model has a smaller random error component, and that the fit will be more useful for prediction.

R-square

This statistic measures how successful the fit is in explaining the variation of the data. Put another way, R-square is the square of the correlation between the response values and the predicted response values. It is also called the square of the multiple correlation coefficient and the coefficient of multiple determination.

R-square is defined as the ratio of the sum of squares of the regression (SSR) and the total sum of squares (SST). SSR is defined as

where SST = SSR + SSE. Given these definitions, R-square is expressed as

R-square can take on any value between 0 and 1, with a value closer to 1 indicating that a greater proportion of variance is accounted for by the model. For example, an R-square value of 0.8234 means that the fit explains 82.34% of the total variation in the data about the average.

If you increase the number of fitted coefficients in your model, R-square will increase although the fit may not improve in a practical sense. To avoid this situation, you should use the degrees of freedom adjusted R-square statistic described below.

Note that it is possible to get a negative R-square for equations that do not contain a constant term. Because R-square is defined as the proportion of variance explained by the fit, if the fit is actually worse than just fitting a horizontal line then R-square is negative. In this case, R-square cannot be interpreted as the square of a correlation. Such situations indicate that a constant term should be added to the model.

Degrees of freedom Adjusted R square

This statistic uses the R-square statistic defined above, and adjusts it based on the residual degrees of freedom. The residual degrees of freedom is defined as the number of response values n minus the number of fitted coefficients m estimated from the response values.

v indicates the number of independent pieces of information involving the n data points that are required to calculate the sum of squares. Note that if parameters are bounded and one or more of the estimates are at their bounds, then those estimates are regarded as fixed. The degrees of freedom is increased by the number of such parameters.

The adjusted R-square statistic is generally the best indicator of the fit quality when you compare two models that are nested — that is, a series of models each of which adds additional coefficients to the previous model.

The adjusted R-square statistic can take on any value less than or equal to 1, with a value closer to 1 indicating a better fit. Negative values can occur when the model contains terms that do not help to predict the response.

ROOR MEAN SQUARED ERROR

This statistic is also known as the fit standard error and the standard error of the regression. It is an estimate of the standard deviation of the random component in the data, and is defined as

where MSE is the mean square error or the residual mean square

Just as with SSE, an MSE value closer to 0 indicates a fit that is more useful for prediction.

CODE:

#CURVE FIT FOR LINEAR AND CUBIC POLYNOMIAL USING PYTHON

import matplotlib.pyplot as plt
import math
import numpy as np
from scipy.optimize import curve_fit

#curve fit function
def linear(t,a,b):
	return a*t + b

def cubic(t,a,b,c,d):
	return a*pow(t,3) + b*pow(t,2) + c*t + d

#reading thermodynamic data file
def read_file():
	temperature =[]
	cp = []
	for line in open('data','r'):
		values = line.split(',')
		temperature.append(float(values[0]))
		cp.append(float(values[1]))

	return [temperature, cp]
	

#main program for linear

temperature, cp = read_file()
#popt = coefficients
popt,pcov = curve_fit(linear,temperature,cp)
linear_fit_cp = linear(np.array(temperature), *popt)

#calculating error
error1 = (np.array(np.array(cp) - np.array(linear_fit_cp)))*100/(np.array(cp))
max_error1 = max(error1)
print('Maximum % error for curve fit using linear polynomial = ', max_error1)
print('\n')

plt.figure(1)
plt.plot(temperature, cp, color = "blue", linewidth =3)
plt.plot(temperature,linear_fit_cp,color = "red", linewidth =3)
plt.legend(['ACTUAL DATA', 'CURVE FIT'])
plt.xlabel('temperature(K)')
plt.ylabel('cp')
plt.title('Linear curve fit')
plt.savefig('Linear curve fit.png'), 
plt.show()

#main program for cubic fit
popt1,pcov1 = curve_fit(cubic,temperature,cp)
fit_cp1 = cubic(np.array(temperature), *popt1)

error2= (np.array(np.array(cp) - np.array(fit_cp1)))*100/(np.array(cp))
max_error2= max(error2)
print('Maximum % error for curve fit using linear polynomial = ', max_error2)
print('\n')

plt.figure(2)
plt.plot(temperature, cp, color = "blue", linewidth =3)
plt.plot(temperature,fit_cp1,color = "red", linewidth =3)
plt.legend(['ACTUAL DATA', 'CURVE FIT'])
plt.xlabel('temperature(K)')
plt.ylabel('cp')
plt.title('cubic curve fit')
plt.savefig('cubic curve fit.png')
plt.show()
	     

#to find best fit

import matplotlib.pyplot as plt
import math
import numpy as np
from scipy.optimize import curve_fit

#curve fit function
def linear(t,a,b):
	return a*t + b

def cubic(t,a,b,c,d):
	return a*pow(t,3) + b*pow(t,2) + c*t + d

#reading thermodynamic data file
def read_file():
	temperature =[]
	cp = []
	for line in open('data','r'):
		values = line.split(',')
		temperature.append(float(values[0]))
		cp.append(float(values[1]))

	return [temperature, cp]
	

#main program for linear

temperature, cp = read_file()
#popt = coefficients
popt,pcov = curve_fit(linear,temperature,cp)
linear_fit_cp = linear(np.array(temperature), *popt)

#number of independent variables
k = 0

#number of data points
n = len(temperature)

#calculating fitness charecteristics
print('SSE, R-SQUARE, ADJUSTED R SQUARE ADN RMSE VALUES FOR LINEAR POLYNOMIAL')
print('\n')

#sum of square of errors(SSE)
SSE = np.sum(pow((np.array(np.array(cp) - np.array(linear_fit_cp))), 2))
print('Value of sse is = ', SSE)
print('\n')

#SUM OF SQUARES OF REGRESSION(SSR)
SSR = np.sum(pow((np.array(np.array(linear_fit_cp) - np.mean(cp))), 2))

#TOTAL SUM OF SQUARES(sST)
SST = SSR + SSE

#R-SQUARE
R_SQUARE = SSR/SST
print('value of R-SQUARE is = ' , R_SQUARE)
print('\n')

#Adjusted R-SQUARE
Adjusted_R_Square = 1 - ((1- R_SQUARE)*(n -1)/(n - k -1))
print('value of ADJUSTED R-SQUARE is = ' , Adjusted_R_Square)
print('\n')

#ROOT MEAN SQUARE ERROR (RMSE)
RMSE = np.sqrt((np.sum(pow((np.array(np.array(cp) - np.array(linear_fit_cp))), 2)))/n)
print('value of RMSE is = ' , RMSE)
print('\n')

#**********************************************************************************************************************************
#main program for cubic fit
popt1,pcov1 = curve_fit(cubic,temperature,cp)
cubic_fit_cp1 = cubic(np.array(temperature), *popt1)

#calculating fitness charecteristics
print('SSE, R-SQUARE, ADJUSTED R SQUARE ADN RMSE VALUES FOR CUBIC POLYNOMIAL')
print('\n')

#sum of square of errors(SSE)
SSE_1 = np.sum(pow((np.array(np.array(cp) - np.array(cubic_fit_cp1))), 2))
print('Value of sse is = ', SSE_1)
print('\n')

#SUM OF SQUARES OF REGRESSION(SSR)
SSR_1 = np.sum(pow((np.array(np.array(cubic_fit_cp1) - np.mean(cp))), 2))

#TOTAL SUM OF SQUARES(sST)
SST_1 = SSR_1 + SSE_1

#R-SQUARE
R_SQUARE_1 = SSR_1/SST_1
print('value of R-SQUARE is = ' , R_SQUARE_1)
print('\n')

#Adjusted R-SQUARE
Adjusted_R_Square_1 = 1 - ((1- R_SQUARE_1)*(n -1)/(n - k -1))
print('value of ADJUSTED R-SQUARE is = ' , Adjusted_R_Square_1)
print('\n')

#ROOT MEAN SQUARE ERROR (RMSE)
RMSE_1 = np.sqrt((np.sum(pow((np.array(np.array(cp) - np.array(cubic_fit_cp1))), 2)))/n)
print('value of RMSE is = ' , RMSE_1)
print('\n')

Results

How to improve curve fit

CONCLUSION:

• By increasing the order of polynomial the error in curve fit can be reduced
• It is found that for order 9 polynomial the error in curve fit is 0.041 which is close to zero and can be considered as good fit.
• Splitwise method can also be used to get better curve fit.

Questions asked:

1.What does popt and pcov mean?

popt- An array of optimal values for the parameters which minimizes the sum of squares of residuals.

pcov-2d array which contains the estimated covariance of popt. The diagonals provide the variance of the parameter estimate.

2. What does np.array(temperature) do?

The command np.array(temperature) is used for accesing the individual elements of the temperature array with the help of numpy module. Numpy module is imported as np and is used to perform mathematical operations like addition, subtraction,multiplication, division etc.

3.What does the * in *popt mean?

The *popt unpacks the popt array so that the optimized coefficients values can be accessed from the array popt