CURVE FITTING USING PYTHON
OBJECTIVE
To draw a curve that describes the variation of specific heat with the temperature at a constant pressure.
INTRODUCTION
Curve fitting is the process of constructing a curve or function that has the best fit to a series of data points. Curve fitting can involve either interpolation, where an exact fit to the data is required or smoothing in which a \"smooth\" function is constructed that approximately fits the data. Fitted curves can be used as an aid for data visualization, to infer values of a function where data is not available and to summarize the relationships among two or more variables. Extrapolation refers to the use of a fitted curve beyond the range of the observed data. This process can be utilised to find the specific heat of non-perfect gases at temperatures where specific heat is unknown. This method is subject to a degree of uncertainty since it may reflect the method used to construct the curve as much as it reflects the observed data. Hence we find the best-fit curve that describes the relationship between specific heat and the temperature by measuring the fitness characteristics of the curves.
THEORY
INTERPOLATION:
In this method, we plot the data and connect them. It is a piece-wise linear interpolation. Since it is a group of small f(x)s, connecting one point to the next will not work for data that has a built-in random error.
CURVE FITTING:
Here we capture the trend in the data by assigning a single function (polynomial equation) across the entire range.
The general form of a polynomial equation nth degree:
`y=f(x)=a_0+a_1x+a_2x^2+....+a_(n)x^(n)`
`y=f(x)=a_0+sum_(m=1)^na_m x^m`
Difference between the actual value and the value predicted by the function is the error. To ensure positive and negative errors value the same and to amplify the larger errors, we square these errors.
`\"Error\"=\"Actual Value\"-\"Predicted Value\"=y_i-f(x_i)`
`(\"Error\")^2=(y_i-f(x_i))^2`
`(\"Total Error\")^2=sum_(i=1)^k(y_i-f(x_i))^2\" \" \"where k=numer of data points i= the current data point being summed\"`
`\"Total Error\"^2=sum_(i=1)^k(y_i-(a_0+sum_(m=1)^na_m x_i^m))^2`
The best curve is the one with the least total error thus coefficients of the polynomial equations are such that the sum of squares of the errors is as small as possible.
CODE TO GENERATE THE CURVE
import numpy as np
from scipy.optimize import curve_fit
import matplotlib.pyplot as plt
import math
def func(t,a,b):
return a*t+b
def func2(t,a,b,c,d):
return a*pow(t,3)+b*pow(t,2)+c*t+d
def func4(t,a,b,c,d,e):
return a*pow(t,4)+b*pow(t,3)+c*pow(t,2)+d*t+e
def func5(t,a,b,c,d,e,f):
return a*pow(t,5)+b*pow(t,4)+c*pow(t,3)+d*pow(t,2)+e*t+f
def read_file():
temperature=[]
cp=[]
for line in open(\'data\',\'r\'):
values=line.split(\',\')
temperature.append(float(values[0]))
cp.append(float(values[1]))
return[temperature,cp]
def evaluation(fit_cp,popt):
error=np.array(cp)-np.array(fit_cp)
error_sq=error*error
rss=sum(error_sq)
y_mean=sum(cp)/len(cp)
ss=np.array(cp)-y_mean
ss_sq=ss*ss
tss=sum(ss_sq)
fvu=rss/tss
r_sqr=1-fvu
r_bar=1-(1-r_sqr)*((len(cp)-1)/(len(cp)-len(popt)-1))
mse=rss/len(cp)
rmse=pow(mse,0.5)
print(\"{:2d}\\t{:21.6f}\\t{:12.4f}\\t{:12.4f}\\t{:19.4f}\".format(len(popt)-1,rss,r_sqr,r_bar,rmse))
temperature,cp=read_file()
popt,pcov=curve_fit(func,temperature,cp)
popt2,pcov2=curve_fit(func2,temperature,cp)
popt4,pcov4=curve_fit(func4,temperature,cp)
popt5,pcov5=curve_fit(func5,temperature,cp)
print(\"Order \\t Least Square Error \\t R-square \\t Adjuste R-square \\t Root Mean Square Error\")
fit_cp=func(np.array(temperature),*popt)
evaluation(fit_cp,popt)
fit_cp2=func2(np.array(temperature),*popt2)
evaluation(fit_cp2,popt2)
fit_cp4=func4(np.array(temperature),*popt4)
evaluation(fit_cp4,popt4)
fit_cp5=func5(np.array(temperature),*popt5)
evaluation(fit_cp5,popt5)
plt.figure()
plt.plot(temperature,cp,label=\'Actual data\')
plt.plot(temperature,fit_cp,label=\'1st order fit\')
plt.plot(temperature,fit_cp2,label=\'3rd order fit\')
plt.legend(loc=\'upper left\')
plt.show()- We open the file in \'read\' mode, that contains the data of variables between which a relationship is to be defined using \'read_file\' function. This file should be present in the python root folder.
- Values of variables are obtained from the file and stored in the respective array corresponding to each variable. In the example, the first and the second column of the \'newdata\' file corresponds to the values of temperature and cp respectively separated by a comma. We split them and store them in the respective arrays.
- \'Curvefit\' function gives the coefficients in the form of an array and variance matrix of the coefficients for the polynomial function. Input order should be the function name of the polynomial, the independent variable followed by the dependent variable at the end.
- We use \'popt\', the coefficient array to predict the value of the dependent variable corresponding to the value of the independent variable at each point. We pass temperature using np.array. The advantage of using np.array is that the list containing temperature is converted into an array, which is passed and operated rather than passing individual elements using a loop. \'*\' is used before \'popt\' to singularize the elements of the array and pass it to the function. This is used to avoid passing each element individually to the function like (popt[0],popt[1])
- Finally, we plot the actual and the best fit curve.
ASSESSING THE CURVE FITNESS
There are two ways of measuring the fitness of a curve:
Perfect-fit: A curve is said to be a perfect-fit if it passes through most of the data points.
Best-fit: The best-fit curve is the one with the least sum of the square of errors.
A best-fit curve results in predicted values close to the observed data values. The mean model, which uses the mean for every predicted value, generally would be used if there were no informative predictor variables. The fit of a proposed curve should be better than the fit of the mean model.
Methods used in Ordinary Least Squares (OLS) regression to evaluate the best curve fitness are R-squared and the Root Mean Square Error (RMSE) based on the Total Sum of Squares(TSS) and Residual Sum of Squares(RSS). TSS measures how far the data are from the mean and RSS measures how far the data are from the model’s predicted values. Different combinations of these two values provide information about how the regression model compares to the mean model.
Total Sum of Squares(TSS):
It is the sum of the squared differences of each observation from the overall mean. It measures how much variation there is in the observed data.
`\"TSS\"=sum(\"Actual Value\"-\"Mean Value\")`
`\"TSS\"=sum_(i=1)^n(y_i-bary)^2`
`bary=1/nsum_(i=1)^ny_i`
Residual Sum of Squares(RSS):
It is the sum of the squares of residuals (deviations predicted from actual empirical values of data). It is a measure of the discrepancy between the data and an estimation model.
`\"RSS\"=sum(\"Actual Value\"-\"Predicted Value\")`
`\"RSS\"=sum_(i=1)^n(y_i-f(x_i))^2`
Fraction of Variance Unexplained(FVU):
The Fraction of Variance Unexplained(FVU) in the context of regression task is the fraction of variance of the dependent variable which cannot be explained, that is which is not correctly predicted, by the explanatory variable. Its value ranges from 0 to 1.
`\"FVU\"=\"RSS\"/\"TSS\"`
Co-efficient of Determination or R-squared:
The coefficient of determination or \"R squared\", is the proportion of the variance in the dependent variable that is predictable from the independent variable.
`\"R\"^2=1-\"FVU\"`
R-squared is a statistic that will give some information about the goodness of fit of a model. In regression, the R-squared coefficient of determination is a statistical measure of how well the regression predictions approximate the real data points. R-squared has the useful property that its scale is intuitive: it ranges from zero to one, with zero indicating that the proposed model does not improve prediction over the mean model, and one indicating perfect prediction. Improvement in the regression model results in proportional increases in R-squared. One pitfall of R-squared is that it can only increase as predictors are added to the regression model. This increase is artificial when predictors are not actually improving the model’s fit. This leads to the alternative approach of looking at the Adjusted R-squared.
Adjusted R-squared
The use of an adjusted R-squared is an attempt to take account of the phenomenon of the R-squared automatically and spuriously increasing when extra explanatory variables are added to the model. Unlike R-squared, the adjusted R-squared increases only when the increase in R-squared (due to the inclusion of a new explanatory variable) is more than one would expect to see by chance. If a set of explanatory variables with a predetermined hierarchy of importance are introduced into a regression one at a time, with the adjusted R-squared computed each time, the level at which adjusted R-squared reaches a maximum, and decreases afterward, would be the regression with the ideal combination of having the best fit without excess/unnecessary terms.
`bar\"R\"^2=1-(1-\"R\"^2)(n-1)/(n-p-1)`
n=number of data points
p=total number of explanatory variables in the model (not including the constant term).
Root Mean Square Error(RMSE):
The RMSE is the square root of the variance of the residuals. It indicates the absolute fit of the model to the data–how close the observed data points are to the model’s predicted values. Whereas R-squared is a relative measure of fit, RMSE is an absolute measure of fit. As the square root of the variance, RMSE can be interpreted as the standard deviation of the unexplained variance and has the useful property of being in the same units as the response variable. Lower values of RMSE indicate better fit. RMSE is a good measure of how accurately the model predicts the response, and it is the most important criterion for fit if the main purpose of the model is the prediction.
`\"RMSE\"=sqrt((y_i-f(x_i))^2/n`
VARIANCE MATRIX
The diagonal entries are the variance of the regression coefficients and the off-diagonals are the covariance between the corresponding regression coefficients. Variance is the expectation of the squared deviation of a random variable from its mean. If the covariance between estimated coefficients is high, then in any sample if one coefficient is high, you can also expect the other coefficient to be high. In a more Bayesian sense, one coefficient contains information about the other. We will get estimates of the correlations between the regression coefficients.
`\"Variance matrix\"=sigma^2(XX\')^-1`
`sigma=\"Standard Deviation\"=(sum_i^n(y_i-f(x_i))^2)/d\" ,\" \"d= degree of freedom=number of observations minus the number of coefficients\"`
`\"X=Design Matrix\"`
RESULTS AND CONCLUSION
We can observe from the graph and table that the cubic curve is better compared to the linear curve in terms of fitness. Also, we can notice that as we increase the degree of polynomial function adjusted R-square is increasing and RMSE is decreasing signifying an improvement in fitness.